Customer side risk analysis method based on voice recognition

Abstract

The invention discloses a client-side risk analysis method based on voice recognition, and the method comprises the steps: carrying out the mixed sampling through obtaining the receiving data, voice interaction background information and voice signals of an equipment-side sensor, and generating a multi-dimensional feature vector; based on a support vector machine model, classifying the multi-dimensional feature vectors by taking historical scene data as training data to obtain scene categories and scene confidence; meanwhile, acoustic characteristics and historical voice behaviors of the voice signals are extracted, voice risk factors are constructed, a total risk score is comprehensively calculated, and risk score judgment is carried out through a dynamic threshold function; according to a judgment result, selecting to check a voice risk factor or regenerate a multi-dimensional feature vector so as to realize accurate analysis of the customer side risk; through feature fusion and classification optimization, the accuracy and real-time performance of risk analysis are effectively improved, and the method is suitable for customer risk management in a complex voice interaction scene.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing and risk analysis, and in particular to a client-side risk analysis method based on speech recognition. Background Art

[0002] Speech recognition technology has made significant progress in recent years and is widely used in intelligent voice assistants, customer service, financial security and other fields. Through feature extraction, pattern matching and training of machine learning models, modern speech recognition systems can convert speech signals into text with high accuracy. However, with the complexity of application scenarios, the challenges of speech recognition are gradually emerging. In multimodal interaction scenarios, problems such as environmental noise, unclear speaker identity, and diversity of speech content have put forward higher requirements for speech recognition. In addition, traditional speech recognition technology pays more attention to the transcription of speech content, but ignores the behavioral intentions and risk information that may be contained in the speech signal, which brings security risks to industries that rely on voice interaction (such as finance, customer service, etc.). In this context, studying how to combine speech recognition with risk analysis and make full use of the acoustic features, semantic information and voiceprint features in speech signals has become an important direction of current technological development.

[0003] But for now, the current technology still has some obvious shortcomings in combining speech recognition and risk analysis. First, traditional speech recognition technology mainly focuses on the accuracy of speech-to-text conversion, but lacks in-depth analysis of the acoustic characteristics of speech signals, which makes it unable to effectively distinguish between normal speech behavior and potential risky behavior; second, existing risk analysis systems usually rely on rule matching or static features, which are very limited when processing multi-scenario and multi-dimensional voice data. In addition, the existing technology is relatively weak in feature fusion and classification of dynamic data, especially when faced with complex environmental signals or identity verification requirements, it is difficult to provide accurate and real-time risk judgments. These shortcomings limit the further development of speech recognition technology in the field of customer-side security management. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a client-side risk analysis method based on speech recognition to solve the problems raised in the background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a customer-side risk analysis method based on speech recognition, comprising:

[0007] Acquire the received data of the device-side sensor and the background information and voice signal during the voice interaction, perform mixed sampling on the received data, and splice the mixed sampled received data with the background information during the voice interaction to obtain a multi-dimensional feature vector;

[0008] Based on a support vector machine model, the historical scene data is used as training data of the support vector machine model to classify the multidimensional feature vector to obtain a scene category and a scene confidence;

[0009] Using a speech recognition method, extracting speech content and acoustic features from the speech signal, and combining historical speech behavior to construct a speech risk factor;

[0010] According to the scenario category, scenario confidence and speech risk factor, a total risk score is obtained. By designing a dynamic threshold function, the total risk score is judged. According to the judgment result, it is chosen to check the speech risk factor or regenerate the multidimensional feature vector, thereby realizing a more accurate and effective client-side real-time speech risk analysis method.

[0011] As a preferred solution of the client-side risk analysis method based on speech recognition of the present invention, wherein: performing mixed sampling on the received data includes:

[0012] The received data of the device-side sensor is sampled in a mixed manner of coarse-grained and fine-grained data, and the coarse-grained and fine-grained data are aligned respectively according to the time period information of the received data in the device-side sensor as the timestamp.

[0013] As a preferred solution of the client-side risk analysis method based on speech recognition according to the present invention, the alignment process includes:

[0014] For coarse-grained data, the coarse-grained data is interpolated to the fine-grained time axis. For fine-grained data, the fine-grained data is downsampled and aligned to the coarse-grained time axis.

[0015] As a preferred solution of the client-side risk analysis method based on voice recognition described in the present invention, the received data after mixed sampling is spliced ​​with the background information during the voice interaction to obtain a multi-dimensional feature vector, including:

[0016] The splicing method adopts horizontal sequential splicing to obtain a multi-dimensional feature vector, and each spliced ​​information in the multi-dimensional feature vector is combined with another spliced ​​information to obtain multiple information pairs, namely scene factors.

[0017] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, the historical scene data is used as the training data of the support vector machine model to classify the multi-dimensional feature vector, including:

[0018] Taking historical scene data as a training sample set, and assigning a scene label to each training sample in the training sample set;

[0019] Each scene factor in the multidimensional feature vector and the corresponding scene label are used as training sample pairs. N-1 training sample pairs are randomly selected each time, and only one set of training sample pairs is left as verification sample pairs. The verification sample pairs are used to compare the training sample pairs.

[0020] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, it also includes:

[0021] In the comparison process, the penalty coefficient C is used to control the comparison priority until the final verification sample is better than the training sample pair, and the optimal classifier is obtained.

[0022] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, wherein: obtaining the scene confidence and scene category includes:

[0023] The input multidimensional feature vector is calculated by the optimal classifier to obtain a decision function of a support vector machine model, and the decision function of the support vector machine model is converted by a Sigmoid function to obtain a scene confidence;

[0024] Among them, the output result of the decision function of the support vector machine model is the scene category.

[0025] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, wherein: extracting the speech content and acoustic features in the speech signal, and combining historical speech behavior to construct a speech risk factor, including:

[0026] Convert continuous speech signals into digital signals, extract acoustic features from speech signals through the first-order and second-order difference coefficients of Mel cepstral coefficients, and output text sequences using language models;

[0027] The number of occurrences and sources of words in the text sequence are recorded, and weight values ​​are assigned to the corresponding words to generate speech risk factors at different risk levels.

[0028] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, the total risk score is obtained according to the scene category, scene confidence and speech risk factor, including:

[0029] The total risk score formula is expressed as:

[0030]

[0031] Among them, θ represents the overall weight of the voice risk factor, v j It is represented as the jth speech risk factor, k is the total number of speech risk factors, and θ2 is represented as the overall weight of the scenario risk E(t);

[0032] The scenario risk E(t) is expressed as:

[0033]

[0034] Among them, f(d) represents the mapping function of the scene confidence d, and increases with the increase of the output scene category Y, s i (t) represents the remaining time factor affecting the scenario risk.

[0035] As a preferred solution of the client-side risk analysis method based on speech recognition described in the present invention, wherein: by designing a dynamic threshold function, the total risk score is judged, and according to the judgment result, the speech risk factor is checked or the multi-dimensional feature vector is regenerated, including:

[0036] The dynamic threshold function T(t) is expressed as:

[0037] T(t)=T0+E(t)

[0038] Among them, T0 is used as the basis to set the threshold;

[0039] The total risk score is judged according to the dynamic threshold function. When the total risk score is greater than the dynamic threshold function, it is judged as a misjudgment and the speech risk factor is checked. Otherwise, when the total risk score is less than the dynamic threshold function, it is judged as an abnormality and the multidimensional feature vector is regenerated.

[0040] Compared with the prior art, the invention has the following beneficial effects:

[0041] 1. The present invention constructs a multidimensional feature vector by comprehensively processing the acoustic features and environmental background data in the speech signal, and realizes accurate classification based on the support vector machine (SVM) model; compared with the classification method of a single feature in the prior art, the scheme of the present invention effectively improves the accuracy of feature fusion through mixed sampling, interpolation completion and downsampling alignment of coarse-grained and fine-grained data, making the classification result more in line with the actual scene;

[0042] 2. Based on the decision function value of SVM, the confidence is obtained through the Sigmoid function and plays a role in environmental factors to form a dual characterization of scene category + confidence, so that the model can flexibly adjust the scene risk according to the depth of classification;

[0043] 3. By considering the voiceprint module, the identity of the speaker in the voice interaction can be accurately verified, and the verification result can be quantified as a voiceprint matching factor for the overall risk score. In addition, combined with voice content analysis and scene classification, it can effectively identify unauthorized behaviors such as impersonation and fraud, further improving the security of voice interaction on the customer side. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0045] Figure 1 This is an overall flow chart of a method for analyzing client-side risks based on speech recognition according to an embodiment of the present invention;

[0046] Figure 2 A performance radar chart comparison diagram of a client-side risk analysis method based on speech recognition according to an embodiment of the present invention;

[0047] Figure 3 This is a dynamic change analysis diagram of the total risk score of the client-side risk analysis method based on voice recognition described in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0052] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0053] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] Example 1

[0055] Reference Figure 1 , which is the first embodiment of the present invention, and provides a method for analyzing client-side risks based on speech recognition, comprising:

[0056] S1. Obtaining the received data of the device-side sensor and the background information and voice signal during the voice interaction, performing mixed sampling on the received data, and splicing the mixed sampled received data with the background information during the voice interaction to obtain a multi-dimensional feature vector;

[0057] Specifically, the data received by the device-side sensor includes, but is not limited to, the ambient volume (the intensity of the sound in the current environment around the customer), the network status (the signal strength of the Wi-Fi or mobile network), the geographic location (GPS), and the time period information (the current system time of the device);

[0058] Specifically, the background information during voice interaction includes, but is not limited to, the noise type (such as traffic noise, crowd noise, wind noise, etc.), reverberation time (the degree of reverberation of the environment, used to judge the indoor and outdoor conditions), and the direction of the sound source (the angle of the sound source relative to the device, used to judge whether the interactor is facing the device);

[0059] It should be explained that since the dynamic range and change frequency of different types of data information collected are different, in order to improve the accuracy of feature fusion, it is necessary to process these data information uniformly;

[0060] Furthermore, the received data of the device-side sensor is sampled in a mixed manner of coarse-grained and fine-grained, and the coarse-grained and fine-grained data are aligned respectively according to the time period information of the received data in the device-side sensor as a timestamp;

[0061] Specifically, the alignment processing method is, for coarse-grained data, the coarse-grained data is interpolated and completed to the fine-grained time axis, and for fine-grained data, the fine-grained data is downsampled and aligned to the coarse-grained time axis;

[0062] It should be noted that interpolation completion (such as linear interpolation, nearest neighbor interpolation, etc.) is to calculate the coarse-grained data values ​​corresponding to the time points of the fine-grained time axis according to the coarse-grained data, while downsampling alignment is to simplify the fine-grained data into data points on the coarse-grained time axis, and compress multiple fine-grained data points into a coarse-grained value through aggregation methods (such as mean, maximum, minimum, etc.);

[0063] For example, interpolation completion takes linear interpolation as an example, assuming that the fine-grained time axis is: T fine =[t1,t2,…,t n ], n is the number of fine-grained time points, and the coarse-grained time point is: T coarse =[t a ,t b ,…,t m ], m is the number of coarse-grained time points; for each t n ∈T fine , can be found in T coarse The corresponding interval range [t a ,t b ], where t a ≤t n ≤t b , for each t n , through the linear interpolation formula, we get:

[0064]

[0065] Among them, v(t a ) and v(t b) represent the coarse-grained time points at t a and t b The value of v(t n ) is expressed as the fine-grained time point t n The interpolation result on ;

[0066] For example, the downsampling alignment takes mean downsampling as an example, assuming that the coarse-grained time axis is: T coarse =[t a ,t b ,…,t m ], the fine-grained time point is: T fine =[t1,t2,…,t n ], for the coarse-grained time point t m , the corresponding fine-grained time point range needs to be defined as And meet By taking fine-grained data in the range The mean value within is:

[0067]

[0068] Among them, k1 represents the coarse-grained time point t m The starting index of k2 represents the coarse-grained time point t m The end index of v(t m ) is expressed as the coarse-grained time point t m The downsampling result of

[0069] Furthermore, the mixed sampled received data is spliced ​​with the background information during voice interaction to obtain a multidimensional feature vector, wherein the splicing method adopts horizontal sequential splicing. For example, [x, y, z] represents the received data of the ambient volume, network status, and longitude and latitude, respectively, and [c, d] represents the noise type and reverberation time under the background information during voice interaction, respectively, and the spliced ​​multidimensional feature vector X is obtained. c Expressed as: X c =[x,y,z,c,d];

[0070] It should be noted that by combining each piece of spliced ​​information with another piece of spliced ​​information, a scene factor can be obtained. For example, (ambient volume, noise type) = (high, traffic sound), it can be judged that the customer may be on a noisy street, (network status, sound source direction) = (weak, lateral), it can be judged that the customer's network signal is poor or the customer is not directly close to the device;

[0071] S2. Based on the support vector machine model, the historical scene data is used as the training data of the support vector machine model to classify the multi-dimensional feature vector to obtain the scene category and scene confidence;

[0072] It should be explained that since the information pairs in the multidimensional feature vector can constitute scenario factors, such as noise and network fluctuations may jointly lead to high risks in some scenarios, but not significantly in other scenarios. Therefore, the support vector machine (SVM) with nonlinear classification capabilities combined with a suitable kernel function is the ideal solution;

[0073] Preferably, common kernel functions include linear kernel, polynomial kernel and RBF (Radial Basis Function) Gaussian kernel. Because scene factors often present a more complex nonlinear distribution and a support vector machine with nonlinear classification capability has been adopted, the selection of RBF kernel can enable the classifier to better divide various scenes in a high-dimensional space.

[0074] Furthermore, the historical scene data is taken as the training sample set, and a scene label y is assigned to each training sample in the training sample set. i ,Scene labels include,normal scene representation (label = +1), suspicious scene representation (label = 0), and high-risk scene (label = -1), where the initial label is 0;

[0075] Furthermore, the multidimensional feature vector X c (Each scene factor, here represented as x i ) and the corresponding scene label y i As a training sample pair (x i ,y i ), randomly select N-1 training sample pairs each time (x i ,y i ), leaving only one set of training sample pairs (x i ,y i ) as a validation sample pair (x a ,y b ), to verify the sample (x a ,y b ) pairs of training samples (x i ,y i ) for comparison;

[0076] Furthermore, in the comparison process, the penalty coefficient C is used to control the priority of the comparison (e.g., a randomly selected training sample pair (x i ,y i ) is close to the validation sample pair (x a ,y b ), then the training sample pair (x i ,y i ) as the benchmark, and increase the penalty coefficient C to find the next pair (x i ,y i) similar training sample pairs, and give priority to the verification sample pairs (x a ,y b ) until the final verification sample pair (x a ,y b ) than the training sample pair (x i ,y i ) results are better, and the optimal classifier f is obtained * (X c );

[0077] Furthermore, through the optimal classifier f * (X c ) for the input multidimensional feature vector X c Calculate and get the decision function D(X c ), the decision function of the support vector machine model is transformed through the Sigmoid function to obtain the scene confidence d;

[0078] Specifically, the decision function D(X c ) is expressed as:

[0079] Y←D(X c )=ω * ×φ(X c )+b *

[0080] Among them, ω * and b * They represent the optimal weight and bias terms obtained by model training, respectively, and φ(·) represents the RBF kernel function;

[0081] It should be noted that the decision function D(X c ) is the scene category Y;

[0082] Specifically, the process of converting the decision function of the support vector machine model through the Sigmoid function can be expressed as:

[0083]

[0084] S3. Use speech recognition methods to extract speech content and acoustic features from speech signals, and combine historical speech behaviors to construct speech risk factors;

[0085] It should be explained that Mel-frequency cepstral coefficient (MFCC) is a feature extraction method widely used in speech signal processing. It can effectively extract acoustic features in speech signals, especially frequency information related to human auditory perception. However, MFCC mainly extracts the spectral features of speech signals, and the extraction of information such as pitch, speech rate and pauses requires further analysis.

[0086] Furthermore, the continuous speech signal is converted into a digital signal, and the acoustic features (pitch, speaking speed, pauses, etc.) in the speech signal are extracted through the first-order difference and second-order difference coefficients of the Mel-frequency cepstral coefficient (MFCC), and then the language model is used to output the text sequence;

[0087] Specifically, considering the voiceprint module in the language model, the voiceprint vector is extracted during the speech recognition process, and the similarity is compared with the registered voiceprint vector in the database to generate a voiceprint matching factor with a value range of [0%, 100%];

[0088] It should be noted that the voiceprint matching factor is used to verify the speaker's identity and identify whether it is an authorized customer, which can prevent impersonation and fraud, and can better identify the voice interaction risks on the customer side in the application scenario, thereby improving the security of the business;

[0089] Furthermore, keyword matching is performed on the text sequence. A number of words that are considered high-risk or sensitive by business scenarios (such as "betting", "brushing orders", "cashing out", "transferring", etc.) and high-risk topics (such as "finance", "account", "cross-border", etc.) are set. The number of times these words appear in the text sequence is counted, initially 0.

[0090] Furthermore, whenever the number of times these words appear in the text sequence is +1, the source of the word is recorded and assigned a specified weight value to generate voice risk factors at different risk levels, for example (4 high-risk or sensitive words ("bet" 1 time, weight 5; "brushing orders" 3 times, weight 7) + 2 high-risk topics ("account" 2 times, weight 8) + voiceprint matching factor (unknown) = voice risk factor 42 under low to medium risk), where the highest weight value is 10;

[0091] Specifically, the low risk level range is [0,50], the medium risk level range is (50,70], and the high risk level range is (70,100];

[0092] S4. Obtain a total risk score based on the scenario category, scenario confidence, and voice risk factor. Judge the total risk score by designing a dynamic threshold function. Based on the judgment result, choose to check the voice risk factor or regenerate the multi-dimensional feature vector, thereby achieving a more accurate and effective real-time voice risk analysis method on the client side.

[0093] Specifically, the formula for the total risk score R(t) is expressed as:

[0094]

[0095] Among them, θ represents the overall weight of the voice risk factor, vj It is represented as the jth speech risk factor, k is the total number of speech risk factors, and θ2 is represented as the overall weight of the scenario risk E(t);

[0096] Furthermore, the scenario risk E(t) is expressed as:

[0097]

[0098] Among them, f(d) represents the mapping function of the scene confidence d, and increases with the increase of the output scene category Y, s i (t) represents other time factors that affect the scenario risk, such as GPS drift rate, etc.;

[0099] Furthermore, a dynamic threshold function is designed to judge the total risk score. When the total risk score is greater than the dynamic threshold function, it is judged as a misjudgment and the voice risk factor is checked. Otherwise, when the total risk score is less than the dynamic threshold function, it is judged as abnormal and the multi-dimensional feature vector is regenerated.

[0100] Specifically, the dynamic threshold function T(t) is expressed as:

[0101] T(t)=T0+E(t)

[0102] Among them, the threshold is set based on T0, which can be changed according to the preference of business risk;

[0103] It should be noted that, in different situations, the dynamic threshold function can be dynamically adjusted as the time factor of the scene risk, the scene confidence, and the scene category change; the higher the initial value of the threshold, the higher the model's tolerance for risk, and only when the total risk score is lower than the originally defined initial threshold will it be considered an abnormality; and the lower the initial value of the threshold, the lower the model's tolerance for risk (that is, it is more sensitive to risk judgment), and a total risk score slightly higher than the threshold will be considered an abnormality; therefore, it is necessary to check the misjudged speech risk factors and do not need to regenerate them, but compared with the multidimensional feature vector, more data needs to be supplemented to generate a new multidimensional feature vector in order to eliminate the abnormality.

[0104] Example 2

[0105] Reference Figure 2 and Figure 3 , which is the second embodiment of the present invention, and which provides a method for analyzing client-side risks based on speech recognition, comprising: in order to verify the effectiveness of the solution of the present invention, by comparing the performance of the prior art with the solution of the present invention in speech risk recognition, the advantages of the solution of the present invention are demonstrated;

[0106] The hardware configuration of the experiment is as follows: the experimental equipment is a high-performance voice acquisition system (device model: AudioWaveX300) with background noise suppression function; the processing platform is a high-performance computing server (Intel Xeon Platinum8358, 128GB RAM, NVIDIA A100 GPU); the data storage uses a high-speed NVMe SSD (capacity 4TB); the sensor has a built-in environmental sensing module (supporting sound intensity, Wi-Fi signal, and GPS information collection);

[0107] The software environment of the experiment is as follows: the speech recognition framework is DeepSpeech 2+TensorFlow; the voiceprint recognition module is a voiceprint feature extraction framework based on i-vector and x-vector; the language model uses a pre-trained language model based on the Transformer architecture for text semantic analysis; the classification model uses a support vector machine (SVM), in which the RBF kernel function is used in the model, and the penalty coefficient C=1.0 is configured; the data analysis tool uses Python+NumPy+Pandas;

[0108] The experimental data includes test voice data and registered user voiceprint database; a total of 2,000 real voice interaction data were selected from the test voice data, including 1,000 high-risk or sensitive data (such as voice fraud, sensitive operations, etc.) and 1,000 normal data; each voice signal is 10 to 20 seconds long, including background noise and a variety of speaking speeds; the registered user voiceprint database includes 500 users' registered voiceprint samples, covering a variety of dialects, genders and age groups; among them, the high-risk words set include 50 keywords such as "transfer", "bet", and "cash out"; the high-risk topics involved include 10 topics such as "finance" and "account";

[0109] The experimental results are as follows Figure 2 and Figure 3 As shown; first, it can be Figure 2It can be seen that the accuracy of the scheme of the present invention is 96.5%, and the recall rate is 92.1%, which are significantly higher than the traditional method (80.2% and 79.5% respectively). The reason for the improvement is that the scheme of the present invention adopts the optimization classification method of dynamic threshold function and support vector machine (SVM), and realizes more accurate classification through multi-dimensional feature vector fusion; and the misjudgment rate of the scheme of the present invention is only 2%, which is lower than the 10% of the traditional method. This shows that the present invention can more effectively distinguish risky behavior from normal behavior by combining the acoustic features of speech signals with background data, reducing false alarms; in addition, the high-risk recognition rate of the scheme of the present invention is 89.7%, while the traditional method is only 72.3%. This advantage is attributed to the in-depth analysis of sensitive words and high-risk topics by the scheme of the present invention, combined with the dynamic threshold adjustment mechanism, which significantly improves the ability to capture risky behaviors in language content; secondly, reference Figure 3 , the impact on the risk score was further verified within 10 seconds of the model output. It can be seen that the risk score of the scheme of the present invention remained at about 70 to 90 points throughout the entire time period, most of which were higher than the set score threshold, and the fluctuation was relatively small, while the score of the traditional method was between about 30 and 70 points, and maintained a large fluctuation, and most of which were lower than the set threshold;

[0110] In summary, the solution of the present invention has a significant improvement over the traditional method and provides a reference value for customer-side risk management in the field of voice interaction.

[0111] Those skilled in the art will appreciate that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0116] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A customer-side risk analysis method based on speech recognition, characterized in that: include: Acquire received data from a sensor on the device side and background information and a voice signal during voice interaction, perform mixed sampling on the received data, and concatenate the mixed sampled received data with the background information during voice interaction to obtain a multi-dimensional feature vector; Based on a support vector machine model, the historical scene data is used as training data of the support vector machine model to classify the multidimensional feature vector to obtain a scene category and a scene confidence; Using a speech recognition method, extracting speech content and acoustic features from the speech signal, and combining historical speech behavior to construct a speech risk factor; According to the scenario category, scenario confidence and voice risk factor, a total risk score is obtained. By designing a dynamic threshold function, the total risk score is judged. According to the judgment result, it is chosen to check the voice risk factor or regenerate the multidimensional feature vector, thereby realizing a more accurate and effective client-side real-time voice risk analysis method.

2. The method for client-side risk analysis based on speech recognition according to claim 1, characterized in that: The received data is subjected to mixed sampling, comprising: The received data of the device-side sensor is sampled in a mixed manner of coarse-grained and fine-grained data. The coarse-grained and fine-grained data are aligned respectively according to the time period information of the received data in the device-side sensor as the timestamp.

3. The method for analyzing client-side risks based on speech recognition according to claim 2, characterized in that: The alignment process comprises: For coarse-grained data, the coarse-grained data is interpolated to the fine-grained time axis. For fine-grained data, the fine-grained data is downsampled and aligned to the coarse-grained time axis.

4. The method for analyzing client-side risks based on speech recognition according to claim 2, characterized in that: The mixed sampled received data is concatenated with the background information during the voice interaction to obtain a multi-dimensional feature vector, including: The splicing method adopts horizontal sequential splicing to obtain a multi-dimensional feature vector, and each spliced ​​information in the multi-dimensional feature vector is combined with another spliced ​​information to obtain multiple information pairs, namely scene factors.

5. The method for client-side risk analysis based on speech recognition as claimed in claim 4, characterized in that: The historical scene data is used as training data for the support vector machine model to classify the multi-dimensional feature vector, including: Taking historical scene data as a training sample set, and assigning a scene label to each training sample in the training sample set; Each scene factor in the multidimensional feature vector and the corresponding scene label are used as training sample pairs. N-1 training sample pairs are randomly selected each time, and only one set of training sample pairs is left as verification sample pairs. The verification sample pairs are used to compare the training sample pairs.

6. The method for client-side risk analysis based on speech recognition as claimed in claim 5, characterized in that: Also includes: In the comparison process, the penalty coefficient C is used to control the comparison priority until the final verification sample is better than the training sample pair, and the optimal classifier is obtained.

7. The method for analyzing client-side risks based on speech recognition according to claim 5, characterized in that: Get scene confidence and scene category, including: The input multidimensional feature vector is calculated by the optimal classifier to obtain a decision function of a support vector machine model, and the decision function of the support vector machine model is converted by a Sigmoid function to obtain a scene confidence; Among them, the output result of the decision function of the support vector machine model is the scene category.

8. The method for client-side risk analysis based on speech recognition according to claim 1, characterized in that: Extracting speech content and acoustic features from the speech signal and combining historical speech behavior to construct a speech risk factor, including: Convert continuous speech signals into digital signals, extract acoustic features from speech signals through the first-order and second-order difference coefficients of Mel cepstral coefficients, and output text sequences using language models; The number of occurrences and sources of words in the text sequence are recorded, and weight values ​​are assigned to the corresponding words to generate speech risk factors at different risk levels.

9. The method for client-side risk analysis based on speech recognition according to claim 7 or 8, characterized in that: Based on the scenario category, scenario confidence, and voice risk factor, the overall risk score is obtained, including: The total risk score formula is expressed as: Among them, θ represents the overall weight of the voice risk factor, v j It is represented as the jth speech risk factor, k is the total number of speech risk factors, and θ2 is represented as the overall weight of the scenario risk E(t); The scenario risk E(t) is expressed as: Among them, f(d) represents the mapping function of the scene confidence d, and increases with the increase of the output scene category Y, s i (t) represents the remaining time factor affecting the scenario risk.

10. The method for client-side risk analysis based on speech recognition according to claim 9, characterized in that: By designing a dynamic threshold function, the total risk score is judged, and according to the judgment result, the speech risk factor is checked or the multi-dimensional feature vector is regenerated, including: The dynamic threshold function T(t) is expressed as: T(t)=T0+E(t) Among them, T0 is used as the basis to set the threshold; The total risk score is judged according to the dynamic threshold function. When the total risk score is greater than the dynamic threshold function, it is judged as a misjudgment and the speech risk factor is checked. Otherwise, when the total risk score is less than the dynamic threshold function, it is judged as an abnormality and the multidimensional feature vector is regenerated.

Images