Intelligent risk control voice monitoring method and system based on multi-modal algorithm

Abstract

The invention relates to the technical field of intelligent decision, and discloses an intelligent risk control voice monitoring method and system based on a multi-modal algorithm, and the method comprises the steps: carrying out the feature extraction of training voice, constructing a pre-trained text recognition model of voice features, and carrying out the model training of the pre-trained text recognition model through the voice features and a training text; dividing digital information and text information in the training text, and performing model training on a pre-trained word vector model by utilizing the preprocessed text and the training word vector; carrying out vector fusion on the training word vector, the digital information and the rule text data, and carrying out semantic association analysis on the training text and the rule text data; and generating a feature vector corresponding to the context vector and the semantic association graph, constructing a pre-trained risk discrimination model of the feature vector, and performing model training on the pre-trained risk discrimination model by using the feature vector and the training category. According to the invention, risk control voice monitoring intelligence can be improved.

Description

Technical Field

[0001] The present invention relates to an intelligent risk control voice monitoring method and system based on a multimodal algorithm, belonging to the technical field of intelligent decision-making. Background Art

[0002] Nowadays, in the current risk control business scenario, for collection voice monitoring, the traditional method relies on manual sampling review, that is, samples are drawn from a large number of collection phone recordings at a certain ratio, and professional quality inspectors listen and judge whether there are violations such as violent collection based on experience and preset rules. For repayment voice monitoring, if the repayment occurs outside the official platform channel, the existing technology can often only detect it through financial reconciliation or abnormal feedback afterwards, and lacks real-time and effective monitoring methods. The repayment voice in the official platform is mainly subject to routine review based on established processes and data records. Therefore, the existing technology has the following defects: 1. Efficiency bottleneck: Manual sampling of collection voice is extremely inefficient, and it is difficult to achieve comprehensive coverage in the face of massive data. Limited sampling samples are prone to omissions of violations, so that risks cannot be controlled in time. 2. Subjective bias: Different quality inspectors have different evaluation standards for the same collection voice due to differences in experience and understanding, resulting in poor stability and accuracy of the evaluation results. In addition, manual work for a long time is prone to fatigue and emotional influence, further weakening the credibility of quality inspection. 3. Lag in repayment monitoring: It is difficult to monitor repayment voices that are separated from the official platform in a timely manner, and one can only passively wait for subsequent financial links to discover abnormalities. During this period, the risk of capital flow increases, financial institutions are prone to potential losses, and the cost of subsequent tracing is high. Among them, it should be noted that when the relevant data of collection voices and repayment voices are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0003] Therefore, the risk control voice monitoring in the existing technology is not intelligent enough. Summary of the invention

[0004] The present invention provides an intelligent risk control voice monitoring method and system based on a multimodal algorithm, the main purpose of which is to improve the intelligence of risk control voice monitoring.

[0005] To achieve the above object, the present invention provides an intelligent risk control voice monitoring method based on a multimodal algorithm, comprising: Acquire a training speech and a training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, and perform model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model; Obtaining a training word vector corresponding to the training text, dividing the digital information and the text information in the training text, performing text preprocessing on the text information to obtain a preprocessed text, constructing a pretrained word vector model for the preprocessed text, and performing model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model; Obtaining preset regular text data, performing vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, performing semantic association analysis on the training text and the regular text data to obtain a semantic association graph; Generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk discrimination model of the feature vector, and perform model training on the pre-trained risk discrimination model using the feature vector and the training category to obtain a trained risk discrimination model; Obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to determine whether the speech to be recognized is in a risky state; When the voice to be recognized is in a risky state, risk warning information corresponding to the voice to be recognized is generated according to the risk category, the voice to be recognized and the text content, and the risk warning information is pushed to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

[0006] Optionally, the extracting features of the training speech to obtain speech features includes: The training speech is preprocessed using the following formula to obtain the preprocessed speech: in, represents the preprocessed speech, represents the preprocessing function, t represents the time variable, and s(t) represents the training speech; The pre-processed speech is converted into the time-frequency domain using the following formula to obtain a spectrum feature matrix: in, represents the spectrum feature matrix, f represents the frequency variable, n represents the time frame index, and w() represents the Heining window function. Indicates that the Hening window function is When , the center point of the time variable contained in the Haining window function, represents the imaginary part; The spectrum feature matrix is ​​used as speech feature.

[0007] Optionally, the using the speech features and the training text to perform model training on the pre-trained text recognition model to obtain a trained text recognition model includes: Performing feature mapping on the speech features to obtain an input speech sequence; Generate an output text sequence corresponding to the training text; In the pre-trained text recognition model, the text recognition probability of the input speech sequence with respect to the output text sequence is calculated using the following formula: in, represents the model output probability, represents the input speech sequence, Represents the output text sequence, represents the total number of words in the vocabulary, represents the total duration, t represents the time variable, Indicates that the pre-trained text recognition model is calculated at a given Lower Output The probability of Based on the text recognition probability, the model loss value corresponding to the pre-trained text recognition model is calculated using the following formula: in, Represents the model loss value, represents the vocabulary size, Indicates The time variable The true probability of a word, represents the text recognition probability, Indicates the total duration; The pre-trained text recognition model is trained using the model loss value to obtain a trained text recognition model.

[0008] Optionally, dividing the digital information and text information in the training text includes: Extracting numerical values ​​and text information from the training text; Constructing a regular expression for a first value among the digital values; Extracting first information of the first value by using the regular expression; Extracting second information of a second value from the digital value using a preset information extraction model; The first information and the second information are used as digital information.

[0009] Optionally, the performing text preprocessing on the text information to obtain a preprocessed text includes: Get the preset stop word list and punctuation library; According to the stop word list and the punctuation mark library, the stop words and punctuation marks in the text information are removed to obtain a preprocessed text.

[0010] Optionally, the using the preprocessed text and the training word vector to perform model training on the pre-trained word vector model to obtain a trained word vector model includes: In the pre-trained word vector model, the word vector probability of the pre-processed text with respect to the training word vector is calculated using the following formula: in, represents the word vector probability, Represents the central word in the preprocessed text The training word vectors of Represents the context words in the preprocessed text The training word vectors of represents the set of words in the vocabulary, Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; Based on the word vector probability, the model loss index corresponding to the pre-trained word vector model is calculated using the following formula: in, represents the model loss index, represents the total number of words in the vocabulary, represents the context window size, Indicates that given the central word The context word appears in the case of The probability of Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; The pre-trained word vector model is trained using the model loss index to obtain a trained word vector model.

[0011] Optionally, the performing vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector includes: Using a preset multi-layer neural network to convert the training word vector and the digital information into a hidden layer text vector; Converting the rule text data into a hidden layer rule vector; The attention score between the hidden layer text vector and the hidden layer rule vector is calculated using the following formula: in, represents the attention score, represents the hidden layer text vector, represents the hidden layer rule vector, express Middle hidden layer text vectors, express Middle hidden layer rule vector, express The weight parameter, express The weight parameter, Indicates bias; The attention weight corresponding to the attention score is calculated using the following formula: in, represents the attention weight, represents the attention score, express The serial number of the hidden layer rule vector in , Indicates The hidden layer text vector is The attention score between the hidden layer rule vectors, express The number of hidden layer rule vectors in ; Based on the attention weight, the training word vector, the digital information and the regular text data are vector-fused using the following formula to obtain a context vector: in, represents the context vector, represents the attention weight, express Middle hidden layer rule vector, express The number of hidden layer rule vectors in .

[0012] Optionally, performing semantic association analysis on the training text and the rule text data to obtain a semantic association graph includes: Extracting a first key semantics in the training text and a second key semantics in the rule text data; The association strength between the first key semantics and the second key semantics is calculated using the following formula: in, represents the strength of association, Indicates the first key semantics, Indicates the second key semantics; A semantic association graph between the training text and the regular text data is generated according to the association strength.

[0013] Optionally, the using the feature vector and the training category to perform model training on the pre-trained risk discrimination model to obtain a trained risk discrimination model includes: In the pre-trained risk discrimination model, the risk type corresponding to the feature vector is calculated using the following formula: in, represents the risk type, g represents the classification function, are the parameters of the pre-trained risk discrimination model; The risk loss value between the risk type and the training category is calculated using the following formula: in, represents the risk loss value, represents the number of samples, Indicates The true risk category labels corresponding to the training categories, represents the risk category probability output by the pre-trained risk discrimination model, represents the semantic association consistency loss function, represents the true association label between the first key semantics u and the second key semantics v, represents the association probability of the semantic association graph output by the pre-trained risk discrimination model, and represents the trade-off coefficient, Representing semantic association graph Middle side, represents the risk classification loss function; The pre-trained risk discrimination model is trained using the risk loss value to obtain a trained risk discrimination model.

[0014] In order to solve the above problems, the present invention also provides an intelligent risk control voice monitoring system based on a multimodal algorithm, the system comprising: A model training module is used to obtain a training speech and a training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, and perform model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model; A word vector training module is used to obtain the training word vector corresponding to the training text, divide the digital information and text information in the training text, perform text preprocessing on the text information to obtain a preprocessed text, construct a pretrained word vector model of the preprocessed text, and perform model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model; An association analysis module is used to obtain preset regular text data, perform vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, and perform semantic association analysis on the training text and the regular text data to obtain a semantic association graph; A risk discrimination module is used to generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk discrimination model of the feature vector, and perform model training on the pre-trained risk discrimination model using the feature vector and the training category to obtain a trained risk discrimination model; A state judgment module is used to obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to judge whether the speech to be recognized is in a risky state; The information push module is used to generate risk warning information corresponding to the voice to be recognized according to the risk category, the voice to be recognized and the text content when the voice to be recognized is in a risk state, and push the risk warning information to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

[0015] Compared with the problems described in the background technology, the embodiment of the present invention can effectively capture the characteristic changes of the speech signal at different times and frequencies by extracting features from the training speech and using STFT for time-frequency domain conversion. Compared with traditional feature extraction methods, it can better process the resonance peaks, pitch and other information in the speech, and provide richer and more accurate feature representation for subsequent speech recognition, thereby improving the accuracy of speech recognition, especially in complex environments (such as background noise, large changes in speaking speed, etc.). The speech recognition effect is significantly improved. Furthermore, the embodiment of the present invention uses the pre-processed text and the training word vector to train the pre-trained word vector model, and adopts a multi-corpus joint training method. In addition to the general corpus, a professional corpus in the financial field is also introduced. The word vector generated in this way can better capture the semantic information of specific vocabulary in the financial field, so that the text representation is more in line with the risk control business needs. In subsequent multimodal fusion and risk assessment, the financial-related semantics in the voice text can be understood more accurately and the misjudgment rate can be reduced. Furthermore, the embodiment of the present invention can mine deep semantic association information by performing semantic association analysis on the training text and the rule text data to explicitly model the semantic relationship between the voice text and the rule text. For example, for some implicitly expressed violent collection intentions or non-standard repayment information descriptions, the semantic association graph can assist the model to better understand its potential risks, improve the recall rate of risk judgment, and avoid risk omissions. Therefore, the intelligent risk control voice monitoring method and system based on multimodal algorithms provided by the embodiment of the present invention can improve the intelligence of risk control voice monitoring through intelligent neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of an intelligent risk control voice monitoring method based on a multimodal algorithm provided in one embodiment of the present invention; Figure 2 A schematic diagram of a module for implementing the intelligent risk control voice monitoring method based on a multimodal algorithm provided in one embodiment of the present invention.

[0017] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0019] The embodiment of the present application provides an intelligent risk control voice monitoring method based on a multimodal algorithm. The execution subject of the intelligent risk control voice monitoring method based on a multimodal algorithm includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the intelligent risk control voice monitoring method based on a multimodal algorithm can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0020] Embodiment 1: Reference Figure 1 FIG. 1 is a flow chart of an intelligent risk control voice monitoring method based on a multimodal algorithm provided by an embodiment of the present invention. In this embodiment, the intelligent risk control voice monitoring method based on a multimodal algorithm includes: S1. Obtain training speech and training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, perform model training on the pre-trained text recognition model using the speech features and the training text, and obtain a trained text recognition model.

[0021] In the embodiment of the present invention, the training voice refers to the training sample used for the subsequent training of the neural network model, including voice information obtained from various collection call lines, communication links that may involve informal repayment voice interaction, etc., and the training text refers to the text data corresponding to the training voice. It should be noted that in this application, the voice information and related data obtained from the communication links involving informal repayment voice interaction, etc., when applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0022] Furthermore, the embodiment of the present invention can effectively capture the characteristic changes of the speech signal at different times and frequencies by extracting features from the training speech and using STFT for time-frequency domain conversion. Compared with traditional feature extraction methods, it can better process the resonance peaks, pitch and other information in the speech, and provide richer and more accurate feature representation for subsequent speech recognition, thereby improving the accuracy of speech recognition, especially in complex environments (such as background noise, large changes in speech speed, etc.).

[0023] In one embodiment of the present invention, extracting features from the training speech to obtain speech features includes: performing speech preprocessing on the training speech using the following formula to obtain preprocessed speech: in, represents the preprocessed speech, represents the preprocessing function, t represents the time variable, and s(t) represents the training speech; The pre-processed speech is converted into the time-frequency domain using the following formula to obtain a spectrum feature matrix: in, represents the spectrum feature matrix, f represents the frequency variable, n represents the time frame index, and w() represents the Heining window function. Indicates that the Hening window function is When , the center point of the time variable contained in the Haining window function, represents the imaginary part; The spectrum feature matrix is ​​used as speech feature.

[0024] in, It includes a series of operations such as a noise filtering algorithm based on spectrum analysis and a volume normalization algorithm. Furthermore, the pre-trained text recognition model refers to an untrained LSTM-CTC architecture speech-to-text model.

[0025] In one embodiment of the present invention, the method of performing model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model includes: performing feature mapping on the speech features to obtain an input speech sequence; generating an output text sequence corresponding to the training text; and in the pre-trained text recognition model, calculating the text recognition probability of the input speech sequence with respect to the output text sequence using the following formula: in, represents the model output probability, represents the input speech sequence, Represents the output text sequence, represents the total number of words in the vocabulary, represents the total duration, t represents the time variable, Indicates that the pre-trained text recognition model is calculated at a given Lower Output The probability of Based on the text recognition probability, the model loss value corresponding to the pre-trained text recognition model is calculated using the following formula: in, Represents the model loss value, represents the vocabulary size, Indicates The time variable The true probability of a word, represents the text recognition probability, Indicates the total duration; The pre-trained text recognition model is trained using the model loss value to obtain a trained text recognition model.

[0026] It should be noted that if the pre-trained text recognition model outputs The words are correct, then is 1, otherwise is 0.

[0027] Optionally, the process of feature mapping the speech features refers to the process of feature mapping and encoding the speech features, which can be achieved by an encoder. Furthermore, the process of training the pre-trained text recognition model through the model loss value to obtain a trained text recognition model refers to the process of optimizing the weights and biases of the model using a neural network parameter optimization algorithm, thereby reducing the model loss value.

[0028] S2. Obtain the training word vector corresponding to the training text, divide the digital information and text information in the training text, perform text preprocessing on the text information to obtain a preprocessed text, construct a pretrained word vector model for the preprocessed text, and perform model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model.

[0029] In the embodiment of the present invention, the training word vector refers to a word vector training sample corresponding to the training text.

[0030] In one embodiment of the present invention, the dividing of the digital information and text information in the training text includes: extracting the digital values ​​and text information in the training text; constructing a regular expression for a first value in the digital values; extracting first information of the first value through the regular expression; extracting second information of a second value in the digital values ​​using a preset information extraction model; and treating the first information and the second information as digital information.

[0031] The first value refers to the repayment amount, the second value refers to the repayment account, and the information extraction model refers to a model for extracting repayment account information, such as a natural language extraction (NLP) model.

[0032] Optionally, the process of extracting the first information of the first value by using the regular expression refers to: for the repayment amount extraction, using the regular expression Matches a string of numbers as possible repayment amounts. It should be noted that the "?" in a regular expression usually indicates an optional item, indicating that the preceding character or subexpression may or may not appear. For example, the regular expression ab?c matches "ac" or "abc" but not "abbc".

[0033] In one embodiment of the present invention, the text information is subjected to text preprocessing to obtain preprocessed text, including: obtaining a preset stop word list and punctuation mark library; and removing stop words and punctuation marks in the text information according to the stop word list and the punctuation mark library to obtain preprocessed text.

[0034] The stop word list and the punctuation mark library refer to pre-built databases of stop words and punctuation marks that need to be cleaned.

[0035] Furthermore, the embodiment of the present invention uses the pre-processed text and the training word vector to train the pre-trained word vector model, so as to adopt a multi-corpus joint training method, and introduces a professional corpus in the financial field in addition to the general corpus, so that the generated word vector can better capture the semantic information of specific vocabulary in the financial field, making the text representation more in line with the risk control business needs. In the subsequent multimodal fusion and risk assessment, the financial-related semantics in the speech text can be understood more accurately, reducing the misjudgment rate.

[0036] The pre-trained word vector model refers to an untrained word vector model Word2Vec.

[0037] In one embodiment of the present invention, the pre-trained word vector model is trained using the pre-processed text and the training word vector to obtain a trained word vector model, including: in the pre-trained word vector model, the word vector probability of the pre-processed text with respect to the training word vector is calculated using the following formula: in, represents the word vector probability, Represents the central word in the preprocessed text The training word vectors of Represents the context words in the preprocessed text The training word vectors of represents the set of words in the vocabulary, Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; Based on the word vector probability, the model loss index corresponding to the pre-trained word vector model is calculated using the following formula: in, represents the model loss index, represents the total number of words in the vocabulary, represents the context window size, Indicates that given the central word The context word appears in the case of The probability of Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; The pre-trained word vector model is trained using the model loss index to obtain a trained word vector model.

[0038] Optionally, the process of training the pre-trained word vector model through the model loss index to obtain a trained word vector model refers to the process of optimizing the weights and biases of the model using a neural network parameter optimization algorithm, thereby reducing the model loss value.

[0039] S3. Obtain preset regular text data, perform vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, perform semantic association analysis on the training text and the regular text data to obtain a semantic association graph.

[0040] In the embodiment of the present invention, the rule text data refers to pre-constructed rule text data, such as data in a database such as a violent debt collection keyword library and an official repayment process text library.

[0041] In one embodiment of the present invention, the vector fusion of the training word vector, the digital information and the rule text data to obtain the context vector includes: using a preset multi-layer neural network to convert the training word vector and the digital information into a hidden layer text vector; converting the rule text data into a hidden layer rule vector; and calculating the attention score between the hidden layer text vector and the hidden layer rule vector using the following formula: in, represents the attention score, represents the hidden layer text vector, represents the hidden layer rule vector, express Middle hidden layer text vectors, express Middle hidden layer rule vector, express The weight parameter, express The weight parameter, Indicates bias; The attention weight corresponding to the attention score is calculated using the following formula: in, represents the attention weight, represents the attention score, express The sequence number of the hidden layer rule vector in , Indicates The hidden layer text vector is The attention score between the hidden layer rule vectors, express The number of hidden layer rule vectors in ; Based on the attention weight, the training word vector, the digital information and the regular text data are vector-fused using the following formula to obtain a context vector: in, represents the context vector, represents the attention weight, express Middle hidden layer rule vector, express The number of hidden layer rule vectors in .

[0042] The multi-layer neural network refers to a neural network including an input layer, a hidden layer and an output layer, such as a BP neural network.

[0043] Optionally, the process of converting the regular text data into a hidden layer rule vector can be implemented by a multi-layer neural network.

[0044] Furthermore, the embodiments of the present invention perform semantic association analysis on the training text and the rule text data to explicitly model the semantic relationship between the speech text and the rule text, thereby being able to mine deep semantic association information. For example, for some implicit expressions of violent debt collection intentions or non-standard repayment information descriptions, the semantic association graph can assist the model to better understand its potential risks, improve the recall rate of risk judgment, and avoid risk omissions.

[0045] In one embodiment of the present invention, the semantic association analysis of the training text and the rule text data to obtain a semantic association graph includes: extracting a first key semantic in the training text and a second key semantic in the rule text data; and calculating the association strength between the first key semantic and the second key semantic using the following formula: in, represents the strength of association, Indicates the first key semantics, Indicates the second key semantics; A semantic association graph between the training text and the regular text data is generated according to the association strength.

[0046] The first key semantics and the second key semantics refer to key information in training text and rule text data, such as words, phrases or concepts.

[0047] Optionally, the process of generating a semantic association graph between the training text and the rule text data through the association strength refers to connecting two key semantics through the size of the association strength, using the key semantics as graph nodes, and the edge weight of the edge between the two nodes is the association strength.

[0048] S4. Generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk discrimination model for the feature vector, and use the feature vector and the training category to perform model training on the pre-trained risk discrimination model to obtain a trained risk discrimination model.

[0049] In the embodiment of the present invention, the feature vector refers to the result of concatenating the context vector and the semantic association graph. Furthermore, the training category refers to the training sample of the risk category corresponding to the feature vector.

[0050] In one embodiment of the present invention, the method of using the feature vector and the training category to perform model training on the pre-trained risk discrimination model to obtain a trained risk discrimination model includes: in the pre-trained risk discrimination model, using the following formula to calculate the risk category corresponding to the feature vector: in, represents the risk type, g represents the classification function, are the parameters of the pre-trained risk discrimination model; The risk loss value between the risk type and the training category is calculated using the following formula: in, represents the risk loss value, represents the number of samples, Indicates The true risk category labels corresponding to the training categories, represents the risk category probability output by the pre-trained risk discrimination model, represents the semantic association consistency loss function, represents the true association label between the first key semantics u and the second key semantics v, represents the association probability of the semantic association graph output by the pre-trained risk discrimination model, and represents the trade-off coefficient, Representing semantic association graph Middle side, represents the risk classification loss function; The pre-trained risk discrimination model is trained using the risk loss value to obtain a trained risk discrimination model.

[0051] Optionally, the process of training the pre-trained risk discrimination model through the risk loss value to obtain a trained risk discrimination model refers to the process of optimizing the weights and biases of the model using a neural network parameter optimization algorithm, thereby reducing the model loss value.

[0052] in, and is a trade-off coefficient used to balance the importance of the two tasks. Represents the actual association label between semantic units u and v (1 if the association label exists, otherwise 0).

[0053] S5. Obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to determine whether the speech to be recognized is in a risky state.

[0054] Optionally, the process of outputting the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model is the process of using the trained model to recognize the text and risk category corresponding to the speech, which is consistent with the aforementioned process of identifying the training text and training category corresponding to the training text, and will not be further elaborated here.

[0055] Among them, the risk category includes multiple categories, among which there are non-risk categories, such as safety, and risk categories. If there is a risk category, it means that the voice to be recognized is in a risky state. For example, the risk category includes the risk of violent debt collection, the risk of repayment outside the platform, and no risk.

[0056] S6. When the voice to be recognized is in a risky state, risk warning information corresponding to the voice to be recognized is generated according to the risk category, the voice to be recognized and the text content, and the risk warning information is pushed to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

[0057] Optionally, the process of generating risk warning information corresponding to the voice to be recognized based on the risk category, the voice to be recognized and the text content means that when the risk intelligent discrimination model determines that there is a risk, warning information containing the risk category and the voice to be recognized and the text content is generated. Furthermore, the process of pushing the risk warning information to the corresponding risk control management personnel means that the generated warning information will be pushed to the risk control management personnel in a timely manner through various means (such as SMS, email, internal message push of the risk control system, etc.) so that they can take corresponding measures quickly.

[0058] Compared with the problems described in the background technology, the embodiment of the present invention can effectively capture the characteristic changes of the speech signal at different times and frequencies by extracting features from the training speech and using STFT for time-frequency domain conversion. Compared with traditional feature extraction methods, it can better process the resonance peaks, pitch and other information in the speech, and provide richer and more accurate feature representation for subsequent speech recognition, thereby improving the accuracy of speech recognition, especially in complex environments (such as background noise, large changes in speaking speed, etc.). The speech recognition effect is significantly improved. Furthermore, the embodiment of the present invention uses the pre-processed text and the training word vector to train the pre-trained word vector model, and adopts a multi-corpus joint training method. In addition to the general corpus, a professional corpus in the financial field is also introduced. The word vector generated in this way can better capture the semantic information of specific vocabulary in the financial field, so that the text representation is more in line with the risk control business needs. In subsequent multimodal fusion and risk assessment, the financial-related semantics in the voice text can be understood more accurately and the misjudgment rate can be reduced. Furthermore, the embodiment of the present invention can mine deep semantic association information by performing semantic association analysis on the training text and the rule text data to explicitly model the semantic relationship between the voice text and the rule text. For example, for some implicitly expressed violent collection intentions or non-standard repayment information descriptions, the semantic association graph can assist the model to better understand its potential risks, improve the recall rate of risk judgment, and avoid risk omissions. Therefore, the intelligent risk control voice monitoring method and system based on multimodal algorithms provided by the embodiment of the present invention can improve the intelligence of risk control voice monitoring through intelligent neural networks.

[0059] Embodiment 2: like Figure 2 The figure shows a functional module diagram of an intelligent risk control voice monitoring system based on a multimodal algorithm of the present invention.

[0060] The intelligent risk control voice monitoring system 200 based on a multimodal algorithm described in the present invention can be installed in an electronic device. According to the functions implemented, the intelligent risk control voice monitoring system based on a multimodal algorithm can include a model training module 201, a word vector training module 202, an association analysis module 203, a risk identification module 204, a state judgment module 205 and an information push module 206. The module described in the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0061] In the embodiment of the present invention, the functions of each module / unit are as follows: The model training module 201 is used to obtain a training speech and a training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, and perform model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model; The word vector training module 202 is used to obtain the training word vector corresponding to the training text, divide the digital information and text information in the training text, perform text preprocessing on the text information to obtain the preprocessed text, construct a pretrained word vector model of the preprocessed text, and perform model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model; The association analysis module 203 is used to obtain preset regular text data, perform vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, and perform semantic association analysis on the training text and the regular text data to obtain a semantic association graph; The risk identification module 204 is used to generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk identification model of the feature vector, and perform model training on the pre-trained risk identification model using the feature vector and the training category to obtain a trained risk identification model; The state judgment module 205 is used to obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to judge whether the speech to be recognized is in a risky state; The information push module 206 is used to generate risk warning information corresponding to the voice to be recognized according to the risk category, the voice to be recognized and the text content when the voice to be recognized is in a risk state, and push the risk warning information to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

[0062] In detail, each module in the intelligent risk control voice monitoring system 200 based on the multimodal algorithm in the embodiment of the present invention is used in the same manner as described above. Figure 1 The intelligent risk control voice monitoring method based on multimodal algorithm described in the present invention has the same technical means and can produce the same technical effects, so I will not go into details here.

[0063] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent risk control voice monitoring method based on a multimodal algorithm, characterized in that: The method comprises: Acquire a training speech and a training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, and perform model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model; Obtaining a training word vector corresponding to the training text, dividing the digital information and the text information in the training text, performing text preprocessing on the text information to obtain a preprocessed text, constructing a pretrained word vector model for the preprocessed text, and performing model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model; Obtaining preset regular text data, performing vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, performing semantic association analysis on the training text and the regular text data to obtain a semantic association graph; Generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk discrimination model of the feature vector, and perform model training on the pre-trained risk discrimination model using the feature vector and the training category to obtain a trained risk discrimination model; Obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to determine whether the speech to be recognized is in a risky state; When the voice to be recognized is in a risky state, risk warning information corresponding to the voice to be recognized is generated according to the risk category, the voice to be recognized and the text content, and the risk warning information is pushed to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

2. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The step of extracting features from the training speech to obtain speech features includes: The training speech is preprocessed using the following formula to obtain the preprocessed speech: in, represents the preprocessed speech, represents the preprocessing function, t represents the time variable, and s(t) represents the training speech; The pre-processed speech is converted into the time-frequency domain using the following formula to obtain a spectrum feature matrix: in, represents the spectrum feature matrix, f represents the frequency variable, n represents the time frame index, and w() represents the Heining window function. Indicates that the Hening window function is When , the center point of the time variable contained in the Haining window function, represents the imaginary part; The spectrum feature matrix is ​​used as speech feature.

3. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The method of performing model training on the pre-trained text recognition model by using the speech features and the training text to obtain a trained text recognition model includes: Performing feature mapping on the speech features to obtain an input speech sequence; Generate an output text sequence corresponding to the training text; In the pre-trained text recognition model, the text recognition probability of the input speech sequence with respect to the output text sequence is calculated using the following formula: in, represents the model output probability, represents the input speech sequence, Represents the output text sequence, represents the total number of words in the vocabulary, represents the total duration, t represents the time variable, Indicates that the pre-trained text recognition model is calculated at a given Lower Output probability; Based on the text recognition probability, the model loss value corresponding to the pre-trained text recognition model is calculated using the following formula: in, Represents the model loss value, represents the vocabulary size, Indicates The time variable The true probability of a word, represents the text recognition probability, Indicates the total duration; The pre-trained text recognition model is trained using the model loss value to obtain a trained text recognition model.

4. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The dividing of the digital information and the text information in the training text includes: Extracting numerical values ​​and text information from the training text; Constructing a regular expression for a first value among the digital values; Extracting first information of the first value by using the regular expression; Extracting second information of a second value from the digital value using a preset information extraction model; The first information and the second information are used as digital information.

5. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The step of performing text preprocessing on the text information to obtain a preprocessed text includes: Get the preset stop word list and punctuation library; According to the stop word list and the punctuation mark library, the stop words and punctuation marks in the text information are removed to obtain a preprocessed text.

6. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The method of performing model training on the pre-trained word vector model using the pre-processed text and the training word vector to obtain a trained word vector model includes: In the pre-trained word vector model, the word vector probability of the pre-processed text with respect to the training word vector is calculated using the following formula: in, represents the word vector probability, Represents the central word in the preprocessed text The training word vectors of Represents the context words in the preprocessed text The training word vectors of represents the set of words in the vocabulary, Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; Based on the word vector probability, the model loss index corresponding to the pre-trained word vector model is calculated using the following formula: in, represents the model loss index, represents the total number of words in the vocabulary, represents the context window size, Indicates that given the central word The context word appears in the case of The probability of Represents the ordinal number of the vocabulary in the preprocessed text, Indicates from Jump to the value of other vocabulary numbers; The pre-trained word vector model is trained using the model loss index to obtain a trained word vector model.

7. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The step of performing vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector includes: Using a preset multi-layer neural network to convert the training word vector and the digital information into a hidden layer text vector; Converting the rule text data into a hidden layer rule vector; The attention score between the hidden layer text vector and the hidden layer rule vector is calculated using the following formula: in, represents the attention score, represents the hidden layer text vector, represents the hidden layer rule vector, express Middle hidden layer text vectors, express Middle hidden layer rule vector, express The weight parameter, express The weight parameter, Indicates bias; The attention weight corresponding to the attention score is calculated using the following formula: in, represents the attention weight, represents the attention score, express The serial number of the hidden layer rule vector in , Indicates The hidden layer text vector is The attention score between the hidden layer rule vectors, express The number of hidden layer rule vectors in ; Based on the attention weight, the training word vector, the digital information and the regular text data are vector-fused using the following formula to obtain a context vector: in, represents the context vector, represents the attention weight, express Middle hidden layer rule vector, express The number of hidden layer rule vectors in .

8. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The performing semantic association analysis on the training text and the rule text data to obtain a semantic association graph includes: Extracting a first key semantics in the training text and a second key semantics in the rule text data; The association strength between the first key semantics and the second key semantics is calculated using the following formula: in, represents the strength of association, Indicates the first key semantics, Indicates the second key semantics; A semantic association graph between the training text and the regular text data is generated according to the association strength.

9. The intelligent risk control voice monitoring method based on multimodal algorithm according to claim 1, characterized in that: The using the feature vector and the training category to perform model training on the pre-trained risk discrimination model to obtain a trained risk discrimination model includes: In the pre-trained risk discrimination model, the risk type corresponding to the feature vector is calculated using the following formula: in, represents the risk type, g represents the classification function, are the parameters of the pre-trained risk discrimination model; The risk loss value between the risk type and the training category is calculated using the following formula: in, represents the risk loss value, represents the number of samples, Indicates The true risk category labels corresponding to the training categories, represents the risk category probability output by the pre-trained risk discrimination model, represents the semantic association consistency loss function, represents the true association label between the first key semantics u and the second key semantics v, represents the association probability of the semantic association graph output by the pre-trained risk discrimination model, and represents the trade-off coefficient, Representing semantic association graph Middle side, represents the risk classification loss function; The pre-trained risk discrimination model is trained using the risk loss value to obtain a trained risk discrimination model.

10. An intelligent risk control voice monitoring system based on a multimodal algorithm, characterized in that: The system comprises: A model training module is used to obtain a training speech and a training text corresponding to the training speech, perform feature extraction on the training speech to obtain speech features, construct a pre-trained text recognition model of the speech features, and perform model training on the pre-trained text recognition model using the speech features and the training text to obtain a trained text recognition model; A word vector training module is used to obtain the training word vector corresponding to the training text, divide the digital information and text information in the training text, perform text preprocessing on the text information to obtain a preprocessed text, construct a pretrained word vector model of the preprocessed text, and perform model training on the pretrained word vector model using the preprocessed text and the training word vector to obtain a trained word vector model; An association analysis module is used to obtain preset regular text data, perform vector fusion on the training word vector, the digital information and the regular text data to obtain a context vector, and perform semantic association analysis on the training text and the regular text data to obtain a semantic association graph; A risk discrimination module is used to generate a feature vector corresponding to the context vector and the semantic association graph, obtain a training category corresponding to the feature vector, construct a pre-trained risk discrimination model of the feature vector, and perform model training on the pre-trained risk discrimination model using the feature vector and the training category to obtain a trained risk discrimination model; A state judgment module is used to obtain the speech to be recognized at the current moment, output the text content and risk category corresponding to the speech to be recognized through the trained text recognition model, the trained word vector model and the trained risk discrimination model, and use the risk category to judge whether the speech to be recognized is in a risky state; The information push module is used to generate risk warning information corresponding to the voice to be recognized according to the risk category, the voice to be recognized and the text content when the voice to be recognized is in a risk state, and push the risk warning information to the corresponding risk control management personnel to obtain the risk control voice monitoring result of the voice to be recognized.

Images