Method for automatically adjusting parameters of intention recognition model by mining new intention labels and fusing structured information

Abstract

The invention provides a method for automatically adjusting parameters of an intention recognition model by mining new intention labels and fusing structured information, and belongs to the technical field of artificial intelligence. The method comprises the following steps: inputting first incoming call data or second incoming call data and an intention label set into a pre-trained new intention label mining model, the first incoming call data is data of a first incoming call and an unrecognized intention, the second incoming call data is data of a second incoming call and an intention recognition result is not matched with an actual intention of a user, and the new intention label mining model is constructed based on a large model; and iteratively updating the intention label set based on the new intention candidate labels to obtain an intention label result set. According to the method, the new intention label mining model is used for iteratively obtaining the new intention label, new intentions outside an intention system are automatically found, the intention coverage rate is increased, and therefore the accuracy of an intention recognition result is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for mining new intent tags and automatically tuning parameters of an intent recognition model by integrating structured information. Background Art

[0002] Real-time intent recognition refers to determining the intent tags to which a customer's incoming call belongs based on the real-time transcribed interaction text of the call between the customer and the agent. The intent tags include two levels. The first level is the major intent classification categories, such as service scenarios, marketing scenarios, etc. The second level is the intent names, such as traffic query, fee challenge - traffic overage, etc. Accurately positioning the intent of the customer's incoming call, helping front-line agents find the best tools that match the intent, and simplifying the operation steps are of great significance for reducing the call duration and improving satisfaction.

[0003] Intent recognition belongs to a typical multi-classification task method. Currently, the research on intent recognition classification only performs recognition based on text information, has not utilized structured information such as interfaces, and can only recognize intents within a preset intent system, and cannot actively discover new intents outside the system that have not been sorted out manually. As a result, the addition and discovery of new intents consume a large amount of labor costs, and the cycle for improving the intent accuracy and coverage is relatively long.

[0004] Therefore, there is an urgent need for a method for mining new intent tags and automatically tuning parameters of an intent recognition model by integrating structured information to solve the problem that the technology cannot output new intents outside the preset intent system. Summary of the Invention

[0005] The present invention provides a method, device, electronic device, and storage medium for mining new intent tags and automatically tuning parameters of an intent recognition model by integrating structured information to solve the defect in the prior art that new intents outside the preset intent system cannot be output.

[0006] The present invention provides a method for mining new intent tags and automatically tuning parameters of an intent recognition model by integrating structured information, including the following steps: Inputting first incoming call data or second incoming call data, and an intent tag set into a pre-trained new intent tag mining model to obtain new intent candidate tags output by the new intent tag mining model. The first incoming call data is data of a first-time incoming call for which no intent is recognized, and the second incoming call data is data of a second-time incoming call for which the intent recognition result does not match the actual intent of the user. The new intent tag mining model is constructed based on a large model; Iteratively updating the intent tag set based on the new intent candidate tags to obtain an intent tag result set.

[0007] A method for automatically tuning parameters of an intent recognition model by mining new intent tags according to the present invention and fusing structured information. After iteratively updating the intent tag set based on the new intent candidate tags to obtain an intent tag result set, the method further includes: Inputting the intent tag result set and the call data to be recognized into a real-time intent recognition model to obtain the user call intent output by the real-time intent recognition model; Wherein, the real-time intent recognition model is determined based on the following steps: Initializing the knowledge base weights of multiple knowledge bases; Training the large model using sample data based on the knowledge base weights of the multiple knowledge bases to obtain the iteratively updated knowledge base weights and large model parameters; Inputting the iteratively updated knowledge base weights and the large model parameters into a small model to obtain the predicted intent tag probability result output by the small model, where the predicted intent tag probability result is the probability that the predicted call intent belongs to each intent tag; Based on the predicted intent tag probability result, using an automatic parameter tuning module to iteratively train to obtain the final knowledge base weights and large model parameters, and completing the training of the real-time intent recognition model.

[0008] According to the method for automatically tuning parameters of an intent recognition model by mining new intent tags according to the present invention and fusing structured information, the step of using an automatic parameter tuning module to iteratively train to obtain the final knowledge base weights and large model parameters based on the predicted intent tag probability result includes: Initializing the weights of each intent tag predicted by the small model; Inputting the weights of each intent tag into the automatic parameter tuning module, and the automatic parameter tuning module updates the large model parameters and the weights of each intent tag using the gradient descent algorithm; The automatic parameter tuning module adjusts the knowledge base weights of each knowledge base based on the contribution of each knowledge base, and the contribution of the knowledge base reflects the influence degree of the knowledge base on the loss value; The automatic parameter tuning module iteratively updates the large model parameters, the weights of each intent tag, and the knowledge base weights of each knowledge base until the weights of each intent tag reach the preset optimal weights.

[0009] A method for automatically tuning the parameters of an intent recognition model by mining new intent tags and fusing structured information according to the present invention. The knowledge base includes an interface class knowledge base, a tag class knowledge base, and an annotation data knowledge base. The interface class knowledge base is a database constructed based on multi-source structured information of the interface class. The tag class knowledge base is a database constructed based on multi-source structured information of the tag class. The annotation data knowledge base is an annotation example database constructed based on the call text content obtained by voice transcription.

[0010] A method for automatically tuning the parameters of an intent recognition model by mining new intent tags and fusing structured information according to the present invention. The method for automatically tuning the parameters of the intent recognition model by mining new intent tags and fusing structured information further includes: If a user with the same number calls again within a preset duration, it is determined that the call is a second call. Based on the intent result of the previous call of the number, a first intent set of the user is determined. The first intent set includes multiple intent tags. Based on the interface class information corresponding to the number, a second intent set of the user is determined. The second intent set includes multiple intent tags. Based on the tag class information corresponding to the number, a third intent set of the user is determined. The third intent set includes multiple intent tags. Based on the first intent set, the second intent set, and the third intent set, an intent result set of the user is determined.

[0011] A method for automatically tuning the parameters of an intent recognition model by mining new intent tags and fusing structured information according to the present invention. The method further includes: The call text and the hit result of the user's call intent are sent to Kafka in real time, so that Kafka uploads the call text and the hit result of the user's call intent to the file server daily to generate an intent analysis report.

[0012] The present invention also provides a device for automatically tuning the parameters of an intent recognition model by mining new intent tags and fusing structured information, including the following modules: A tag mining module, configured to: input first call data or second call data, and an intent tag set into a pre-trained new intent tag mining model, and obtain new intent candidate tags output by the new intent tag mining model. The first call data is data of a first call for which no intent is recognized, and the second call data is data of a second call for which the intent recognition result does not match the actual intent of the user. The new intent tag mining model is constructed based on a large model. A tag update module, configured to: iteratively update the intent tag set based on the new intent candidate tags to obtain an intent tag result set.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information as described in any one of the above is implemented.

[0016] For the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information provided by the present invention, the first incoming call data or the second incoming call data, and an intent tag set are input into a pre-trained new intent tag mining model to obtain new intent candidate tags output by the new intent tag mining model. The first incoming call data is data of a first incoming call for which no intent is recognized, and the second incoming call data is data of a second incoming call for which the intent recognition result does not match the actual intent of the user. The new intent tag mining model is constructed based on a large model; based on the new intent candidate tags, the intent tag set is iteratively updated to obtain an intent tag result set. The present invention uses a new intent tag mining model to continuously iterate to obtain new intent tags, realizing the automatic discovery of new intents outside the manually sorted intent system. On the one hand, it reduces the cost of manually sorting the intent system, and on the other hand, it improves the intent coverage rate and comprehensiveness, thereby improving the accuracy of the intent recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is one of the flow diagrams of the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information provided by the present invention; Figure 2 is another flow diagram of the method for automatically tuning parameters of an intent recognition model by mining new intent tags and fusing structured information provided by the present invention; Figure 3 It is the implementation framework diagram of the new intent label mining model provided by the present invention; Figure 4 It is the implementation framework diagram of the real-time intent recognition model provided by the present invention; Figure 5 It is the exemplary schematic diagram of the intent annotation set data provided by the present invention; Figure 6 It is the schematic diagram of the real-time intent result data provided by the present invention; Figure 7 It is the structural schematic diagram of the device for automatically adjusting parameters of the new intent label mining and fusing structured information for intent recognition model provided by the present invention; Figure 8 It is the structural schematic diagram of the electronic device provided by the present invention. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. The orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. Unless otherwise clearly defined and limited, the terms "install", "connect", "couple" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0021] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0022] Figure 1 is one of the flow schematic diagrams of the method for automatically tuning the intent recognition model by mining new intent tags and fusing structured information provided by the present invention. As Figure 1 shown, the method includes the following: S110, input the first incoming call data or the second incoming call data, and the intent tag set into the pre-trained new intent tag mining model to obtain new intent candidate tags output by the new intent tag mining model; S120, iteratively update the intent tag set based on the new intent candidate tags to obtain an intent tag result set.

[0023] It should be noted that the execution subject of the method for automatically tuning the intent recognition model by mining new intent tags and fusing structured information provided in the embodiments of the present invention can be a server or a computer device, such as a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, etc.

[0024] In S110, the first incoming call data is data of a first incoming call that has not been recognized for intent, and the second incoming call data is data of a second incoming call whose intent recognition result does not match the actual intent of the user. The new intent tag mining model is constructed based on a large model.

[0025] Specifically, the first incoming call data refers to the first incoming call of a user, and when the real-time intent recognition model performs intent recognition on the first incoming call of this user, the call data determined by the model to be empty; the second incoming call data refers to the second incoming call of a user, and the predicted intent for the second incoming call does not hit, that is, the predicted intent for the second incoming call does not match the actual intent of the user.

[0026] Figure 2 is the second flow schematic diagram of the method for automatically tuning the intent recognition model by mining new intent tags and fusing structured information provided by the present invention. As Figure 2 shown, in the embodiments of the present invention, an initial intent tag set is obtained based on manual sorting ; Then, obtain the data for which the real-time intent recognition model determines the first call real-time intent model to be empty or the second call fails to hit the intent, that is, obtain the first call data or the second call data. After formatting and preprocessing, input it into the new intent label mining model constructed based on the large model, and automatically output the discovered new intent candidate labels 。

[0027] Figure 3 is the implementation framework diagram of the new intent label mining model provided by the present invention. As Figure 3 shown, in the specific implementation process, the business expert manually sorts out and determines the initial intent label set according to the actual customer call situation ; Input the initial intent label set , as well as the first call data / second call data into the real-time intent recognition model. The real-time intent recognition model outputs the intent hit result to Kafka in real time, and precipitates information such as the intent result and the call serial number to the file server (Secure File Transfer Protocol, abbreviated as SFTP). Obtain the data determined to be empty by the real-time intent recognition model IntentNullData={Text1,Text2,...Text i ,...Text n}, where Text i represents the call text data of the i-th call that fails to hit the intent. After preprocessing such as removing stop words and formatting, it is input into the new intent label mining model

[0028] Optionally, the cached intent result is in the key-value form. The key value structure is: REDI_Realtime_Intent_Secondary_Transfer_or_Incoming_Call, and the value format is: timestamp :{ "sceneAnswer":"Second-level intent label", "sceneName":"First-level intent label", "gcid":"Real-time transcription integration number", "serialNo":"Call serial number" }.

[0029] In the embodiment of the present invention, the new intent label mining model is constructed based on the large model. Specifically, input the data for which the intent is determined to be empty / the data for which the second call fails to hit, together with the preset Prompt hint, into the new intent label mining model constructed based on the large model, and the new intent label mining model automatically outputs the discovered new intent candidate labels 。

[0030] In some embodiments, assume a standard answer library , where is the i-th question, i.e., the i-th incoming call data, is the i-th answer, i.e., the intent label corresponding to the i-th incoming call data, then the constructed Prompt is: [{"Question":" ","Answer":" "}, {"Question":" ","Answer":" "} ...... {"Question":" ","Answer":" "}]。

[0031] In S120, based on the new intent candidate table circle, determine the intent set , append it to the original intent label set Tag, and combine to obtain a new intent label result set. That is to say, append to the initial intent label set to obtain the combined intent label set IntentTag_i = {Tag ∪ CheckTag_i}, where CheckTag_i represents the intent label set mined and generated when the new intent label mining model is executed for the i-th time, and Tag ∪ CheckTag_i represents the merger of the two label sets, and the merged label set is the updated intent label set.

[0032] In the embodiments of the present invention, continuously iterate and loop the real-time intent recognition and new intent label mining process until a preset stop condition is reached, such as reaching a preset number of iteration loops.

[0033] It can be understood that the setting of the number of iteration loops needs to meet the following conditions: after the iteration loop process is completed, the final intent label result set IntentTag covers all traffic, so as to realize intent recognition for all traffic.

[0034] In the specific implementation process, the new intent tag mining and real-time intent recognition processes are continuously iterated until an intent tag set IntentTag covering all traffic is obtained as the intent tag result set. Suppose the iteration loop is performed n times, then CheckTag = {CheckTag_1, CheckTag_2, CheckTag_3,..., CheckTag_n}, and IntentTag = {Tag ∪ CheckTag}, where Tag ∪ CheckTag represents the union of the two tag sets.

[0035] The method for mining new intent tags and automatically tuning the parameters of an intent recognition model by fusing structured information provided in the embodiments of the present invention inputs the first incoming call data or the second incoming call data into a pre-trained new intent tag mining model to obtain new intent candidate tags output by the new intent tag mining model. The first incoming call data is data of a first incoming call for which no intent has been recognized, and the second incoming call data is data of a second incoming call for which the intent recognition result does not hit; based on the new intent candidate tags, the intent tag set is updated; the intent tag set is iteratively updated until a preset iteration loop count is reached to obtain an intent tag result set; the intent tag result set is input into a real-time intent recognition model to obtain the user's incoming call intent output by the real-time intent recognition model. The present invention uses a new intent tag mining model constructed based on a large model to continuously iterate to obtain new intent tags, realizing the automatic discovery of new intents outside the intent system that has not been sorted out manually. On the one hand, it reduces the cost of manually sorting out the intent system, and on the other hand, it improves the intent coverage rate and comprehensiveness, thereby improving the accuracy of the intent recognition result.

[0036] In an alternative embodiment, after iteratively updating the intent tag set based on the new intent candidate tags to obtain an intent tag result set, it further includes: Inputting the intent tag result set and the incoming call data to be recognized into a real-time intent recognition model to obtain the user's incoming call intent output by the real-time intent recognition model; wherein, the real-time intent recognition model is determined based on the following steps: S210, initialize the knowledge base weights of multiple knowledge bases; S220, based on the knowledge base weights of the multiple knowledge bases, use sample data to train the large model to obtain iteratively updated knowledge base weights and large model parameters; S230, input the iteratively updated knowledge base weights and the large model parameters into a small model to obtain a predicted intent tag probability result output by the small model. The predicted intent tag probability result is the probability that the predicted incoming call intent belongs to each intent tag; S240. Based on the predicted intent label probability results, use the automatic parameter tuning module to iteratively train to obtain the final knowledge base weights and large model parameters, and complete the training of the real-time intent recognition model.

[0037] In the embodiment of the present invention, the intent label result set IntentTag and the incoming call data to be recognized are input into the real-time intent recognition model. The real-time intent recognition model performs intent recognition on the incoming call data to be recognized according to the intent label result set IntentTag, and outputs the intent recognition result in real time, that is, the user incoming call intent label.

[0038] Figure 4 is the implementation framework diagram of the real-time intent recognition model provided by the present invention. As Figure 4 shown, for the customer's first call, construct multi-source data input; construct a weight automatic update model to automatically update the optimal knowledge base weight combination and the corresponding large model parameters; use the optimal knowledge base weight combination and the corresponding large model parameters obtained after iterative update as features and input them into the small model to construct a full-process real-time intent recognition model with cooperation between the large and small models, and obtain the probability results of the small model predicting each intent label; construct an automatic parameter tuning module, and construct an optimal real-time intent recognition model based on the automatic parameter tuning module.

[0039] In S210, construct multiple knowledge bases, and assign initial knowledge base weights to each knowledge base in combination with expert experience. Taking the construction of three knowledge bases as an example, initial weights are assigned to each knowledge base in combination with expert experience 、 、 ,and = 1.

[0040] Optionally, the knowledge base includes an interface type knowledge base, a label type knowledge base, and an annotation data knowledge base. The interface type knowledge base is a database constructed based on interface type multi-source structured information. The label type knowledge base is a database constructed based on label type multi-source structured information. The annotation data knowledge base is an annotation example database constructed based on the call text content of speech-to-text conversion.

[0041] Specifically, the interface type knowledge base contains traffic-related data obtained from multiple interfaces through multi-source structured data acquisition. The label type knowledge base contains multiple traffic-related label data obtained through multi-source structured data acquisition. The annotation data knowledge base is obtained through expert annotation example data. Initial weights are assigned to the interface type knowledge base , the label type knowledge base , and the annotation data knowledge base respectively 、 、 ,and =1.

[0042] The new intent label mining and automatic parameter adjustment method for the intent recognition model by integrating structured information provided in the embodiment of the present invention not only considers single call text data, but also integrates structured information such as label class and interface class for intent recognition, so that the modeling information of the real-time intent recognition model is more comprehensive and rich, thereby improving the accuracy of intent recognition.

[0043] In the embodiment of the present invention, for the first call from a customer, a multivariate data input Data={( , , ),( , , ),( , , ),...,( , , )},in =( , , ) represents the multi-dimensional data of the i-th call, the transcribed text content of the call , Interface information and tag information .

[0044] In the specific implementation process, obtain the call text after voice transcription: obtain the real-time transcription text of each call through the voice transcription platform ={ ,..., ... },in Represents the jth sentence of the conversation text in the i-th call.

[0045] In the specific implementation process, develop an automatic intent configuration program to obtain the intent configuration file: develop a Redis-based program to automatically read the intent configuration information file on the self-configuration page, and automatically upload the intent configuration file IntentFile to the Object Storage Service (OSS). The format of the configuration file obtained from OSS is as follows: Use ngshView_realtime_province code as redis-key, and store map-key in redis-value in the format of map: product ID_activity\tID_intent Figure 2 Level name, map-value: product ID\tactivity ID\tscenario name\trule\tspeech 1 Dialogue 2 Dialogue 3\tScenario Link\tKnowledge ID. New change flag: ngshView_flag_1: 1 represents a change, 0 means no update.

[0046] Furthermore, build a multi-source structured information database and obtain interface class data Interface= , label class data Label= etc.; Examples of label class data are as follows: {"params":{"provinceCode":"Three-digit province code","userId":"Encrypted mobile phone number"},"callTime":"2024-11-13 16:20:28", "tagValue":"1,2"}, where tagValue is the intention label result, for example: 1 represents querying phone bill, 2 represents querying bill; Examples of interface class data are as follows: {"params":{"provinceCode":"Three-digit province code","userId":"Encrypted mobile phone number"},"callTime":"2024-11-13 16:20:28", "intentionNew":"Verify the reason for suspension"}, where intentionNew represents the intention information obtained from the interface class.

[0047] In S220, the sample data is a manually labeled intention dataset, that is, the standard answer library: build the standard answer library through manual labeling , where Si is the text of the i-th call text sentence, is the corresponding intention label. An example of the standard answer library is as Figure 5 shown.

[0048] In S220, use the large model to train and infer the data sample S= in the standard answer library D, and obtain the predicted label probability P(L|S)=f(S;θ) of the given sample S, where θ represents the model parameters; use the preset loss function to calculate the prediction error of the large model; use the preset algorithm to update the large model parameters, and adjust the knowledge base weights according to the contribution of each knowledge base.

[0049] Optionally, use the cross-entropy loss function to calculate the model prediction error, then for the i-th sample Si, the model prediction error (loss function) L(θ)= , where yi is the true label and f(Si;θ) is the predicted label probability.

[0050] Optionally, use the gradient descent algorithm to update the model parameters and knowledge base weights, then = - , where represents the model parameters in the next iteration step, represents the model parameters in the current iteration step, represents the learning rate, represents the gradient of the loss function L(θ) with respect to the parameters; for the knowledge base weights, , where α represents the learning rate for weight update, represents the gradient of the loss function with respect to the weight And perform weight normalization, then .

[0051] In S230, the best knowledge base weight combination obtained after iterative update and the corresponding large model parameters θ are used as features to be input into the small model, and a full-process real-time intent recognition model that coordinates the large and small models is constructed to obtain the predicted intent label probability results output by the small model, that is, the probabilities that the predicted incoming call intent belongs to each intent label, so as to determine the predicted incoming call intent label.

[0052] In S240, an optimal real-time intent recognition model is constructed based on the automatic tuning module, and iterative training is performed according to the error between the predicted incoming call intent label and the true intent label. Substituting the final knowledge base weights and large model parameters obtained from the iterative training, the finally trained real-time intent recognition model is obtained.

[0053] The new intent label mining and automatic tuning method for the intent recognition model by fusing structured information provided by the embodiments of the present invention adopts the method of fusing large and small models, designs an automatic tuning model to obtain the best real-time intent recognition model, can effectively utilize the respective advantages of the large model and the small model, and achieves a better application effect. It has advantages over the existing method of using only the large model or the small model for intent recognition. It can not only solve the problem that the large model cannot meet the requirements of high-timeliness scenarios, but also solve the problem of insufficient reasoning ability of the small model, and realize high-precision real-time intent recognition.

[0054] In an optional embodiment, the iterative training to obtain the final knowledge base weights and large model parameters based on the predicted intent label probability results using the automatic tuning module includes: S241, initializing the weights of the small model for predicting each of the intent labels; S242, inputting the weights of each of the intent labels into the automatic tuning module, and the automatic tuning module updates the large model parameters and the weights of each of the intent labels using the gradient descent algorithm; S243. The automatic parameter adjustment module adjusts the knowledge base weights of each knowledge base based on the contributions of each knowledge base, and the contribution of the knowledge base reflects the degree of influence of the knowledge base on the loss value. S244. The automatic parameter adjustment module iteratively updates the large model parameters, the weights of each intent label, and the knowledge base weights of each knowledge base until the weights of each intent label reach the preset optimal weights.

[0055] In S241, the weights of each intent label predicted by the initialized small model are IntentWight = { ,..., ,..., }, where n represents the number of types of intents, that is, the number of intent labels in the intent label result set, and .

[0056] In S242, the weights of each intent label are input into the intent automatic parameter adjustment module constructed based on the large model. For the standard answer knowledge base , the model prediction error is calculated using the preset loss function, and the model parameters and the weights of each intent label are updated using the gradient descent algorithm.

[0057] Specifically, for the standard answer knowledge base , when the number of categories is large, compared with the cross-entropy loss function, the mean squared error loss function does not have strict requirements for probability output and can handle errors more smoothly. Therefore, the mean squared error loss function is used to calculate the model prediction error. Then, for the i-th sample Si, the model prediction error (loss function) LM(θ) = , where is the true label, and is the predicted label.

[0058] Here, the gradient descent algorithm is used to update the large model parameters and the weights of each intent label. Then = - , where represents the model parameters in the next iteration step, represents the model parameters in the current iteration step, represents the learning rate, and represents the gradient of the loss function LM(θ) with respect to the parameters.

[0059] In S243, during the gradient descent optimization process, the gradient reflects the trend of the loss function changing with the weights. A larger gradient indicates that the knowledge base has a greater impact on the current loss, and thus its importance can be inferred. The weights are adjusted according to the contribution of each knowledge base. Then , where α represents the learning rate of weight update, Denote the gradient of the loss function with respect to the weights . And perform weight normalization, then .

[0060] In S244, the automatic parameter tuning module iteratively updates the large model parameters, the weights of the intent labels, and the knowledge base weights until the weights of each intent label reach the optimal weights OPWeight = { ,..., ,..., } obtained by manually sorting out each intent, and obtain a real-time intent recognition model with high accuracy.

[0061] The method for mining new intent labels and fusing structured information to automatically tune the intent recognition model provided by the embodiments of the present invention effectively improves the intent recognition accuracy by jointly training the large model and the small model through the automatic parameter tuning module and automatically learning to obtain the optimal real-time intent recognition model.

[0062] In an optional embodiment, the method for mining new intent labels and fusing structured information to automatically tune the intent recognition model further includes: If a user with the same number makes a call again within a preset duration, determine that the call is a second call; Based on the intent result of the user's previous call with the number, determine the first intent set of the user, where the first intent set contains multiple intent labels; Based on the interface class information corresponding to the number, determine the second intent set of the user, where the second intent set contains multiple intent labels; Based on the label class information corresponding to the number, determine the third intent set of the user, where the third intent set contains multiple intent labels; Based on the first intent set, the second intent set, and the third intent set, determine the intent result set of the user.

[0063] In the embodiments of the present invention, design the Kafka data format for real-time intent model precipitation, and upload it to the file server (Secure File Transfer Protocol, abbreviated as SFTP) to send information such as call text, whether the intent is hit, and the hit intent result to Kafka in real time; pull data from Kafka by the data processing program to extract the real-time intent result data to SFTP daily to support customer second call analysis and intent report analysis applications.

[0064] In some embodiments, the Kafka data format is as follows: {"requestTime":"2024-12-04 16:26:38.081","classCode":"RealTimeTranferController","timeConsuming":17,"modelTimeConsuming":17,"modelRequestTime":"2024-12-04 16:26:38.081","modelReturnTime":"2024-12-04 16:26:38.098","modelResponse":"{\"bean\":{\"sceneId\":\"\",\"sceneCustom\":\"\",\"sceneAnswer\":\"Consultation on family numbers\",\"sceneUrl\":\"\",\"sceneName\":\"Service scenario\"},\"rtnCode\":\"0\",\"rtnMsg\":\"Success\"}","appCode":"realTimeTranfer","otherParams":"{\"callId\":\"59xxxxxxxxxxxx\",\"sentence\":\"Do you mean the family numbers in the family package or the main number?\",\"provCode\":\"xxxxx\",\"index\":\"1\",\"contactType\":\"10086online\",\"callType\":\"2\",\"dataTime\":\"2024 / 12 / 04 16:26:37\",\"speaker\":\"0\",\"startTime\":\"2024 / 12 / 04 16:26:19\",\"endTime\":\"\",\"staffId\":\"xxxx\",\"subsNumber\":\"xxx\",\"channelCode\":\"intellwords\",\"tel\":\"xxxxxx\",\"intentionName\":\"Do you mean the family numbers in the family package or the main number?\"}","returnTime":"2024-12-04 16:26:38.098"}。

[0065] In some embodiments, the SFTP landing data format is as follows: Each field information is separated by a vertical bar, and the precipitated information includes content such as province, channel, contact point, call text, serial number, intention hit result, etc. The specific format is as Figure 6As shown: Serial number | text sentence | province | contact | first-level intent name | second-level intent name | speech... The intent report contains information such as date, province code, province name, hit intent, number of seats used, and number of hit intent calls per day.

[0066] In the embodiment of the present invention, the secondary call analysis application is supported based on real-time intent results, tag classes, interface classes and other information. Specifically, the intent result information of the customer's previous call is cached by Redis. When the customer with the same mobile phone number calls again within a preset time (such as 24 hours), it is determined to be a repeated call. If this mobile phone number does not have more than one call within the preset time (24 hours), it is determined to be a non-repeated call.

[0067] In the embodiment of the present invention, the secondary incoming call intention recognition is performed based on the following steps: first, the cached last call intention result is output to obtain the intention result set SentenceIntent={SI1, SI2, ... SI i ,..SI n}; The customer intent set to obtain interface information based on the mobile phone number is InterfaceIntent={II1,II2,...II i ,...II n}、According to the mobile phone number, the intent set of the label class is LabelIntent={LI1,LI2,...LI i ,...LI n}, where SI i represents the i-th intent obtained based on the mobile phone number and call text, II i Represents the customer intention set based on mobile phone number and interface type results, LI i Represents the i-th label class intent obtained based on the mobile phone number and label class results; the final intent result set FinalIntent of the customer's second call is obtained based on the SentenceIntent, InterfaceIntent, and LabelIntent results. For example, the real-time intent result of a customer's last call is broadband fault reporting. According to the interface class and label class information, it is obtained that the customer is a broadband user and the broadband status is single fault. The output interface class and label class intent results are also broadband fault reporting. Then it is predicted that the intent of the same customer's second call is broadband fault reporting, so the call traffic of this customer is assigned to the broadband special seat for handling and traffic diversion. In addition, the mobile phone number of the customer's second call, whether it hits various intent results, and other information are precipitated, and the result data of the second call that did not hit the intent is transmitted to the new intent label mining model to quickly discover new intent labels.

[0068] The method for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information provided by the embodiments of the present invention caches the intent results in Redis, thereby predicting the intent of repeated user calls, achieving call traffic diversion, and reducing the cost of manual call answering.

[0069] Further, determining the intent result set of the user based on the first intent set, the second intent set, and the third intent set includes: If the intersection of the first intent set, the second intent set, and the third intent set is not empty, determining the intent result set as the intersection of the first intent set, the second intent set, and the third intent set; If the intersection of the first intent set, the second intent set, and the third intent set is empty, determining the intent result set as the union of the first intent set, the second intent set, and the third intent set.

[0070] Specifically, if SentenceIntent, InterfaceIntent, and LabelIntent are all not empty, and SentenceIntent ∩ InterfaceIntent ∩ LabelIntent is not empty, then FinalIntent = SentenceIntent ∩ InterfaceIntent ∩ LabelIntent, and the result of FinalIntent is used as the intent of the customer's second call. If FinalIntent is empty, considering the diversity of obtaining intents based on information such as call texts, interface classes, and label classes, in order to improve the hit rate of the intent of the customer's second call, then determine FinalIntent = SentenceIntent ‌∪ InterfaceIntent ‌∪ LabelIntent.

[0071] The method for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information provided by the embodiments of the present invention directly uses the intersection as the intent of the user's second call when there is an intersection among the intent result set, the interface class intent set, and the label class intent set, thereby quickly determining the user's intent; when there is no intersection among the three, the union of the three is used as the intent of the user's second call, thereby improving the hit rate of the intent recognition of the user's second call.

[0072] Next, a device for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information provided by the embodiments of the present invention will be described. The device for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information described below can be correspondingly referred to the method for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information described above.

[0073] Figure 7 This is a schematic structural diagram of an apparatus for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information provided by the present invention. As Figure 7 shown, the apparatus for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information may include, but is not limited to: A tag mining module 710, configured to: input first incoming call data or second incoming call data, and an intent tag set into a pre-trained new intent tag mining model, to obtain new intent candidate tags output by the new intent tag mining model, where the first incoming call data is data of a first incoming call for which no intent is recognized, the second incoming call data is data of a second incoming call for which the intent recognition result does not match the actual intent of the user, and the new intent tag mining model is constructed based on a large model; A tag update module 720, configured to: iteratively update the intent tag set based on the new intent candidate tags to obtain an intent tag result set.

[0074] It should be noted that the apparatus for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information provided in the embodiments of the present invention, during specific operation, may execute the method for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information described in any of the above embodiments, and details thereof are not described in this embodiment.

[0075] Figure 8 Illustrates a schematic structural diagram of an electronic device. As Figure 8 shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call logic instructions in the memory 830 to execute the method for automatically tuning parameters of an intent recognition model by mining new intent tags and integrating structured information, and the method includes: inputting first incoming call data or second incoming call data, and an intent tag set into a pre-trained new intent tag mining model, to obtain new intent candidate tags output by the new intent tag mining model, where the first incoming call data is data of a first incoming call for which no intent is recognized, the second incoming call data is data of a second incoming call for which the intent recognition result does not match the actual intent of the user, and the new intent tag mining model is constructed based on a large model; iteratively updating the intent tag set based on the new intent candidate tags to obtain an intent tag result set.

[0076] In addition, when the logical instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the new intent label mining and fusion of structured information for automatic tuning of the intent recognition model method provided by the above-mentioned various methods. The method includes: inputting the first incoming call data or the second incoming call data, and the intent label set into a pre-trained new intent label mining model to obtain new intent candidate labels output by the new intent label mining model. The first incoming call data is data of a first incoming call for which no intent is recognized, and the second incoming call data is data of a second incoming call for which the intent recognition result does not match the actual intent of the user. The new intent label mining model is constructed based on a large model; based on the new intent candidate labels, iteratively update the intent label set to obtain an intent label result set.

[0078] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method of new intent label mining and fusion of structured information for automatic tuning of the intent recognition model provided by the above-mentioned various methods. The method includes: inputting the first incoming call data or the second incoming call data, and the intent label set into a pre-trained new intent label mining model to obtain new intent candidate labels output by the new intent label mining model. The first incoming call data is data of a first incoming call for which no intent is recognized, and the second incoming call data is data of a second incoming call for which the intent recognition result does not match the actual intent of the user. The new intent label mining model is constructed based on a large model; based on the new intent candidate labels, iteratively update the intent label set to obtain an intent label result set.

[0079] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0080] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for mining new intent labels and integrating structured information to automatically adjust the parameters of an intent recognition model, characterized in that: include: Inputting the first incoming call data or the second incoming call data and the intent label set into a pre-trained new intent label mining model to obtain a new intent candidate label output by the new intent label mining model, wherein the first incoming call data is data of a first incoming call and no intent is recognized, and the second incoming call data is data of a second incoming call and the intent recognition result does not match the actual intent of the user, and the new intent label mining model is constructed based on a large model; Based on the new intent candidate label, the intent label set is iteratively updated to obtain an intent label result set.

2. According to claim 1, the method for mining new intent labels and integrating structured information to automatically adjust the parameters of the intent recognition model is characterized in that: After the intent tag set is iteratively updated based on the new intent candidate tag to obtain the intent tag result set, the method further includes: Input the intention label result set and the incoming call data to be identified into a real-time intention recognition model to obtain the user's incoming call intention output by the real-time intention recognition model; The real-time intention recognition model is determined based on the following steps: Initialize the knowledge base weights of multiple knowledge bases; Based on the knowledge base weights of the multiple knowledge bases, the large model is trained using sample data to obtain iteratively updated knowledge base weights and large model parameters; Input the iteratively updated knowledge base weights and the large model parameters into the small model to obtain a predicted intent label probability result output by the small model, wherein the predicted intent label probability result is the probability that the predicted incoming call intent belongs to each intent label; Based on the predicted intent label probability results, an automatic parameter adjustment module is used to iteratively train to obtain the final knowledge base weights and large model parameters to complete the training of the real-time intent recognition model.

3. The method for automatically adjusting parameters of an intention recognition model by mining new intention labels and fusing structured information according to claim 2 is characterized in that: Based on the predicted intention label probability result, the automatic parameter adjustment module is used to iteratively train to obtain the final knowledge base weights and large model parameters, including: Initialize the small model to predict the weight of each of the intent labels; The weight of each of the intent labels is input into the automatic parameter adjustment module, and the automatic parameter adjustment module uses a gradient descent algorithm to update the large model parameters and the weight of each of the intent labels; The automatic parameter adjustment module adjusts the knowledge base weight of each knowledge base based on the contribution of each knowledge base, and the contribution of the knowledge base reflects the influence of the knowledge base on the loss value; The automatic parameter adjustment module iteratively updates the large model parameters, the weight of each of the intent labels, and the knowledge base weight of each of the knowledge bases until the weight of each of the intent labels reaches a preset optimal weight.

4. The method for automatically adjusting parameters of an intent recognition model by mining new intent labels and fusing structured information according to claim 2 or 3, characterized in that: The knowledge base includes an interface knowledge base, a tag knowledge base and an annotated data knowledge base. The interface knowledge base is a database constructed based on multi-source structured information of the interface type, the tag knowledge base is a database constructed based on multi-source structured information of the tag type, and the annotated data knowledge base is an annotated sample database constructed based on the text content of calls transcribed from speech.

5. The method for mining new intent labels and integrating structured information to automatically adjust the parameters of the intent recognition model according to claim 4 is characterized in that: The method for mining new intent labels and fusing structured information to automatically adjust the parameters of the intent recognition model also includes: If the user with the same number calls again within the preset time, the call is determined to be a second call; Determine a first intent set of the user based on the intent result of the last incoming call to the number, where the first intent set includes multiple intent tags; Determine a second intent set of the user based on the interface class information corresponding to the number, where the second intent set includes multiple intent tags; Determine a third intent set of the user based on the tag information corresponding to the number, where the third intent set includes multiple intent tags; Based on the first intent set, the second intent set and the third intent set, determine the user's intent result set.

6. The method for automatically adjusting parameters of an intention recognition model by mining new intention labels and fusing structured information according to claim 5 is characterized in that: The method further comprises: The call text and the user's incoming call intention hit results are sent to Kafka in real time, so that Kafka can upload the call text and the user's incoming call intention hit results to the file server on a daily basis to generate an intent analysis report.

7. A device for mining new intent labels for an intent recognition device and fusing structured information to automatically adjust the parameters of an intent recognition model, characterized in that: include: A label mining module is used to: input the first incoming call data or the second incoming call data, and the intent label set into a pre-trained new intent label mining model to obtain a new intent candidate label output by the new intent label mining model, wherein the first incoming call data is data of a first incoming call and no intent is recognized, and the second incoming call data is data of a second incoming call and the intent recognition result does not match the actual intent of the user, and the new intent label mining model is constructed based on a large model; The label updating module is used to iteratively update the intent label set based on the new intent candidate label to obtain an intent label result set.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the method for automatically adjusting the parameters of the intent recognition model by mining new intent labels and fusing structured information as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for automatically adjusting the parameters of the intent recognition model by mining new intent labels and fusing structured information as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the method for automatically adjusting the parameters of the intent recognition model by mining new intent labels and fusing structured information as described in any one of claims 1 to 6.

Images