Language recognition model training method, language recognition method and related equipment

Abstract

The invention provides a language recognition model training method, a language recognition method and related equipment, and the method comprises the steps: determining a target statement containing a preset field, marking a first field value of the preset field, obtaining a sample statement, and obtaining a language recognition model through the sample statement and reference recognition accuracy. And training the language recognition model for N times to obtain the corresponding recognition accuracy after the Nth training, and when the corresponding recognition accuracy after the Nth training is greater than an accuracy threshold, taking the language recognition model obtained by the Nth training as a target language recognition model. According to the method, the language recognition model learns the sample statement based on the reference recognition accuracy, and the target language recognition model is determined by comparing the recognition accuracy with the accuracy threshold, so that the accurate recognition of the field value of the statement to be detected is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a language recognition model training method, a language recognition method and related equipment. Background Art

[0002] With the development of data processing technology, a large number of financial businesses are completed on the system, and the financial field involves many specific amount values, which can be balances, fees, etc. Currently, these amount values ​​need to be processed by manual customer service. As the business volume increases, the number of manual customer service also needs to increase, which will increase the labor cost. In order to solve the above problems, a language recognition model is introduced. The language recognition model is used to query the user's query statement. The speech recognition model can determine the amount value corresponding to the query statement. Summary of the invention

[0003] The present application provides a language recognition model training method, a language recognition method and related equipment to improve the recognition accuracy of a target language recognition model.

[0004] In a first aspect, the present application provides a language recognition model training method, the method comprising:

[0005] Determine a target sentence containing a preset field, and mark a first field value of the preset field to obtain a sample sentence; train a language recognition model N times using the sample sentence to obtain a recognition accuracy corresponding to the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model in recognizing the first field value, and N is an integer greater than 1;

[0006] When the recognition accuracy corresponding to the Nth training is greater than the accuracy threshold, the language recognition model obtained by the Nth training is used as the target language recognition model.

[0007] Through the above method, the sample sentences marked with the first field value are used to train the language recognition model, and the model training is stopped only when the recognition accuracy after training is greater than the accuracy threshold. The language recognition model trained for the last time is used as the target language recognition model to avoid screening out the language recognition model that inaccurately recognizes the sample sentences, thereby ensuring the accuracy of the target language recognition model.

[0008] In a second aspect, the present application provides a language recognition method, the method comprising:

[0009] Determine the sentence similarity between the sentence to be detected and the sample sentence in the target database, wherein the target database contains sample sentences corresponding to various business scenarios, each sample sentence contains a preset field and a field value corresponding to the preset field;

[0010] Filter out a target sentence similarity greater than a preset threshold from the sentence similarities, and determine a target sample sentence corresponding to the target sentence similarity;

[0011] The target sample sentence is input into a language recognition model to obtain the field value of the sentence to be detected.

[0012] Through the above method, the target sample sentence with the greatest similarity to the sentence to be detected is determined from the target database, and then the target sample sentence is input into the language recognition model for inference, thereby ensuring the accuracy of the field value of the output sentence to be detected.

[0013] In a third aspect, the present application provides a language recognition model training device, the device comprising:

[0014] A determination module, used to determine a target sentence containing a preset field, and mark a first field value of the preset field to obtain a sample sentence;

[0015] A training module, which trains the language recognition model N times through the sample sentences to obtain a recognition accuracy corresponding to the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model in recognizing the first field value, and N is an integer greater than 1;

[0016] The target module is used to use the language recognition model obtained by the Nth training as the target language recognition model when the recognition accuracy corresponding to the Nth training is greater than the accuracy threshold.

[0017] In a fourth aspect, the present application provides a language recognition device, the device comprising:

[0018] An acquisition module is used to determine the sentence similarity between the sentence to be detected and the sample sentence in the target database, wherein the target database contains sample sentences corresponding to various business scenarios, each sample sentence contains a preset field and a field value corresponding to the preset field;

[0019] A screening module, used to screen out target sentence similarities greater than a preset threshold from the sentence similarities, and determine a target sample sentence corresponding to the target sentence similarity;

[0020] The output module is used to input the target sample sentence into the language recognition model to obtain the field value of the sentence to be detected.

[0021] In a fifth aspect, the present application provides an electronic device, including:

[0022] Memory, used to store computer programs;

[0023] The processor is used to implement the above-mentioned language recognition model training method steps and language recognition method steps when executing the computer program stored in the memory.

[0024] In a sixth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned language recognition model training method steps and language recognition method steps.

[0025] For each aspect from the first to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here.

[0026] In a seventh aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the above-mentioned language recognition model training method and language recognition method steps.

[0027] For each of the above-mentioned aspects from the first to the seventh aspects and the technical effects that may be achieved by each of the aspects, please refer to the above-mentioned description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of an application environment for the language recognition model training method provided in this application;

[0029] Figure 2 A flow chart of the steps of a language recognition model training method provided in this application;

[0030] Figure 3 A flow chart of the steps of a language recognition method provided in this application;

[0031] Figure 4 A structural diagram of a language recognition model training device provided in this application;

[0032] Figure 5 A schematic diagram of the structure of a language recognition device provided in this application;

[0033] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0035] See also Figure 1 , Figure 1 A schematic diagram of an application environment of the language recognition model training method provided in an embodiment of the present application is shown. As an implementation mode, the data optimization method provided in the embodiment of the present application can be applied to an electronic device. The electronic device can be as follows: Figure 1 The server 110 shown in the figure can be connected to the terminal device 120 through a network. The network is used to provide a medium for a communication link between the server 110 and the terminal device 120. The network can include various connection types, such as a wired communication link, a wireless communication link, etc., which are not limited in the embodiments of the present application. Optionally, in other embodiments, the electronic device can also be a smart phone, a laptop computer, etc.

[0036] It should be understood that Figure 1 The server 110, network and terminal device 120 are merely illustrative. Depending on the implementation requirements, there may be any number of servers, networks and terminal devices. For example, the server 110 may be a physical server or a server cluster composed of multiple servers, and the terminal device 120 may be a mobile phone, a tablet, a desktop computer, a laptop computer, and the like. It is understood that the embodiments of the present application may also allow multiple terminal devices 120 to access the server 110 at the same time.

[0037] Based on the above application scenarios, the server can realize the function of intelligent customer service, that is, to identify the sentences to be detected input by the customer, so as to determine the customer's intention, and reply to the sentences to be detected based on the customer's intention.

[0038] For the current intelligent customer service, intelligent customer service is used to replace manual customer service in various business scenarios to realize the dialogue between customers. The specific implementation process of the dialogue function is as follows:

[0039] First, the intelligent customer service receives the sentence sent by the customer, and uses the intent recognition model to identify the intent corresponding to the sentence. The intent is a word or a short sentence, and the intent is determined based on the operation indicated by the user. After determining the intent corresponding to the sentence, based on the correspondence between the intent and the preset field, the preset fields corresponding to the intent are determined, and the similarity between the determined preset fields and the intent is calculated to obtain multiple similarities. Then, the maximum similarity is screened from all the similarities, and the preset field corresponding to the maximum similarity is determined. The preset field corresponding to the maximum similarity is used as the target preset field, and the target preset field has a corresponding relationship with the business scenario. Therefore, the business scenario corresponding to the target preset field is used as the target business scenario corresponding to the sentence, thereby realizing the division of the target business scenario of the sentence, and then the field value corresponding to the target preset field is queried from the database. The database records the preset fields corresponding to various business scenarios and the field values ​​corresponding to each preset field. After determining the field value corresponding to the target preset field, the target preset field and the field value are filled into the reply template corresponding to the target business scenario to generate a reply sentence, and the intelligent customer service sends the reply sentence to the customer.

[0040] For example, when the intelligent customer service receives the question "How much money did I have left last month?", the intent recognition model is used to identify that the intent corresponding to the sentence is "query last month's balance", and the preset fields corresponding to "query last month's balance" include at least: last month's balance, query time, query balance, remaining balance, monthly remaining balance, etc. The similarity between "query last month's balance" and "query time, query balance, remaining balance, monthly remaining balance, etc." is calculated respectively to obtain multiple similarities, and then the maximum similarity is selected from all similarities. If the preset field corresponding to the maximum similarity is "last month's balance", the business field corresponding to "last month's balance" is The business scenario is business scenario A, "last month's balance" is used as the target preset field, and business scenario A is used as the target business scenario corresponding to the statement. Then, the corresponding field value of "last month's balance" is found from the database as 2,000 yuan. Then, the reply template corresponding to the target business scenario is determined. The reply template can be "Hello, the total balance is xxx yuan, and the expenditure is xxx yuan". Fill "last month's balance" and "2,000 yuan" into the reply template to generate a reply statement, which is: "Hello, the total balance of last month is 2,000 yuan, and the expenditure is 300 yuan". The intelligent customer service then sends the reply statement to the customer.

[0041] The above-mentioned sentence corresponds to the target business scenario in the intent recognition model. When a sentence to be detected corresponding to a new business scenario appears, in order to ensure the accuracy of the intent recognition model, it is necessary to re-collect all historical data corresponding to the new business scenario, and manually mark the corresponding intent in each historical data, and then re-train the intent recognition model based on all the marked historical data until the intent recognition model can accurately identify the intent corresponding to each historical data. This training process will consume a lot of time and manpower costs. At the same time, when the new business scenario lacks a corresponding reply template, the intelligent customer service will randomly select an existing reply template for the new business scenario, and the content of the reply in the existing reply template will be different from the content required to be replied to the new business scenario, resulting in inaccurate responses by the intelligent customer service to the question statements entered by the user.

[0042] In order to solve the above-described problems, the embodiment of the present application provides a language recognition model training method, the method comprising: determining a target sentence containing a preset field, and marking the first field value of the preset field to obtain a sample sentence; training the language recognition model N times through the sample sentence to obtain the corresponding recognition accuracy after the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model identifying the first field value, N is an integer greater than 1, and when the recognition accuracy corresponding to the Nth training is greater than the accuracy threshold, the language recognition model obtained by the Nth training is used as the target language recognition model. Through this method, the accuracy of the field value recognized by the language recognition model is used as the training termination condition, thereby ensuring the accuracy of the output result of the language recognition model finally trained, thereby ensuring that the target language recognition model can accurately output the field value corresponding to the sentence to be detected, so that the intelligent customer service can accurately reply to the question sentence input by the user based on the field value. Among them, the method and device described in the embodiment of the present application are based on the same technical concept. Since the principles of the problems solved by the method and the device are similar, the embodiments of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0043] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0044] Reference Figure 2 The present application provides a language recognition model training method, which can improve the accuracy of the target language recognition model. The implementation process of the method is as follows:

[0045] Step S21: determining a target sentence containing a preset field, and marking a first field value of the preset field to obtain a sample sentence.

[0046] In order to train the language recognition model, it is necessary to obtain an initial sentence for training the language recognition model. The initial sentence is a question sentence input by a user in various business scenarios. The initial sentence can come from a third-party data platform. In the embodiment of the present application, in order to ensure the accuracy of the language recognition model, the initial sentence can be processed to obtain a sample sentence. The process of determining the sample sentence is as follows:

[0047] Since the recognition degree of the initial sentence directly recognized by the language recognition model is inaccurate, in an embodiment of the present application, the initial sentence can also be clustered, and the sentence at the center of each cluster after clustering is used as the target sentence. The clustering here is to cluster the initial sentence into various clusters. The categories of the initial sentences of different clusters are different. In an embodiment of the present application, the sentence at the center of the cluster (target sentence) can be marked with a category. The category mark is used to characterize different business scenarios, such as: balance calculation scenario, life expenditure scenario, arrears scenario, etc. The specific business scenario can be adjusted according to the actual situation, which is only used as an example here.

[0048] In the embodiment of the present application, the clustering method can be a k-means clustering algorithm (full name in English: k-means clustering algorithm, abbreviated as: K-means). The clustering algorithm in practical applications can be selected according to the specific situation, which will not be elaborated here one by one.

[0049] After the target sentence is determined, in order to mark the target sentence, it is necessary to determine the preset fields in the target sentence.

[0050] In an embodiment of the present application, the preset field may be a pre-configured keyword or key word, and thus the preset field may be determined directly in the target sentence by identifying the keyword or key word.

[0051] The first field value corresponding to the preset field is further determined through the preset field in the target statement. The first field value may be the numerical value corresponding to the preset field. For example, if the preset field is "balance", the first field value corresponding to the preset field is "500".

[0052] After determining the first field value corresponding to the preset field, the first field value can be annotated by the system so that the first field value can be labeled, thereby obtaining a sample sentence ultimately used to train the language recognition model.

[0053] Based on the above method, the initial sentence is processed to obtain a target sentence with a category identifier, and then the field value in the target sentence is accurately standardized to ensure that the sample sentence can be accurately obtained.

[0054] Step S22: training the language recognition model N times using sample sentences to obtain the corresponding recognition accuracy after the Nth training.

[0055] After the sample sentences are determined, the sample sentences are imported into the language recognition model for training, that is, the language recognition model is trained based on the sample sentences. In one training process, the language recognition model will output the second field value corresponding to the preset field of the sample sentence.

[0056] Since the first field value corresponding to the preset field in the sample sentence is marked in advance, it is possible to determine whether the prediction of the sample sentence by the language recognition model is accurate by comparing whether the first field value is consistent with the second field value. After one training is completed, the recognition accuracy of the sample sentence is calculated based on the first field value and the second field value. The specific method of calculating the recognition accuracy is as follows:

[0057] Method 1:

[0058] A first number of sample sentences is determined, and a second number of sample sentences whose first field value is consistent with the second field value is determined, and a ratio of the second number to the first number is calculated, and the ratio is used as the recognition accuracy.

[0059] For example: the first number of sample sentences is 10, and 10 sample sentences are imported into the language recognition model to obtain the second field value corresponding to each sample sentence. The first field value of the sample sentence is compared with the second field output after importing the language recognition model, and it is determined that the second number of sample sentences whose first field value is consistent with the second field value is 2 (that is, there are 2 sample sentences whose first field value is consistent with the second field value). Then the recognition accuracy of the language recognition model is 2 / 10=20%.

[0060] Method 2:

[0061] The recognition accuracy is calculated based on the sentence complexity.

[0062] Specifically, first determine the sentence complexity of each sample sentence. The sentence complexity may be the complexity of the language structure of the sentence. The more language structures in the sample sentence, the more complex the sample sentence. For example, a sample sentence with a first-level language structure only includes: subject, predicate, and object. A sample sentence with a second-level language structure only includes: subject, predicate, object, and attributive. The sentence complexity may also be the number of character categories included in the sample sentence. The more character categories, the higher the sentence complexity of the sample sentence. The character categories may include: numbers, letters, Chinese, etc. For example, a sample sentence with a first-level sentence complexity only includes numbers and letters. A sample sentence with a second-level sentence complexity only includes numbers, letters, and Chinese.

[0063] Then, the total number of sample sentences corresponding to the sentence complexity (i.e., the third number) is determined from all sample sentences, and the total number of sample sentences whose first field values ​​are consistent with the second field values ​​(i.e., the fourth number) is determined from the sample sentences corresponding to the sentence complexity, and the ratio of the fourth number to the third number is calculated, and the ratio is used as the recognition accuracy.

[0064] For example: the number of all sample sentences is 10, among which the number of sample sentences with sentence complexity level 1 is 5, and the number of sample sentences with sentence complexity level 2 is also 5. Among the sample sentences with sentence complexity level 1, the number of sample sentences in which the first field value is consistent with the second field value is 3. Therefore, the recognition accuracy corresponding to the sentence complexity level 1 is 3 / 5=60%; among the sample sentences with sentence complexity level 2, the number of sample sentences in which the first field value is consistent with the second field value is 1. Therefore, the recognition accuracy corresponding to the sentence complexity level 2 is 1 / 5=20%.

[0065] The embodiment of the present application can use the above method 1 or method 2 to calculate the recognition accuracy, which will not be elaborated here. After calculating the recognition accuracy, when the recognition accuracy is lower than the accuracy threshold, the sample sentence is input into the language recognition model for retraining. The specific training method is as follows:

[0066] Method 1: Train the language recognition model N times.

[0067] Import all sample sentences into the language recognition model and perform N trainings, where N is a positive integer. Each time the language recognition model is trained, a recognition accuracy will be obtained. When the language recognition model is trained for the Nth time, N recognition accuracies can be obtained.

[0068] Method 2: Train the language recognition model based on the sentence complexity of sample sentences.

[0069] After all sample sentences are input into the language recognition model for training, the second field value corresponding to each sample sentence is output, and each second field value is compared with the first field value marked in the corresponding sample sentence. When the recognition accuracy is determined, all sample sentences can be classified according to the sentence complexity, and the recognition accuracy corresponding to each sentence complexity can be calculated respectively. When there is a recognition accuracy lower than the accuracy threshold, the sample sentences corresponding to the sentence complexity are input into the language recognition model for retraining, so as to obtain a new recognition accuracy until the recognition accuracy is higher than the accuracy threshold.

[0070] Method 3: Classify all sample sentences according to sentence complexity, and then input sample sentences corresponding to the same sentence complexity into the language recognition model for training, and obtain the recognition accuracy of the language recognition model for the sample sentences corresponding to the sentence complexity.

[0071] For example: the total number of sample sentences is 20, the total number of sample sentences with sentence complexity level one is 10, and the total number of sample sentences with sentence complexity level two is also 10. All sample sentences with sentence complexity level one are imported into the language recognition model for training, and the second field value corresponding to the preset field of each sample sentence is output. Then each second field value is compared with the first field value marked in the corresponding sample sentence, and it is determined that the total number of sample sentences with the same first field value is 4. The recognition accuracy corresponding to the simple sentence complexity is 4 / 10=40%.

[0072] Through the above method, the sample sentences are trained N times based on the language recognition model, so that the language recognition model repeatedly learns the sample sentences, and determines the recognition accuracy of the sample sentences by the language recognition model each time it is trained, which is conducive to ensuring that the language recognition model can accurately recognize the sample sentences.

[0073] Step S23: When the recognition accuracy after the Nth training is greater than the accuracy threshold, the language recognition model obtained by the Nth training is used as the target language recognition model.

[0074] After training the language recognition model multiple times, the target language recognition model can be determined in the following way:

[0075] In a possible example, the training of the language recognition model N times includes: inputting the sample sentence into the language recognition model obtained after the i-th training to obtain a second field value corresponding to the preset field, where i is a positive integer less than N; matching the first field value with the second field value to determine the recognition accuracy; when the recognition accuracy is lower than the accuracy threshold, inputting the sample sentence into the language recognition model for the i+1th training.

[0076] The purpose of each training is to make the second field value of the sample sentence output by the language recognition model close to the first field value of the sample sentence. After the i-th training, the sample sentence is input into the language recognition model obtained after the i-th training to obtain the second field value corresponding to the i-th preset field. The second field value obtained after the i-th training is matched with the first field value to obtain the recognition accuracy of the language recognition model after the i-th training. If the recognition accuracy is lower than the accuracy threshold, the speech recognition model needs to be trained for the i+1th time so that the corresponding recognition accuracy of the trained language recognition model can be higher than the accuracy threshold. Wherein, i is a positive integer less than N.

[0077] In a possible example, a language recognition model is trained N times using sample sentences, including: classifying the sample sentences based on their sentence complexity to obtain sample sentences corresponding to each sentence complexity; determining the recognition accuracy of the sample sentences corresponding to each sentence complexity based on the language recognition model; determining a target sentence complexity whose recognition accuracy is lower than the accuracy threshold, and training the language recognition model for the next time based on the sample sentences corresponding to the target sentence complexity.

[0078] Specifically, the recognition accuracy corresponding to the same sentence complexity is determined. In order to ensure the accuracy of the language recognition model in recognizing sample sentences, it is necessary to determine whether the recognition accuracies obtained after each training are greater than the accuracy threshold. If so, it means that the language recognition model can accurately recognize sample sentences with different sentence complexities, and the currently trained language recognition model is used as the target language recognition model; if not, all sample sentences corresponding to sentence complexities below the accuracy threshold are determined, and all sample sentences corresponding to sentence complexities below the accuracy threshold are imported into the language recognition model for retraining until the recognition accuracy obtained after training is greater than the accuracy threshold, and the current language recognition accuracy is used as the target language recognition model, and the sample sentences are used as the target sample sentences.

[0079] For example: classify the sample sentences according to sentence complexity, and obtain 10 sample sentences with sentence complexity of level one and 20 sample sentences with sentence complexity of level two. Input the sample sentences with sentence complexity of level one into the language recognition model for training, output the second field value corresponding to each sample sentence, and compare the output second field value with the marked first field value. If there are 4 sample sentences whose second field values ​​are consistent with the first field value among the 10 sample sentences, then it is determined that the recognition accuracy corresponding to the sentence complexity of level one is 40%; similarly, input the sample sentences with sentence complexity of level two into the language recognition model for training, and output the second field value corresponding to each sample sentence. If there are 12 sample sentences whose second field values ​​are the same as the first field value among the 20 sample sentences, then it is determined that the recognition accuracy corresponding to the sentence complexity of level two is 60%.

[0080] Furthermore, if the accuracy threshold is 50%, the recognition accuracy of 40% corresponding to the sentence complexity of level one is lower than 50%, and the recognition accuracy of 60% corresponding to the sentence complexity of level two is higher than 50%, then level one is taken as the target sentence complexity, 10 sample sentences corresponding to the target sentence complexity are determined, and the language recognition model is retrained based on the 10 sample sentences.

[0081] Based on the above description, by labeling the preset fields in the target sentence, a sample sentence is obtained, the sample sentence is used as input to train the language recognition model, and the target language recognition model is determined by the recognition accuracy, thereby ensuring the recognition accuracy of the target language recognition model for the sample sentence and improving the accuracy of the target language recognition model.

[0082] Reference Figure 3 The embodiment of the present application also provides a language recognition method for accurately recognizing a sentence to be detected. The implementation process of the method is as follows:

[0083] Step S31: Determine the sentence similarity between the sentence to be detected and the sample sentence in the target database.

[0084] In an embodiment of the present application, after the user inputs a question statement in the terminal device, the system will receive a statement to be tested from the terminal device, which statement to be tested contains a numerical value, such as: "Is the balance this month 500?" After determining the statement to be tested, the statement to be tested is matched with each sample statement in the target database, which contains all sample statements corresponding to each business scenario.

[0085] All sample statements in the target database cover various business scenarios, and each sample statement includes at least one set of preset fields and field values ​​corresponding to the preset fields.

[0086] After matching the sentence to be detected with each sample sentence, the sentence similarity between the sentence to be detected and each sample sentence can be calculated by comparing whether the keywords or key words of the sentence to be detected and the sample sentences are consistent.

[0087] By using the above method, the sentence to be detected is matched with the sample sentences, and the sentence similarity between the sentence to be detected and each sample sentence is determined, which is conducive to determining the sample sentence that is highly similar to the sentence to be detected.

[0088] Step S32: Filter out target sentence similarities greater than a preset threshold from the sentence similarities, and determine target sample sentences corresponding to the target sentence similarities.

[0089] After obtaining all sentence similarities, all sentence similarities can be sorted in order of size to screen out the maximum sentence similarity, use the maximum sentence similarity as the target sentence similarity, and determine the sample sentence corresponding to the target sentence similarity, and use the sample sentence as the target sample sentence that matches the sentence to be detected.

[0090] It should be noted that, in order to accurately determine the target sample sentence, a preset threshold may be set to prevent the target sentence from having a low similarity, thereby causing a large difference between the determined target sample sentence and the sentence to be detected.

[0091] After determining the similarities of all sentences, compare all sentence similarities with the preset threshold. If the similarities of all sentences are not greater than the preset threshold, it means that there is a large difference between the determined sample sentence and the sentence to be detected. In this way, the sentence to be detected is marked as a new sentence, so that the language recognition model can subsequently train and learn the new sentence, thereby ensuring that the language recognition model can be automatically optimized and can recognize various types of sentences.

[0092] If the similarity of the target sentence is greater than the preset threshold, it means that the difference between the determined sample sentence and the sentence to be detected is small, then the determined sample sentence will be used as the target sample sentence, thereby determining the target sample sentence corresponding to the sentence to be detected, thereby ensuring that the accuracy of language recognition after using the target sample sentence instead of the sentence to be detected is higher.

[0093] Through the above method, the target sample sentence corresponding to the sentence to be detected can be determined, and the similarity with the target sentence can be compared through a preset threshold to prevent the screening of sample sentences that are significantly different from the sentence to be detected, and perform subsequent language recognition, which is conducive to achieving accurate reasoning of the sentence to be detected.

[0094] Step S33: Input the target sample sentence into the language recognition model to obtain the field value of the sentence to be detected.

[0095] After the target sample sentence is determined, the target sample sentence is input into the language recognition model, and the language recognition model outputs the field value corresponding to the target sample sentence, and the field value is used as the field value corresponding to the sentence to be detected.

[0096] The above-mentioned language recognition model can accurately identify the field value corresponding to the sentence to be tested, so that the intelligent customer service can generate a reply sentence to the user based on the field value, ensuring the accuracy of the intelligent customer service's reply to the user's sentence to be tested.

[0097] Based on the method described above, the target sample sentence corresponding to the sentence to be detected is determined, and the language recognition model recognizes the screened target sample sentence instead of directly recognizing the sentence to be detected, and finally determines the field value corresponding to the sentence to be detected, thereby ensuring the recognition accuracy of the language recognition model for the sentence to be detected, and is conducive to improving the accuracy of determining the field value corresponding to the sentence to be detected.

[0098] Based on the same inventive concept, a language recognition model training device is also provided in the embodiment of the present application. The language recognition model training device is used to implement the function of a language recognition model training method. Figure 4 , the device comprises:

[0099] The determination module 401 is used to determine a target sentence containing a preset field, and mark the first field value of the preset field to obtain a sample sentence;

[0100] The training module 402 trains the language recognition model N times using the sample sentences and the reference recognition accuracy to obtain the recognition accuracy corresponding to the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model in recognizing the first field value, and N is an integer greater than 1;

[0101] The target module 403 is used to use the language recognition model obtained by the Nth training as the target language recognition model when the recognition accuracy corresponding to the Nth training is greater than the accuracy threshold.

[0102] In one possible design, the determination module 401 is specifically used to obtain an initial sentence, cluster the initial sentence, and obtain a target sentence at the center of the cluster, wherein the target sentence carries a category identifier corresponding to the business scenario; determine a preset field in the target sentence, and a first field value corresponding to the preset field, and mark the first field value in the target sentence to obtain a sample sentence.

[0103] In one possible design, the training module 402 is specifically used to input the sample sentence into the language recognition model obtained after the i-th training, obtain the second field value corresponding to the preset field, match the first field value with the second field value, determine the recognition accuracy, and when the recognition accuracy is lower than the accuracy threshold, input the sample sentence into the language recognition model for the i+1-th training.

[0104] In one possible design, the training module 402 is also used to determine a first number of the sample sentences and a second number of sample sentences whose first field value is consistent with the second field value, calculate a ratio between the second number and the first number, and use the ratio as recognition accuracy.

[0105] In one possible design, the training module 402 is also used to determine the sentence complexity corresponding to the sample sentence, and determine a third number of sample sentences corresponding to the sentence complexity, determine a fourth number of sample sentences whose first field value is consistent with the second field value from the sample sentences corresponding to the complexity, calculate the ratio between the fourth number and the third number, and use the ratio as the recognition accuracy.

[0106] In one possible design, the training module 402 is also used to classify the sample sentences based on the complexity of the sample sentences to obtain sample sentences corresponding to each complexity, determine the recognition accuracy of the sample sentences corresponding to each complexity based on the language recognition model, determine the target complexity at which the recognition accuracy is lower than the accuracy threshold, and perform next training on the language recognition model based on the sample sentences corresponding to the target complexity.

[0107] Based on the same inventive concept, a language recognition device is also provided in the embodiment of the present application. The language recognition device is used to implement the function of a language recognition method. Figure 5 , the device comprises:

[0108] An acquisition module 501 is used to determine the sentence similarity between the sentence to be detected and the sample sentence in the target database, wherein the target database contains sample sentences corresponding to various business scenarios, each sample sentence contains a preset field and a field value corresponding to the preset field;

[0109] A screening module 502 is used to screen out target sentence similarities greater than a preset threshold from the sentence similarities, and determine a target sample sentence corresponding to the target sentence similarity;

[0110] The output module 503 is used to input the target sample sentence into the language recognition model to obtain the field value of the sentence to be detected.

[0111] In a possible design, the output module 503 is specifically used to mark the sentence to be detected as a new sentence when there is no target sentence similarity greater than the preset threshold in the sentence similarities, wherein the new sentence is used for retraining the language recognition model.

[0112] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the functions of the aforementioned language recognition model training device and the language recognition device, referring to Figure 6 , the electronic device comprises:

[0113] At least one processor 601, and a memory 602 connected to the at least one processor 601. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6 In the example, the processor 601 and the memory 602 are connected via a bus 600. Figure 6 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller, and there is no limitation on the name.

[0114] In the embodiment of the present application, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute a language recognition model training method and a language recognition method discussed above by executing the instructions stored in the memory 602. The processor 601 can implement Figure 4 and 5 The functions of each module in the device shown.

[0115] Among them, the processor 601 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 602 and calling data stored in the memory 602, the various functions of the device and processing data, the device can be monitored as a whole.

[0116] In one possible design, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0117] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. A language recognition model training method and a language recognition method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to execute, or a combination of hardware and software modules in the processor to execute.

[0118] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0119] By programming the processor 601, a language recognition model training method and a code corresponding to a language recognition method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 A language recognition model training step and execution of the embodiment shown Figure 3 A language recognition step of the embodiment shown. How to design and program the processor 601 is a technique known to those skilled in the art and will not be described in detail here.

[0120] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a language recognition model training method and a language recognition method discussed above.

[0121] In some possible implementations, the present application provides a language recognition model training method and various aspects of a language recognition method, which can also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a language recognition model training method and a language recognition method according to various exemplary embodiments of the present application described above in this specification.

[0122] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A language recognition model training method, characterized in that: include: Determine a target sentence containing a preset field, and mark a first field value of the preset field to obtain a sample sentence; Training the language recognition model N times using the sample sentences to obtain a recognition accuracy corresponding to the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model in recognizing the first field value, and N is an integer greater than 1; When the recognition accuracy corresponding to the Nth training is greater than the accuracy threshold, the language recognition model obtained by the Nth training is used as the target language recognition model.

2. The method according to claim 1, characterized in that The training of the language recognition model N times includes: Input the sample sentence into the language recognition model obtained after the i-th training to obtain a second field value corresponding to the preset field, where i is a positive integer less than N; Matching the first field value with the second field value to determine recognition accuracy; When the recognition accuracy is lower than the accuracy threshold, the sample sentence is input into the language recognition model for the (i+1)th training.

3. The method according to claim 1, characterized in that The step of determining a target sentence including a preset field and marking a first field value of the preset field to obtain a sample sentence includes: Acquire an initial sentence, and cluster the initial sentence to obtain a target sentence at the center of the cluster, wherein the target sentence carries a category identifier corresponding to the business scenario; Determine a preset field in the target sentence, and a first field value corresponding to the preset field; The first field value in the target sentence is marked to obtain a sample sentence.

4. The method according to claim 2, characterized in that The matching the first field value with the second field value to determine the recognition accuracy includes: Determining a first number of the sample sentences, and determining a second number of the sample sentences in which the first field value is consistent with the second field value; A ratio between the second number and the first number is calculated, and the ratio is used as the recognition accuracy.

5. The method according to claim 2, characterized in that The matching the first field value with the second field value to determine the recognition accuracy includes: Determining the sentence complexity corresponding to the sample sentence, and determining a third number of sample sentences corresponding to the sentence complexity; Determining, from the sample sentences corresponding to the complexity, a fourth number of sample sentences in which the first field value is consistent with the second field value; A ratio between the fourth number and the third number is calculated, and the ratio is used as the recognition accuracy.

6. The method according to claim 1, characterized in that The training of the language recognition model N times by using the sample sentences includes: Classifying the sample sentences based on the sentence complexity of the sample sentences to obtain sample sentences corresponding to each sentence complexity; Determining the recognition accuracy of the sample sentences corresponding to each sentence complexity based on the language recognition model; The target sentence complexity whose recognition accuracy is lower than the accuracy threshold is determined, and the language recognition model is trained for the next time based on the sample sentences corresponding to the target sentence complexity.

7. A language recognition method, characterized in that: include: Determine the sentence similarity between the sentence to be detected and the sample sentence in the target database, wherein the target database contains sample sentences corresponding to various business scenarios, each sample sentence contains a preset field and a field value corresponding to the preset field; Filter out a target sentence similarity greater than a preset threshold from the sentence similarities, and determine a target sample sentence corresponding to the target sentence similarity; The target sample sentence is input into a language recognition model to obtain the field value of the sentence to be detected, wherein the language recognition model is trained according to any one of the methods according to claims 1-6.

8. The method according to claim 7, characterized in that Before inputting the target sample sentence into the language recognition model, the method further includes: When there is no target sentence similarity greater than the preset threshold value in the sentence similarities, the sentence to be detected is marked as a newly added sentence, wherein the newly added sentence is used to train the language recognition model.

9. A language recognition model training device, characterized in that: include: A determination module, used to determine a target sentence containing a preset field, and mark a first field value of the preset field to obtain a sample sentence; A training module, which trains the language recognition model N times through the sample sentences to obtain a recognition accuracy corresponding to the Nth training, wherein the recognition accuracy includes the accuracy of the language recognition model in recognizing the first field value, and N is an integer greater than 1; The target module is used to use the language recognition model obtained by the Nth training as the target language recognition model when the Nth recognition accuracy is greater than the accuracy threshold.

10. A language recognition device, characterized in that: include: An acquisition module is used to determine the sentence similarity between the sentence to be detected and the sample sentence in the target database, wherein the target database contains sample sentences corresponding to various business scenarios, each sample sentence contains a preset field and a field value corresponding to the preset field; A screening module, used to screen out target sentence similarities greater than a preset threshold from the sentence similarities, and determine a target sample sentence corresponding to the target sentence similarity; The output module is used to input the target sample sentence into a language recognition model to obtain the field value of the sentence to be detected, and the language recognition model is trained according to any one of the methods of claims 1-5.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1-6 and 7-8 when executing the computer program stored in the memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-6 and 7-8 are implemented.

13. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6 and 7 to 8.

Images