Model training method and device, semantic recognition method and device and electronic equipment

Abstract

The invention provides a model training method, a semantic recognition method, a semantic recognition device and electronic equipment, which are used for improving the problem that the balance between the accuracy rate and the recall rate cannot be considered at the same time. The method comprises the following steps: performing matching identification on a plurality of user texts to be processed by using preset keywords to obtain identified user texts and unidentified user texts; randomly sampling the unidentified user text to obtain a sampled user text; identifying and screening the identified user text and the sampled user text by using a large language model according to a preset keyword to obtain a screened user text; performing semantic alignment annotation on the screened user text to obtain a screened alignment annotation; and training the small network model by taking the screened user text and the preset keyword as training data and the screened alignment label as a training label, the model parameters of the small network model being less than the model parameters of the large language model.

Description

Technical Field

[0001] The present application relates to the technical fields of deep learning, neural networks and semantic recognition, and more specifically, to a model training method, a semantic recognition method, a device and an electronic device. Background Art

[0002] The current large language model has good context understanding capabilities in semantic recognition. In some business scenarios with higher fault tolerance, it can better meet the business requirements for accuracy and recall. However, in some business scenarios with lower fault tolerance, it is impossible to strike a balance between accuracy and recall in the business. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a model training method, a semantic recognition method, a device and an electronic device, which are used to improve the problem of being unable to strike a balance between accuracy and recall in business.

[0004] The embodiment of the present application provides a model training method, including: using preset keywords to match and identify multiple user texts to be processed, obtaining recognized user texts and unrecognized user texts; randomly sampling unrecognized user texts to obtain sampled user texts; using a large language model to identify and filter recognized user texts and sampled user texts according to preset keywords to obtain filtered user texts; semantically aligning and annotating the filtered user texts to obtain filtered alignment annotations, wherein the filtered alignment annotations represent whether the semantic intentions between the filtered user texts and the preset keywords are the same; using the filtered user texts and preset keywords as training data, and the filtered alignment annotations as training labels, to train a small network model, wherein the model parameters of the small network model are less than the model parameters of the large language model. In the implementation process of the above scheme, the text containing specific vocabulary can be quickly located through keyword matching, and the interference of irrelevant texts is reduced to improve the accuracy. Random sampling increases the opportunity for the small network model to learn semantic patterns that are not matched by keywords, thereby improving the recall rate. Since the large language model with strong semantic understanding is used for identification and screening, the recall rate can be further improved, and the use of semantically aligned and annotated samples to train the small network model can achieve a higher accuracy rate. Therefore, we combine keyword matching, random sampling, large language model screening, and semantically aligned labeled samples to iteratively train the small network model, effectively allowing the small network model to find a better balance between accuracy and recall, thereby taking into account the business requirements for a balance between accuracy and recall.

[0005] Optionally, in an embodiment of the present application, a large language model is used to identify and screen the identified user text and the sampled user text according to preset keywords, including: for each user text in the identified user text and the sampled user text, a prompt engineering text is generated according to the user text and the preset keywords; the prompt engineering text is input into the large language model so that the large language model returns the semantic similarity probability between the user text and the preset keywords; if the semantic similarity probability is less than the preset similarity threshold, the user text is removed from the identified user text and the sampled user text. In the implementation process of the above scheme, the identified user text and the sampled user text are preliminarily screened (i.e., pre-screened) by a large language model with a higher recall rate, so that after the low-relevance text is removed during the model training process, it can focus more on learning and understanding the context closely related to the keywords, thereby improving its ability to understand and generate high-quality texts, and can effectively solve the problem of insufficient number of artificial samples, ensuring that the model can accurately identify texts containing keywords in practical applications, and can also identify those texts related to keywords but not directly using keywords, thereby ultimately further balancing the accuracy and recall rates.

[0006] Optionally, in an embodiment of the present application, after obtaining the filtered user text, it also includes: using a large language model to perform data enhancement on the filtered user text according to preset keywords to obtain model-enhanced text, the model-enhanced text is a text with the same semantics as the preset keywords; semantically aligning and annotating the model-enhanced text to obtain enhanced alignment annotations, the enhanced alignment annotations characterize whether the semantic intent between the model-enhanced text and the preset keywords is the same; using the model-enhanced text and the preset keywords as training data, and the enhanced alignment annotations as training labels, the small network model is trained. In the implementation process of the above scheme, data enhancement enables the small network model to be exposed to more diverse forms of expression, which helps to improve the model's adaptability to unknown data, and semantically aligns and annotates the enhanced text to ensure that the model can more accurately capture the relationship between the text and the preset keywords, thereby improving the accuracy of semantic understanding. Further, the use of carefully selected data sets (i.e., model-enhanced texts) can significantly reduce the time and computing resources required for training because these data sets have been preprocessed and optimized.

[0007] Optionally, in an embodiment of the present application, after training the small network model, it also includes: determining whether the performance index of the small network model exceeds a preset index threshold; if so, using the small network model to perform semantic recognition on the unrecognized user text according to preset keywords to obtain a semantic recognition result, the semantic recognition result characterizing whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than a preset similarity threshold. In the implementation process of the above scheme, after the performance of the small network model reaches the preset index threshold, computing resources can be effectively utilized to perform large-scale semantic recognition tasks, avoiding the computing overhead caused by using a larger model. Since the small network model has a faster reasoning speed, real-time or near real-time semantic recognition processing can be achieved, thereby improving the speed and efficiency of semantic recognition of the small network model.

[0008] Optionally, in an embodiment of the present application, after training the small network model, it also includes: using the large language model to perform semantic recognition on unrecognized user text according to preset keywords to obtain a first recognition result; using the small network model to perform semantic recognition on the first recognition result according to preset keywords to obtain a second recognition result; determining the final recognition result according to the first recognition result and the second recognition result. In the implementation process of the above scheme, by first using the large language model for preliminary recognition and then using the small network model for secondary recognition, the advantages of each other can be complemented, and the two recognitions can reduce the false positive problem that may exist in a single model, thereby improving the accuracy of the final recognition result.

[0009] Optionally, in an embodiment of the present application, the final recognition result is determined based on the first recognition result and the second recognition result, including: for each user text in the unrecognized user text, determining whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold; if so, the second recognition result is determined as the final recognition result. In the implementation process of the above scheme, a larger language model can capture a wider range of semantic relationships, while a small network model is more focused on specific tasks. If both models confirm the correlation between the user text and the preset keywords, then this double confirmation can reduce the possibility of false positives, and two recognitions can reduce the false positives that may exist in a single model, especially when dealing with fuzzy or boundary situations, thereby improving the accuracy of the final recognition result through these two recognition processes.

[0010] The embodiment of the present application also provides a semantic recognition method, including: using preset keywords to match and recognize multiple user texts to be processed to obtain recognized user texts and unrecognized user texts; using the small network model trained by the method described above to perform semantic recognition on unrecognized user texts according to preset keywords to obtain semantic recognition results, and the semantic recognition results represent whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than a preset similarity threshold. In the implementation process of the above scheme, after the performance of the small network model reaches the preset indicator threshold, computing resources can be effectively used to perform large-scale semantic recognition tasks, avoiding the computing overhead caused by using a larger model. Since the small network model has a faster reasoning speed, real-time or near real-time semantic recognition processing can be achieved, thereby improving the speed and efficiency of semantic recognition of the small network model.

[0011] Optionally, in an embodiment of the present application, after obtaining the recognized user text and the unrecognized user text, it also includes: using a small network model to perform semantic recognition on the recognized user text according to preset keywords; and / or, using a large language model to perform semantic recognition on the recognized user text according to preset keywords. In the implementation process of the above scheme, by using a small network model for semantic recognition, the accuracy of recognition can be improved, especially when processing data in specific fields, the small model may be more focused on the semantic relations in these fields. Alternatively, the large language model has a strong generalization ability and can capture a wider range of semantic relations, thereby improving the robustness of the model when processing different types of text. Alternatively, by combining the results of the small network model and the large language model, the false alarm problem that may exist in a single model can be reduced, especially when dealing with fuzzy or boundary cases, the accuracy of semantic recognition can be effectively improved.

[0012] The embodiment of the present application also provides a semantic recognition method, including: using preset keywords to match and recognize multiple user texts to be processed to obtain recognized user texts and unrecognized user texts; using a large language model to perform semantic recognition on unrecognized user texts according to preset keywords to obtain a first recognition result; using a small network model trained using the method described above to perform semantic recognition on unrecognized user texts according to preset keywords to obtain a second recognition result, wherein the model parameters of the small network model are less than the model parameters of the large language model; and determining a final recognition result based on the first recognition result and the second recognition result. In the implementation process of the above scheme, by first using a large language model for preliminary recognition and then using a small network model for secondary recognition, the advantages of each other can be complemented, and the two recognitions can reduce the false alarm problem that may exist in a single model, thereby improving the accuracy of the final recognition result.

[0013] Optionally, in an embodiment of the present application, the final recognition result is determined based on the first recognition result and the second recognition result, including: for each user text in the unrecognized user text, judging whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold; if so, the second recognition result is used to determine the final recognition result. In the implementation process of the above scheme, through two recognition processes, the first recognition (large language model) can filter out irrelevant texts, and the second recognition (small network model) further confirms the relevance. This double check mechanism can significantly reduce the false alarm rate. Since both models confirm the relevance between the user text and the preset keywords, this double confirmation can reduce the possibility of false alarms, thereby improving the accuracy of the final recognition result.

[0014] The embodiment of the present application also provides a model training device, including: a text matching and recognition module, which is used to use preset keywords to match and recognize multiple user texts to be processed, and obtain recognized user texts and unrecognized user texts; a text random sampling module, which is used to randomly sample unrecognized user texts to obtain sampled user texts; an identification and screening processing module, which is used to use a large language model to identify and screen the recognized user texts and sampled user texts according to preset keywords, and obtain filtered user texts; a semantic alignment and annotation module, which is used to perform semantic alignment and annotation on the filtered user texts, and obtain filtered alignment annotations, wherein the filtered alignment annotations represent whether the semantic intentions between the filtered user texts and the preset keywords are the same; a network model training module, which is used to train a small network model using the filtered user texts and preset keywords as training data and the filtered alignment annotations as training labels, wherein the model parameters of the small network model are less than the model parameters of the large language model.

[0015] Optionally, in an embodiment of the present application, the identification, screening and processing module includes: a prompt text generation submodule, which is used to generate a prompt engineering text for each user text in the identified user text and the sampled user text according to the user text and preset keywords; a prompt text input submodule, which is used to input the prompt engineering text into the large language model so that the large language model returns the semantic similarity probability between the user text and the preset keywords; and a user text removal submodule, which is used to remove the user text from the identified user text and the sampled user text if the semantic similarity probability is less than a preset similarity threshold.

[0016] Optionally, in an embodiment of the present application, the model training device also includes: a text data enhancement module, which is used to use a large language model to perform data enhancement on the filtered user text according to preset keywords to obtain model-enhanced text, where the model-enhanced text is a text with the same semantics as the preset keywords; an enhanced alignment annotation module, which is used to perform semantic alignment annotation on the model-enhanced text to obtain enhanced alignment annotation, where the enhanced alignment annotation represents whether the semantic intent between the model-enhanced text and the preset keywords is the same; an enhanced model training module, which is used to train the small network model using the model-enhanced text and preset keywords as training data and the enhanced alignment annotation as training labels.

[0017] Optionally, in an embodiment of the present application, the model training device also includes: a performance indicator judgment module, used to judge whether the performance indicator of the small network model exceeds the preset indicator threshold; a recognition result acquisition module, used to use the small network model to perform semantic recognition on the unrecognized user text according to preset keywords if the performance indicator of the small network model exceeds the preset indicator threshold, and obtain a semantic recognition result, wherein the semantic recognition result represents whether the probability of semantic similarity between the unrecognized user text and the preset keywords is greater than the preset similarity threshold.

[0018] Optionally, in an embodiment of the present application, the model training device also includes: a first semantic recognition module, used to use a large language model to perform semantic recognition on unrecognized user text according to preset keywords to obtain a first recognition result; a second semantic recognition module, used to use a small network model to perform semantic recognition on the first recognition result according to preset keywords to obtain a second recognition result; a recognition result determination module, used to determine the final recognition result based on the first recognition result and the second recognition result.

[0019] Optionally, in an embodiment of the present application, the recognition result determination module includes: a recognition result judgment submodule, which is used to determine, for each user text in the unrecognized user text, whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold; and a final result determination submodule, which is used to determine the second recognition result as the final recognition result if the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold.

[0020] The embodiment of the present application also provides a semantic recognition device, including: a first matching and recognition module, used to use preset keywords to match and recognize multiple user texts to be processed, and obtain recognized user texts and unrecognized user texts; a text semantic recognition module, used to use the small network model obtained after training to perform semantic recognition on unrecognized user texts according to preset keywords, and obtain semantic recognition results, which represent whether the probability of semantic similarity between the unrecognized user text and the preset keywords is greater than a preset similarity threshold.

[0021] Optionally, in an embodiment of the present application, the semantic recognition device also includes: a first text recognition module, used to use a small network model to perform semantic recognition on the recognized user text according to preset keywords; and / or, a second text recognition module, used to use a large language model to perform semantic recognition on the recognized user text according to preset keywords.

[0022] The embodiment of the present application also provides a semantic recognition device, including: a second matching and recognition module, used to use preset keywords to match and recognize multiple user texts to be processed, and obtain recognized user texts and unrecognized user texts; a first result acquisition module, used to use a large language model to perform semantic recognition on unrecognized user texts according to preset keywords, and obtain a first recognition result; a second result acquisition module, used to use a small network model obtained after training, to perform semantic recognition on unrecognized user texts according to preset keywords, and obtain a second recognition result, wherein the model parameters of the small network model are less than the model parameters of the large language model; a final result determination module, used to determine the final recognition result based on the first recognition result and the second recognition result.

[0023] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the method described above when executed by the processor.

[0024] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is executed.

[0025] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flow chart of a model training method provided in an embodiment of the present application is shown;

[0028] Figure 2 A flow chart of a semantic recognition method combining keyword matching and a small network model provided in an embodiment of the present application is shown;

[0029] Figure 3 A flow chart of a semantic recognition method combining keyword matching, a small network model and a large language model provided in an embodiment of the present application is shown;

[0030] Figure 4 A schematic diagram of the flow chart of the model training device provided in an embodiment of the present application is shown;

[0031] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the embodiment of the present application only serve the purpose of explanation and description, and are not used to limit the protection scope of the embodiment of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in the embodiment of the present application shows the operation implemented according to some embodiments of the embodiment of the present application. It should be understood that the operation of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the embodiment of the present application, or remove one or more operations from the flowchart.

[0033] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings generally here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the embodiments of the present application claimed, but merely represents the selected embodiments of the embodiments of the present application.

[0034] It is understandable that the "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit the difference. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there may be three relationships, such as A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship. The term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups).

[0035] Before introducing the model training method provided in the embodiment of the present application, some concepts involved in the embodiment of the present application are first introduced:

[0036] Semantic recognition is to determine whether the target text has a similar probability of semantic intent with a preset keyword, or whether the semantic intents of the two are the same. Semantic recognition in the embodiment of the present application refers to semantic alignment recognition, that is, comparing the semantic similarity and consistency between two or more natural language texts.

[0037] It should be noted that the model training method provided in the embodiment of the present application can be executed by an electronic device, where the electronic device refers to a device terminal or server with the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant, or a mobile Internet device. A server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, and non-x86 servers include mainframes, minicomputers, and UNIX servers.

[0038] In some related technologies, full-text matching recognition using keywords can accurately match predefined text patterns, such as matching and recognizing complex text patterns such as dates, phone numbers, or email addresses. This matching recognition method has a fast processing speed, but it is limited by the predefined patterns and makes it difficult to capture semantic relationships and contextual information. Therefore, the recall rate of full-text matching recognition using keywords is low.

[0039] In other related technologies, since the Generative Artificial Intelligence (GAI) large language model has a good ability to capture semantic relationships and understand context, a large language model is used for semantic recognition to improve accuracy and recall. However, in some business scenarios with low error tolerance (such as manufacturing or medical scenarios), it is found that large language models often use general language data when training, and the expertise of specific industries may not be fully included. This means that the model may misunderstand when dealing with professional terms or specific situations, which in turn affects the balance between accuracy and recall. Furthermore, large language models are usually designed to handle multiple language tasks, but this generalization ability does not always meet the needs of specific business scenarios. For example, in medical diagnosis, the model needs a very high accuracy to avoid misdiagnosis, but also a high recall rate to ensure that no important information is missed. Therefore, it is difficult for the current large language model to balance the business requirements between accuracy and recall in semantic recognition.

[0040] For the above questions, please see Figure 1 The flowchart of the model training method provided by the embodiment of the present application is shown; the embodiment of the present application provides a model training method, which combines keyword matching, random sampling, screening of a large language model, and samples with semantic alignment annotation to iteratively train a small network model, effectively allowing the small network model to find a better balance between accuracy and recall, thereby taking into account the balance between accuracy and recall in the business. The implementation method of the specific model training method of the above-mentioned small network model may include:

[0041] Step S110: using preset keywords to match and identify the multiple user texts to be processed, and obtaining identified user texts and unidentified user texts.

[0042] It is understandable that keyword matching can quickly locate texts containing specific words. These words are often task-related keywords. Texts that are highly relevant to the task can be quickly screened out, and a large amount of text that does not contain keywords can be excluded. This means that subsequent processing flows (such as screening of large language models, training of small models, etc.) only need to focus on texts related to keywords, reducing interference from irrelevant texts, thereby improving accuracy.

[0043] Step S120: Randomly sample unrecognized user text to obtain sampled user text.

[0044] It is understandable that random sampling can extract samples from texts that are not matched by keywords. These texts may contain semantic patterns related to the preset keywords, but they do not directly use the keywords. In this way, the diversity of training data can be increased, giving the model the opportunity to contact and learn semantic patterns that are not matched by keywords. For example, randomly sampled texts may contain semantic intents related to preset keywords, but these texts do not directly use keywords. In this way, more task-related texts can be captured to improve the recall rate.

[0045] Step S130: using the large language model to perform recognition and screening processing on the recognized user text and the sampled user text according to preset keywords to obtain screened user text.

[0046] It is understandable that after being trained with a large amount of data, the large language model can understand complex language structures and contextual relationships. When using the large language model to identify and filter texts selected by keyword matching and randomly sampled texts, the large language model can identify semantic patterns related to preset keywords, even if these patterns do not directly use keywords. Therefore, the recognition and filtering processing of the large language model can further improve the relevance of the text, and the powerful generalization ability of the large language model can identify more complex semantic relationships, which helps to improve the overall accuracy.

[0047] Step S140: semantically aligning and annotating the filtered user text to obtain a filtered alignment annotation, where the filtered alignment annotation indicates whether the semantic intentions of the filtered user text and the preset keywords are the same.

[0048] The above semantic alignment annotations are completed by carefully understanding the semantics of the filtered user text, ensuring the high quality of the training data. These semantic alignment annotations clearly indicate whether the semantic intent between the user text and the preset keywords is the same, thus providing a clear training signal. High-quality training data and training labels can help the small network model learn more accurate semantic patterns, thereby improving accuracy.

[0049] Step S150: Using the filtered user text and preset keywords as training data and the filtered alignment annotations as training labels, the small network model is trained to obtain a trained small network model, wherein the model parameters of the small network model are less than the model parameters of the large language model.

[0050] The small network model has fewer model parameters and can be trained on less training data and training labels. At the same time, it focuses on specific tasks and combines keyword matching, random sampling, screening of large language models, and semantic alignment labeling to improve the accuracy of semantic recognition using small network models.

[0051] In the implementation of the above solution, keyword matching can quickly locate texts containing specific words, reducing the interference of irrelevant texts to improve accuracy. Random sampling increases the chances of the small network model to learn semantic patterns that are not matched by keywords, thereby improving the recall rate. The large language model with strong semantic understanding can further improve the recall rate for identification and screening, and the use of semantically aligned labeled samples to train the small network model can achieve higher accuracy. Therefore, combining keyword matching, random sampling, large language model screening, and semantically aligned labeled samples to iteratively train the small network model effectively allows the small network model to find a better balance between accuracy and recall, thereby taking into account the balance between accuracy and recall in the business.

[0052] Optionally, as an optional implementation of the above step S110, full-text matching and recognition are performed on the multiple user texts to be processed using preset keywords. For example, assuming that the keyword is return, you can directly search whether the user text contains the keyword "return". Assuming that the user text includes "Hello, there is a problem with the product I recently purchased. I want to return it. What should I do?" The full-text matching and recognition result at this time is a successful match. This full-text matching and recognition method will directly search whether the text contains specific keywords. If the text does not contain these keywords, it will not be selected. This method has high accuracy because it can accurately locate texts containing keywords, but it is also easy to miss texts that do not directly use keywords but are related in meaning.

[0053] Optionally, as another optional implementation of the above step S110, a plurality of user texts to be processed are matched and identified using a regular expression using preset keywords. For example, assuming that the keyword is (return|exchange), a regular expression can be used to match keyword variants or related words. The user text includes "I want to exchange the goods because the received product is defective". At this time, the full text matching recognition result is a successful match. Using regular expressions to match keyword variants or related words can handle different forms of keywords more flexibly than full text matching, but it may still ignore some texts that do not directly use keywords.

[0054] Optionally, as an optional implementation of the above-mentioned step S120, for example: assuming that the unidentified user text includes "I am not satisfied with this shopping and hope for a refund", "I have received this product and the experience of using it is very good" and "There is a problem with the things I bought and I want to return them", after randomly sampling the unidentified user text, the sampled user text obtained may include: "I am not satisfied with this shopping and hope for a refund" and "There is a problem with the things I bought and I want to return them".

[0055] It is understandable that the number of user texts in the customer service operation quality inspection scenario is very large. In this case, it is very difficult to manually collect training data and training labels suitable for the small network model, resulting in insufficient training data samples of certain categories of the small network model. In order to overcome this problem, the embodiment of the present application uses a large language model with a higher recall rate to perform preliminary screening (i.e., initial screening) of recognized user texts and sampled user texts, which can effectively solve the problem of insufficient number of manual samples, thereby improving the training effect of the model.

[0056] As an optional implementation of the above step S130, when using a large language model to identify and filter the recognized user text and the sampled user text according to preset keywords, the user text can be eliminated through a prompting project. This implementation may include:

[0057] Step S131: for each user text in the recognized user text and the sampled user text, a prompt engineering text is generated according to the user text and preset keywords.

[0058] The implementation method of the above step S131 includes: it can be generated by using a direct prompt word generation method (i.e., constructing a simple prompt to directly ask the model text about the relevance between the keyword). For a user text "This dress is the wrong size, I want to return it", the generated prompt engineering text can be "Is this text related to 'return'? If so, what is the probability of semantic similarity between this user text and 'return'?". It can also be generated by using a keyword emphasis method (i.e., explicitly emphasizing the preset keywords in the prompt to ensure that the model can focus more on the relevance between the keywords and the text). Assuming the user text "This dress is the wrong size, I want to return it", the generated prompt engineering text can be "Please evaluate the semantic similarity probability between this user text and the keyword 'return'", which can more accurately measure the matching degree between the text and the keyword. In addition, the context-assisted method (providing additional context information in the prompt to help the model better understand the relationship between the keyword and the user text) and the exclusion method (inquiring the model text through the prompt whether there is information that clearly indicates that it is not related to the preset keyword) can be used to generate the prompt engineering text according to the user text and the preset keyword, which will not be explained one by one here.

[0059] Step S132: inputting the prompt engineering text into the large language model so that the large language model returns the semantic similarity probability between the user text and the preset keywords.

[0060] The implementation method of the above-mentioned step S132 includes: inputting the prompt engineering text into a large language model such as a Generative Pre-trained Transformer (GPT) model, a RoBERTa (Robustly Optimized BERTPretraining Approach) model, a DistilBERT model or a SBERT (Sentence-BERT), so that the large language model returns the semantic similarity probability between the user text and the preset keywords.

[0061] It is understandable that the large language model, due to its strong generalization and semantic understanding capabilities, can filter out samples related to the task from a large amount of unlabeled data, even if these samples are not manually labeled. Therefore, through the initial screening of the large language model, the number of training samples can be effectively increased, and the relevance and quality of the samples can be improved. Even if the number of manual samples is insufficient, the samples screened by the large language model can provide more training data for the small network model, thereby improving the training effect of the small network model.

[0062] Step S133: if the semantic similarity probability is less than a preset similarity threshold, the user text is removed from the recognized user texts and the sampled user texts.

[0063] Of course, if the semantic similarity probability is greater than or equal to the preset similarity threshold, the user text can be retained (no operation is performed). If the semantic similarity probability is less than the preset similarity threshold, the user text is removed from the recognized user text and the sampled user text. Considering the high accuracy of the above-mentioned full-text matching recognition or regular expression matching recognition, some corpora that do not contain keywords are mistakenly filtered, resulting in a small number of samples obtained. Both full-text matching recognition and regular expression matching recognition tend to accurately match keywords, so those texts that do not directly contain keywords but are related to the meaning of keywords may be mistakenly filtered out. This means that although the recognition accuracy is high, the recall rate may be low, that is, many related texts are not correctly recognized and included. In order to avoid the problem of accuracy distortion caused by excessive reliance on labeled samples during model training, random sampling can be used to increase text samples that are not hit by keywords or regular expressions to improve the diversity and representativeness of the overall sample. By increasing the text samples that are not matched by keywords, the recall rate of the model can be improved, that is, the model can identify more relevant texts. At the same time, by combining these samples for training, we ensure that the model can accurately identify texts containing keywords in practical applications, as well as texts that are related to keywords but do not directly use keywords, thereby ultimately further balancing precision and recall.

[0064] As an optional implementation of the above model training method, after obtaining the filtered user texts, if the amount of the filtered user texts is less than a preset amount threshold, a large language model may be used for data enhancement, and the implementation may include:

[0065] Step S134: Use a large language model to perform data enhancement on the filtered user text according to preset keywords to obtain model-enhanced text, which is text with the same semantics as the preset keywords.

[0066] For example, the implementation method of the above step S134 is as follows: assuming that the preset keywords include: "return", "refund", "exchange", "dissatisfied", "damaged", etc., and the filtered user text includes "I want to return the product because the product has quality problems" and "I am not satisfied with the product and want to apply for a refund", a large language model can be used to generate a new text with the same semantics but different expression as the original user text (i.e., the filtered user text) according to the preset keywords, and the obtained model-enhanced text includes: "Because the product is defective, I hope to return it" and "This product disappoints me and I want to return it and get a refund". Then,

[0067] Optionally, after using the large language model for data augmentation, when the training data and training labels of the small network model are insufficient to train the small network model, especially when the performance index of the small network model exceeds a preset index threshold, the model augmentation text may be used to annotate and train the small network model. This implementation may include:

[0068] Step S135: semantically aligning and annotating the model-enhanced text to obtain enhanced alignment annotations, where the enhanced alignment annotations indicate whether the semantic intentions between the model-enhanced text and the preset keywords are the same.

[0069] The implementation method of the above-mentioned step S135 includes: performing semantic alignment annotation on the model-enhanced text through a general artificial intelligence (General Artificial Intelligence) large model to obtain enhanced alignment annotation of the large model annotation, or performing semantic alignment annotation on the model-enhanced text through manual annotation to obtain enhanced alignment annotation of manual annotation.

[0070] Step S136: Using the model-enhanced text and preset keywords as training data and the enhanced alignment annotations as training labels, the small network model is trained to obtain a trained small network model.

[0071] The implementation method of the above step S136 is, for example: construct a prompt engineering text based on the model enhanced text and preset keywords, and input the prompt engineering text into the small network model so that the small network model returns the enhanced semantic similarity probability between the model enhanced text and the preset keywords. Then, the loss function is used to calculate the loss value between the enhanced semantic similarity probability and the enhanced alignment annotation, and the model parameters of the small network model are updated according to the loss value until the small network model converges, and the trained small network model can be obtained. In the implementation process of the above scheme, the small network model is exposed to more diverse forms of expression through data enhancement, which helps to improve the model's adaptability to unknown data, and the enhanced text is semantically aligned and annotated to ensure that the model can more accurately capture the relationship between the text and the preset keywords, thereby improving the accuracy of semantic understanding. Further, the use of carefully selected data sets (i.e., model enhanced text) can significantly reduce the time and computing resources required for training because these data sets have been preprocessed and optimized.

[0072] As an optional implementation of the above model training method, after training the small network model, the trained small network model can also be used for semantic recognition. This implementation may include:

[0073] Step S160: Determine whether the performance index of the small network model exceeds a preset index threshold.

[0074] Step S170: If the performance index of the small network model exceeds the preset index threshold, the trained small network model is used to perform semantic recognition on the unrecognized user text according to preset keywords to obtain a semantic recognition result.

[0075] For example, the implementation method of the above steps S160 to S170 is as follows: the performance index of the small network model may include accuracy and recall, and the preset index threshold may include an accuracy threshold and a recall threshold. It can be judged whether the accuracy of the small network model exceeds the accuracy threshold, and whether the recall of the small network model exceeds the recall threshold. If the accuracy of the small network model exceeds the accuracy threshold, and the recall of the small network model exceeds the recall threshold, a prompt engineering text is constructed according to the preset keywords and the unrecognized user text, and the prompt engineering text is input into the small network model, so that the small network model returns the semantic similarity probability between the unrecognized user text and the preset keywords. Then, the semantic recognition result is determined according to the semantic similarity probability. Specifically, if the semantic similarity probability is greater than the preset similarity threshold, the semantic recognition result is determined to be the intention of the preset keyword. If the semantic similarity probability is less than or equal to the preset similarity threshold, the semantic recognition result is determined to be not the intention of the preset keyword. Among them, the above semantic recognition result represents whether the semantic similarity probability between the unrecognized user text and the preset keyword is greater than the preset similarity threshold.

[0076] As an optional implementation of the above method, after training the small network model, the trained small network model may also be used for semantic recognition. This implementation may include:

[0077] Step S180: Use the large language model to perform semantic recognition on the unrecognized user text according to preset keywords to obtain a first recognition result.

[0078] An implementation example of the above step S180 is as follows: construct a prompt engineering text based on preset keywords and unrecognized user text, and input the prompt engineering text into the large language model so that the large language model returns the semantic similarity probability between the unrecognized user text and the preset keywords. Then, the first recognition result is determined based on the semantic similarity probability. Specifically, if the semantic similarity probability is greater than the preset similarity threshold, the first recognition result is determined to be the intention of the preset keyword. If the semantic similarity probability is less than or equal to the preset similarity threshold, the first recognition result is determined to be not the intention of the preset keyword. Among them, the above-mentioned first recognition result represents whether the semantic similarity probability between the unrecognized user text and the preset keyword is greater than the preset similarity threshold.

[0079] Step S190: Use the trained small network model to perform semantic recognition on the first recognition result according to preset keywords to obtain a second recognition result.

[0080] An implementation example of the above step S190 is as follows: find out the user text and unrecognized user text that have been recognized by the large language model from the first recognition result, construct a prompt engineering text based on the preset keywords and the unrecognized user text, and input the prompt engineering text into the small network model so that the small network model returns the semantic similarity probability between the unrecognized user text and the preset keywords. Then, the second recognition result is determined based on the semantic similarity probability. Specifically, if the semantic similarity probability is greater than the preset similarity threshold, the second recognition result is determined to be the intention of the preset keyword. If the semantic similarity probability is less than or equal to the preset similarity threshold, the second recognition result is determined to be not the intention of the preset keyword. Among them, the above second recognition result represents whether the semantic similarity probability between the unrecognized user text and the preset keyword is greater than the preset similarity threshold.

[0081] Step S200: determining a final recognition result according to the first recognition result and the second recognition result.

[0082] As an optional implementation of the above step S200, the implementation of determining the final recognition result according to the first recognition result and the second recognition result may include: for each user text in the unrecognized user text, judging whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold. If the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold, the second recognition result is determined as the final recognition result, that is, the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold and is determined as the final recognition result. In the implementation process of the above scheme, a larger language model can capture a wider range of semantic relationships, while a small network model is more focused on specific tasks. If both models confirm the correlation between the user text and the preset keyword, then this double confirmation can reduce the possibility of false positives, and the two recognitions can reduce the false positives that may exist in a single model, especially when dealing with fuzzy or boundary situations, thereby improving the accuracy of the final recognition result through these two recognition processes.

[0083] Optionally, after obtaining the final recognition result of the above-mentioned semantic recognition, in the customer service operation quality inspection scenario, the customer service operation data can also be obtained, and the final recognition result of the above-mentioned semantic recognition and the customer service operation are judged based on the customer service operation data to determine whether they are compliant, that is, a quality inspection is performed on each user text in multiple user texts to verify whether the customer service meets the customer's demands in accordance with preset rules. Specifically, for example: assuming that the final recognition result of the above-mentioned semantic recognition is a return, and the customer service operation data is a refund, then the final recognition result of the above-mentioned semantic recognition (return) and the customer service operation (refund) are non-compliant. Assuming that the final recognition result of the above-mentioned semantic recognition is a return, and the customer service operation data is also a return, then the final recognition result of the above-mentioned semantic recognition (return) and the customer service operation (return) are compliant.

[0084] See also Figure 2 The flowchart of the semantic recognition method combining keyword matching and a small network model provided in the embodiment of the present application is shown; the embodiment of the present application provides a semantic recognition method, including:

[0085] Step S210: using preset keywords to match and identify the multiple user texts to be processed, and obtaining identified user texts and unidentified user texts.

[0086] Among them, the technical principle and implementation method of step S210 are similar to the technical principle and implementation method of the above-mentioned step S110, so the implementation method of step S210 is not repeated. If there are any unclear points, please refer to the implementation method of step S110.

[0087] Step S220: Use the small network model trained by the model training method described above to perform semantic recognition on the unrecognized user text according to preset keywords to obtain a semantic recognition result, which represents whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than a preset similarity threshold.

[0088] The implementation method of the above step S220 is, for example: if the performance index of the small network model exceeds the preset index threshold, the small network model trained by the model training method described above is used to perform semantic recognition on the unrecognized user text according to the preset keywords to obtain the semantic recognition result. Among them, the performance index of the small network model may include accuracy and recall, and the preset index threshold may include accuracy threshold and recall threshold. It can be judged whether the accuracy of the small network model exceeds the accuracy threshold and whether the recall of the small network model exceeds the recall threshold. If the accuracy of the small network model exceeds the accuracy threshold and the recall of the small network model exceeds the recall threshold, a prompt engineering text is constructed according to the preset keywords and the unrecognized user text, and the prompt engineering text is input into the small network model so that the small network model returns the semantic similarity probability between the unrecognized user text and the preset keywords. Then, the semantic recognition result is determined according to the semantic similarity probability. Specifically, if the semantic similarity probability is greater than the preset similarity threshold, the semantic recognition result is determined to be the intention of the preset keyword. If the semantic similarity probability is less than or equal to the preset similarity threshold, the semantic recognition result is determined to be not the intention of the preset keyword. The above semantic recognition result indicates whether the semantic similarity probability between the unrecognized user text and the preset keyword is greater than a preset similarity threshold.

[0089] As an optional implementation of the above method, after obtaining the recognized user text and the unrecognized user text, the following may also be included:

[0090] Step S230: Use the small network model to perform semantic recognition on the recognized user text according to preset keywords.

[0091] The technical principle and implementation of step S230 are similar to those of step S210, so the implementation of step S230 will not be described in detail. If there are any unclear points, please refer to the implementation of step S210. In the implementation of the above solution, the accuracy of recognition can be improved by using a small network model for semantic recognition, especially when processing data in specific fields, the small model may be more focused on the semantic relationships in these fields.

[0092] And / or, after obtaining the recognized user text and the unrecognized user text, the method may further include:

[0093] Step S240: using a large language model to perform semantic recognition on the recognized user text according to preset keywords.

[0094] The technical principle and implementation of step S240 are similar to those of step S180, so the implementation of step S240 will not be described in detail. If there are any unclear points, please refer to the implementation of step S180. Since the large language model has strong generalization ability, it can capture a wider range of semantic relationships, thereby improving the robustness of the model when processing different types of text.

[0095] In the implementation process of the above solution, by combining the results of the small network model and the large language model, the false positive problem that may exist in a single model can be reduced, especially when dealing with ambiguous or boundary situations, which can effectively improve the accuracy of semantic recognition.

[0096] See also Figure 3 The flowchart of the semantic recognition method combining keyword matching, a small network model and a large language model provided in the embodiment of the present application is shown; the embodiment of the present application provides a semantic recognition method, including:

[0097] Step S310: Use preset keywords to match and identify multiple user texts to be processed to obtain identified user texts and unidentified user texts.

[0098] Among them, the technical principle and implementation method of step S310 are similar to the technical principle and implementation method of the above-mentioned step S110, so the implementation method of step S310 is not repeated. If there are any unclear points, please refer to the implementation method of step S110.

[0099] Step S320: Use the large language model to perform semantic recognition on the unrecognized user text according to preset keywords to obtain a first recognition result.

[0100] Among them, the technical principle and implementation method of step S320 are similar to the technical principle and implementation method of the above-mentioned step S180, so the implementation method of step S320 will not be repeated. If there are any unclear points, please refer to the implementation method of step S180.

[0101] Step S330: Use the small network model trained using the method described above to perform semantic recognition on unrecognized user text according to preset keywords to obtain a second recognition result, wherein the model parameters of the small network model are less than the model parameters of the large language model.

[0102] Among them, the technical principle and implementation method of step S330 are similar to the technical principle and implementation method of the above-mentioned step S190, so the implementation method of step S330 will not be repeated. If there are any unclear points, please refer to the implementation method of step S190.

[0103] Step S340: Determine a final recognition result according to the first recognition result and the second recognition result.

[0104] As an optional implementation of the above step S340, the implementation of determining the final recognition result based on the first recognition result and the second recognition result may include: for each user text in the unrecognized user text, judging whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold. If the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold, the second recognition result is determined as the final recognition result, that is, the probability of semantic similarity between the user text and the preset keyword is greater than the preset similarity threshold and is determined as the final recognition result. In the implementation process of the above scheme, by first using a large language model for preliminary recognition and then using a small network model for secondary recognition, the advantages of each other can be complemented, and the two recognitions can reduce the false alarm problem that may exist in a single model, thereby improving the accuracy of the final recognition result.

[0105] See also Figure 4 The flowchart of the model training device provided in the embodiment of the present application is shown; the embodiment of the present application provides a model training device 400, including:

[0106] The text matching and recognition module 410 is used to use preset keywords to match and recognize multiple user texts to be processed, and obtain recognized user texts and unrecognized user texts.

[0107] The text random sampling module 420 is used to randomly sample unrecognized user text to obtain sampled user text.

[0108] The recognition and screening processing module 430 is used to use the large language model to perform recognition and screening processing on the recognized user text and the sampled user text according to preset keywords to obtain the screened user text.

[0109] The semantic alignment annotation module 440 is used to perform semantic alignment annotation on the filtered user text to obtain the filtered alignment annotation, which indicates whether the semantic intentions of the filtered user text and the preset keywords are the same.

[0110] The network model training module 450 is used to train a small network model using the filtered user text and preset keywords as training data and the filtered alignment annotations as training labels. The model parameters of the small network model are less than the model parameters of the large language model.

[0111] As an optional implementation of the above device, the identification screening processing module includes:

[0112] The prompt text generation submodule is used to generate a prompt engineering text for each user text in the recognized user text and the sampled user text according to the user text and preset keywords.

[0113] The prompt text input submodule is used to input the prompt engineering text into the large language model so that the large language model returns the semantic similarity probability between the user text and the preset keywords.

[0114] The user text elimination submodule is used to eliminate the user text from the recognized user text and the sampled user text if the semantic similarity probability is less than a preset similarity threshold.

[0115] As an optional implementation of the above device, the model training device further includes:

[0116] The text data enhancement module is used to use a large language model to perform data enhancement on the filtered user text according to preset keywords to obtain model-enhanced text, which is text with the same semantics as the preset keywords.

[0117] The enhanced alignment annotation module is used to perform semantic alignment annotation on the model enhanced text to obtain enhanced alignment annotation, which represents whether the semantic intentions between the model enhanced text and the preset keywords are the same.

[0118] The enhanced model training module is used to train the small network model using model enhanced text and preset keywords as training data and enhanced alignment annotations as training labels.

[0119] As an optional implementation of the above device, the model training device further includes:

[0120] The performance indicator judgment module is used to judge whether the performance indicator of the small network model exceeds the preset indicator threshold.

[0121] The recognition result acquisition module is used to use the small network model to perform semantic recognition on the unrecognized user text according to the preset keywords if the performance index of the small network model exceeds the preset index threshold, and obtain the semantic recognition result. The semantic recognition result indicates whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than the preset similarity threshold.

[0122] As an optional implementation of the above device, the model training device further includes:

[0123] The first semantic recognition module is used to use a large language model to perform semantic recognition on unrecognized user text according to preset keywords to obtain a first recognition result.

[0124] The second semantic recognition module is used to use a small network model to perform semantic recognition on the first recognition result according to preset keywords to obtain a second recognition result.

[0125] The recognition result determination module is used to determine the final recognition result according to the first recognition result and the second recognition result.

[0126] As an optional implementation of the above device, the recognition result determination module includes:

[0127] The recognition result judgment submodule is used to judge, for each user text in the unrecognized user text, whether the first recognition result and the second recognition result of the user text both indicate that the semantic similarity probability between the user text and the preset keyword is greater than a preset similarity threshold.

[0128] The final result determination submodule is used to determine the final recognition result from the second recognition result if both the first recognition result and the second recognition result of the user text show that the semantic similarity probability between the user text and the preset keyword is greater than a preset similarity threshold.

[0129] The present application also provides a semantic recognition device, including:

[0130] The first matching and identifying module is used to use preset keywords to perform matching and identifying on a plurality of user texts to be processed, and obtain identified user texts and unidentified user texts.

[0131] The text semantic recognition module is used to use the small network model obtained after training to perform semantic recognition on unrecognized user text according to preset keywords to obtain semantic recognition results. The semantic recognition results represent whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than a preset similarity threshold.

[0132] As an optional implementation of the above device, the semantic recognition device further includes:

[0133] The first text recognition module is used to use a small network model to perform semantic recognition on the recognized user text according to preset keywords.

[0134] And / or, a second text recognition module is used to use a large language model to perform semantic recognition on the recognized user text according to preset keywords.

[0135] The present application also provides a semantic recognition device, including:

[0136] The second matching and identifying module is used to use preset keywords to match and identify the multiple user texts to be processed, and obtain identified user texts and unidentified user texts.

[0137] The first result obtaining module is used to use the large language model to perform semantic recognition on the unrecognized user text according to preset keywords to obtain the first recognition result.

[0138] The second result acquisition module is used to use the small network model obtained after training to perform semantic recognition on unrecognized user text according to preset keywords to obtain a second recognition result, and the model parameters of the small network model are less than the model parameters of the large language model.

[0139] The final result determination module is used to determine the final recognition result according to the first recognition result and the second recognition result.

[0140] It should be understood that the device corresponds to the above-mentioned model training method embodiment and can perform the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0141] See also Figure 5 An electronic device 500 provided in an embodiment of the present application includes: a processor 510 and a memory 520, wherein the memory 520 stores machine-readable instructions executable by the processor 510, and when the machine-readable instructions are executed by the processor 510, the above method is executed.

[0142] The embodiment of the present application also provides a computer-readable storage medium 530, on which a computer program is stored, and the computer program is executed by the processor 510 to execute the above method. The computer-readable storage medium 530 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0143] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0144] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0145] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also be different from the order of occurrence marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is mainly determined by the functions involved.

[0146] In addition, each functional module of each embodiment in the embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present application. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradiction.

[0147] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.

Claims

1. A model training method, characterized in that: include: Use preset keywords to match and identify multiple user texts to be processed, and obtain identified user texts and unidentified user texts; Randomly sampling the unrecognized user text to obtain sampled user text; Using a large language model to perform recognition and screening processing on the recognized user text and the sampled user text according to the preset keywords to obtain screened user text; Performing semantic alignment annotation on the filtered user text to obtain a filtered alignment annotation, wherein the filtered alignment annotation indicates whether the semantic intentions of the filtered user text and the preset keyword are the same; The filtered user text and the preset keywords are used as training data, and the filtered alignment annotations are used as training labels to train the small network model, wherein the model parameters of the small network model are less than the model parameters of the large language model.

2. The method according to claim 1, characterized in that The using of the large language model to identify and filter the identified user text and the sampled user text according to the preset keywords includes: For each user text in the identified user text and the sampled user text, generating a prompt engineering text according to the user text and the preset keywords; Inputting the prompt engineering text into the large language model so that the large language model returns the semantic similarity probability between the user text and the preset keyword; If the semantic similarity probability is less than a preset similarity threshold, the user text is removed from the recognized user text and the sampled user text.

3. The method according to claim 1, characterized in that After obtaining the filtered user text, the method further includes: Using the large language model to perform data enhancement on the filtered user text according to the preset keywords to obtain model-enhanced text, where the model-enhanced text is text with the same semantics as the preset keywords; Performing semantic alignment annotation on the model enhanced text to obtain enhanced alignment annotation, wherein the enhanced alignment annotation indicates whether the semantic intentions between the model enhanced text and the preset keyword are the same; The model enhanced text and the preset keywords are used as training data, and the enhanced alignment annotations are used as training labels to train the small network model.

4. The method according to any one of claims 1 to 3, characterized in that: After the small network model is trained, the method further includes: Determine whether the performance index of the small network model exceeds a preset index threshold; If so, the small network model is used to perform semantic recognition on the unrecognized user text according to the preset keywords to obtain a semantic recognition result, wherein the semantic recognition result indicates whether the semantic similarity probability between the unrecognized user text and the preset keywords is greater than a preset similarity threshold.

5. The method according to any one of claims 1 to 3, characterized in that: After the small network model is trained, the method further includes: Using the large language model to perform semantic recognition on the unrecognized user text according to the preset keywords to obtain a first recognition result; Using the small network model to perform semantic recognition on the first recognition result according to the preset keywords to obtain a second recognition result; A final recognition result is determined according to the first recognition result and the second recognition result.

6. The method according to claim 5, characterized in that The determining a final recognition result according to the first recognition result and the second recognition result includes: For each user text in the unrecognized user text, determining whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold; If yes, the second recognition result is determined as the final recognition result.

7. A semantic recognition method, characterized in that: include: Use preset keywords to match and identify multiple user texts to be processed, and obtain identified user texts and unidentified user texts; A small network model is obtained by training using the method described in any one of claims 1 to 6, and semantic recognition is performed on the unrecognized user text according to the preset keywords to obtain a semantic recognition result, wherein the semantic recognition result represents whether the probability of semantic similarity between the unrecognized user text and the preset keywords is greater than a preset similarity threshold.

8. The method according to claim 7, characterized in that After obtaining the recognized user text and the unrecognized user text, the method further includes: Using the small network model to perform semantic recognition on the identified user text according to the preset keywords; And / or, using a large language model to perform semantic recognition on the recognized user text according to the preset keywords.

9. A semantic recognition method, characterized in that: include: Use preset keywords to match and identify multiple user texts to be processed, and obtain identified user texts and unidentified user texts; Using a large language model to perform semantic recognition on the unrecognized user text according to the preset keywords to obtain a first recognition result; Using a small network model trained by the method according to any one of claims 1 to 6, semantic recognition is performed on the unrecognized user text according to the preset keywords to obtain a second recognition result, wherein the model parameters of the small network model are less than the model parameters of the large language model; A final recognition result is determined according to the first recognition result and the second recognition result.

10. The method according to claim 9, characterized in that The determining a final recognition result according to the first recognition result and the second recognition result includes: For each user text in the unrecognized user text, determining whether the first recognition result and the second recognition result of the user text are both that the probability of semantic similarity between the user text and the preset keyword is greater than a preset similarity threshold; If yes, the second recognition result is determined as the final recognition result.

11. A model training device, characterized in that: include: A text matching and recognition module, used to use preset keywords to match and recognize multiple user texts to be processed, and obtain recognized user texts and unrecognized user texts; A text random sampling module, used for randomly sampling the unrecognized user text to obtain a sampled user text; An identification and screening processing module, used to use a large language model to perform identification and screening processing on the identified user text and the sampled user text according to the preset keywords to obtain screened user text; A semantic alignment annotation module, used to perform semantic alignment annotation on the filtered user text to obtain a filtered alignment annotation, wherein the filtered alignment annotation indicates whether the semantic intentions of the filtered user text and the preset keyword are the same; The network model training module is used to train a small network model using the filtered user text and the preset keywords as training data and the filtered alignment annotations as training labels, wherein the model parameters of the small network model are less than the model parameters of the large language model.

12. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is executed.

14. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images