Text processing model training method, text processing method and related equipment

Abstract

The invention discloses a training method of a text processing model, a text processing method and related equipment, which are beneficial to improving the case reasoning effect, and the obtained answer text is more accurate and more suitable for business. The text processing model training method comprises the steps of selecting a first case text of which the similarity with a first problem text meets a condition from a case library through a text processing model, and generating a first prompt word based on the first problem text and the first case text; based on the first prompt word, determining a first answer text of the first question text; and optimizing the text processing model based on the first answer text to obtain a trained text processing model.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a training method for a text processing model, a text processing method, and related equipment. Background Art

[0002] The classic case-based reasoning (CBR) system includes at least the following three main steps: retrieval, reuse and revision, that is, to infer the solution to the problem by reusing and revising the retrieved similar cases.

[0003] However, the above three main steps need to be implemented with the help of different technologies. These technologies are isolated from each other and cannot bring the performance of the case reasoning system to a better state, resulting in inaccurate reasoning results. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a text processing model training method, a text processing method and related equipment, which are conducive to improving the case reasoning effect, and the obtained answer text is more accurate and more suitable for the business.

[0005] In order to achieve the above objectives, the present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for training a text processing model, comprising:

[0007] Selecting, from a case library, a first case text whose similarity with a first question text satisfies a condition through a text processing model, and generating a first prompt word based on the first question text and the first case text;

[0008] Based on the first prompt word, determining a first answer text for the first question text;

[0009] The text processing model is optimized based on the first answer text to obtain a trained text processing model.

[0010] In a second aspect, an embodiment of the present application provides a text processing method, including:

[0011] Selecting a target case text that satisfies a condition of similarity with a target question text from a case library through a text processing model, and generating a target prompt word based on the target question text and the target case text;

[0012] Based on the target prompt word, a target answer text of the target question text is determined.

[0013] In a third aspect, an embodiment of the present application provides a training device for a text processing model, comprising:

[0014] A first generating unit is used to select a first case text whose similarity with the first question text satisfies a condition from the case library through a text processing model, and to generate a first prompt word based on the first question text and the first case text;

[0015] A first determining unit, configured to determine a first answer text of the first question text based on the first prompt word;

[0016] An optimization unit is used to optimize the text processing model based on the first answer text to obtain a trained text processing model.

[0017] In a fourth aspect, an embodiment of the present application provides a text processing device, including:

[0018] A second generating unit is used to select a target case text whose similarity with the target question text satisfies a condition from the case library through a text processing model, and to generate a target prompt word based on the target question text and the target case text;

[0019] The second determining unit is used to determine a target answer text of the target question text based on the target prompt word.

[0020] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0021] processor;

[0022] a memory for storing instructions executable by the processor;

[0023] The processor is configured to execute the instructions to implement the text processing model training method as described in the first aspect; or, the processor is configured to execute the instructions to implement the text processing method as described in the second aspect.

[0024] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the training method of the text processing model as described in the first aspect; or, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the text processing method as described in the second aspect.

[0025] In the seventh aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute part or all of the steps in the training method of the text processing model as described in the first aspect or the text processing method as described in the second aspect.

[0026] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0027] The text processing model is introduced to implement the various steps of the above-mentioned case reasoning system, specifically including: first, using the text processing model to select the first case text that meets the similarity conditions with the first question from the case library to implement the retrieval step; second, generating the first prompt word based on the first question text and the first case text, and then performing case reasoning on the first question text based on the first prompt word to obtain the first answer text of the first question text, and implementing the reuse and revision steps. On this basis, using the optimization algorithm in the field of machine learning, the goal is to make the first answer text meet the quality requirements, optimize the text processing model based on the first answer text, and implement end-to-end optimization training of the text processing model. This optimization process will closely link the above-mentioned retrieval, reuse and revision steps, that is, the processing result of the previous step will be used as the processing process that affects the next step, and then affect the entire case reasoning process. Therefore, the above-mentioned optimization process can achieve the overall optimization of the case reasoning process without the need to optimize each step separately, avoiding the problem of poor overall case reasoning performance caused by the separation of the implementation technologies of each step. As a result, the trained text processing model has a higher accuracy overall. Case reasoning based on the text processing model can not only improve the accuracy of the reasoning results, but also be adapted to various tasks and better adapt to technology iterations and frequently changing business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 A schematic diagram of a case-based reasoning process provided for an embodiment of the present application;

[0030] Figure 2 A flowchart of a method for training a text processing model provided for one embodiment of the present application;

[0031] Figure 3 A flowchart of a method for training a text processing model provided in another embodiment of the present application;

[0032] Figure 4 A flowchart of a method for training a text processing model provided in another embodiment of the present application;

[0033] Figure 5 A flowchart of a text processing method provided for an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the structure of a text processing model training device provided in one embodiment of the present application;

[0035] Figure 7 A schematic diagram of the structure of a text processing device provided for one embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0038] The terms "first", "second", etc. in this specification and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in this specification and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0039] Some concept descriptions:

[0040] Case-based reasoning: using solutions to similar problems previously solved to solve new problems. The case-based reasoning process can be seen as a cycle, that is, a cycle of four main steps: retrieval, reuse, revision, and retention. Figure 1As shown, when encountering a problem to be solved, the problem to be solved is input into the case reasoning system in text form, and the case reasoning system will retrieve cases similar to the problem to be solved in the case library; then, the solutions in similar cases are reused in the reasoning process of the problem to be solved, that is, case reasoning is performed on the problem to be solved with reference to similar cases, and an initial solution to the problem to be solved is obtained, and the final solution to the problem to be solved is obtained by revising the initial solution; finally, the quality of the final solution is evaluated, and if the final solution meets the quality requirements, the final solution is retained, that is, based on the problem to be solved and the final solution, a new case is generated and added to the case library.

[0041] Prompt: text used by people to communicate with the model, used to guide the model to output text according to the user's needs. Prompts can be simple instructions or complex text, such as a text that includes a task background introduction, several task cases, etc. Generally speaking, the more detailed the prompts, the more accurately the model can understand and execute the text.

[0042] In-context Learning (ICL): A machine learning method that combines contextual information of the prompt word and the input text so that the model can perform tasks more accurately. This method uses the prompt word to guide the model to focus on a specific topic or task and uses the contextual information to generate the required output. Contextual learning of the prompt word helps improve the performance and generalization of the model in natural language processing tasks. In short, by adding multiple cases to the prompt word, it helps the model to answer questions better.

[0043] Chain of Thoughts (CoT): A technique used to improve the reasoning ability of models. It guides the model to generate a series of reasoning steps to solve the problem. Chain of Thoughts can be understood as a high-level prompt word. Chain of Thoughts not only includes the problem and the solution, but also includes the intermediate reasoning process to obtain the solution. At the same time, Chain of Thoughts can also guide the model to use its own reasoning process as output when generating a solution, thereby effectively improving the accuracy of the model output.

[0044] Prompt word template (Template): A structured text framework used to guide the model to generate a specific type of text. This template usually contains keywords, placeholders, and some special symbols (such as line breaks). After receiving these specific information, the model fills the content into the placeholders of the template according to these specific information to generate the corresponding text. For example, the template = "Please answer the following question\nQuestion: {}\nYour answer:", where \n is a line break and {} is a placeholder. Fill the question into the placeholder {} to get the corresponding prompt word.

[0045] Tokenizer: A concept used in natural language processing. In the process of text processing, the role of the tokenizer is to convert the raw text into a numerical form that the model can understand. This process usually involves splitting the text into smaller units (such as words, subwords, or characters), and then mapping these units to corresponding numerical IDs. These numerical IDs are usually indexes in a predefined vocabulary. The tokenizer is also responsible for processing special tokens, such as the Start of Sequence (SOS) and End of Sequence (EOS) tokens, as well as padding tokens and unknown tokens.

[0046] Embedding Layer is another concept used in natural language processing. During text processing, the embedding layer is used to receive the numeric IDs output by the tokenizer and convert them into dense vectors. These vectors capture the semantic information of the words and are trainable, which means that they are updated and optimized during the model training process. The output of the embedding layer is a vector of fixed dimension, which is usually the first layer of the model and is used to convert discrete symbolic representations into continuous vectors so that subsequent neural network processing can be performed. In general, the tokenizer is responsible for the preprocessing and conversion of the text, while the embedding layer is part of the model and is used to convert the discrete numerical values ​​output by the tokenizer into continuous vector representations. Both are important steps for the model to process text.

[0047] Encoder module: The encoder module is the core part of the text processing model and is responsible for converting the input text into a high-dimensional vector. The encoder module captures the semantic information of the input text through a multi-layer stacked neural network structure. The embedding layer is usually considered to be part of the encoder module, although in some text processing models, especially in Transformer architecture models, the embedding layer is sometimes viewed separately. In the Transformer architecture model, the embedding layer is responsible for converting the input discrete symbols (such as words, characters, etc.) into continuous dense vectors that contain the semantic information of the vocabulary. These dense vectors are then passed to the encoder module, which captures the semantic information of the input text through multi-layer attention mechanisms (such as self-attention mechanisms) and feedforward neural networks.

[0048] As mentioned above, since the case-based reasoning system is a multi-link system, each of the above steps needs to be implemented with the help of different technologies. These technologies are isolated from each other, making it impossible to bring the performance of the case-based reasoning system to a better state, resulting in inaccurate reasoning results.

[0049] In view of this, the embodiment of the present application proposes a training method for a text processing model, and introduces a text processing model to implement the various steps of the above-mentioned case reasoning system, specifically including: first, using the text processing model to select the first case text that meets the similarity conditions with the first question from the case library to implement the retrieval step; second, generating a first prompt word based on the first question text and the first case text, and then performing case reasoning on the first question text based on the first prompt word to obtain the first answer text of the first question text, and implement the reuse and revision steps. On this basis, using the optimization algorithm in the field of machine learning, the first answer text will be made to meet the quality requirements as the goal orientation, and the text processing model will be optimized based on the first answer text to achieve end-to-end optimization training of the text processing model. This optimization process will closely link the above-mentioned retrieval, reuse and revision steps, that is, the processing result of the previous step will be used as the processing process that affects the next step, and then affect the entire case reasoning process. Therefore, the above-mentioned optimization process can achieve the overall optimization of the case reasoning process without the need to optimize each step separately, avoiding the problem of poor overall case reasoning performance caused by the separation of the implementation technologies of each step. As a result, the trained text processing model has a higher accuracy overall. Case reasoning based on the text processing model can not only improve the accuracy of the reasoning results, but also be adapted to various tasks and better adapt to technology iterations and frequently changing business scenarios.

[0050] The embodiment of the present application also provides a text processing method, which uses a trained text processing model to perform case reasoning on any problem text to obtain a corresponding solution. The entire reasoning process can avoid the problem of inaccurate reasoning results caused by the separation of the implementation technologies of each step in the case reasoning process. In addition, the case reasoning solution based on the text processing model can be adapted to various tasks and is more adaptable to technology iterations and frequently changing business scenarios.

[0051] It should be understood that the training method of the text processing model and the text processing method provided in the embodiments of the present application can be performed by an electronic device. The so-called electronic device here can include terminal devices, such as smart phones, tablet computers, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, smart watches, vehicle terminals, aircraft, etc.; or, the electronic device can also include a server, such as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0052] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0053] Please refer to Figure 2, is a flow chart of a method for training a text processing model provided by an embodiment of the present application, the method comprising the following steps:

[0054] S202, selecting a first case text whose similarity with the first question text meets a condition from the case library through a text processing model, and generating a first prompt word based on the first question text and the first case text.

[0055] The text processing model has a retrieval function, which uses the retrieval function to retrieve the first case text whose similarity with the first question text meets the conditions in the case library. The first case text includes a case question text and a reference answer text of the case question text. As an example, the reference answer text is used to describe the solution to the case question text. As another example, the reference answer text is also used to describe the intermediate reasoning process of obtaining a solution from the case question culture. The first case text can be used as a reference for case reasoning on the first question text to obtain the corresponding answer text. In practical applications, the number of first case texts can be determined according to actual needs, and the embodiments of the present application are not limited to this.

[0056] In the above S202, the conditions to be satisfied by the first case text can be set according to actual needs. In addition, the text processing model can search for similar cases in the case library in various ways, which can be selected according to actual needs, and the embodiment of the present application does not limit this.

[0057] In one embodiment, selecting a first case text whose similarity with a first question text meets a condition from a case library includes the following steps: encoding the first question text to obtain a representation vector of the first question text, and encoding each case text in the case library to obtain a representation vector of each case text; then, calculating the distance between the representation vector of the first question text and the representation vector of each case text as the similarity between the first question text and each case text; finally, selecting the top k case texts with the highest similarity with the first question text from the case library to obtain k first case texts, where k is a positive integer.

[0058] In the above implementation, the similarity between the first question text and each case text is calculated by calculating the distance between the representation vectors obtained after encoding the first question text and each case text in the case library, and then the case text with higher similarity is selected as the first case text. The implementation is simple, and the text processing model can be simplified, computing resources can be saved, and retrieval efficiency can be improved.

[0059] In another embodiment, selecting a first case text whose similarity with the first question text meets the conditions from the case library includes the following steps: performing a cross-attention operation on each case text and the first question text in the case library to obtain the similarity of each case text; based on the similarity of each case text, selecting a case text that meets the first screening condition from the case library as the first case text.

[0060] The similarity of the case text is used to indicate the similarity between the case text and the first question text. As an example, the similarity of each case text can be obtained by: encoding the first question text to obtain a representation vector of the first question text, and encoding each case text to obtain a representation vector of each case text; determining a query representation vector based on the representation vector of the first question text; determining a key representation vector and a value representation vector of each case text based on the representation vector of each case text; performing a cross-attention operation based on the query representation vector, the key representation vector and the value representation vector of each case text, and obtaining the similarity of each case text.

[0061] For example, a first linear mapping is performed on the representation vector of the first question text to obtain a query representation vector. The case library contains N case texts, and a second linear mapping is performed on the representation vector of each case text to obtain a key representation vector of each case text, and the key representation vectors of all case texts are concatenated to obtain a first key representation vector; a third linear mapping is performed on the representation vector of each case text to obtain a value representation vector of each case text, and the value representation vectors of all case texts are concatenated to obtain a first value representation vector. Then, the following formula (1) is used to perform an attention operation on the query representation vector, the first key representation vector, and the first value representation vector to obtain a similarity distribution vector, which includes N elements, and the N elements correspond one-to-one to the N case texts, and each element represents the similarity of the corresponding case text.

[0062]

[0063] Among them, P represents the similarity distribution vector, R Q represents the query representation vector, C K represents the first key characterization vector, (C K ) T Characterizes the transpose of the first key characterization vector, C V represents the first value characterization vector, d K represents the dimension of the first key representation vector, and softmax represents the normalized exponential function.

[0064] In the above method, the query representation vector is determined based on the representation vector of the first question text, and the key representation vector and value representation vector of each case text are determined based on the representation vector of each case text. Then, cross-attention operation is performed based on the three representation vectors, which is equivalent to taking the features of the first question text as the main feature and calculating the similarity between the feature and the features of each case text on this basis. The similarity calculated in this way is more accurate, which is conducive to accurately screening out the first case text that meets the similarity conditions with the first question text as a reference for case reasoning on the first question text.

[0065] The case retrieval of the case library based on the similarity of the case texts can be implemented in various ways, which can be selected according to actual needs, and the embodiments of the present application do not limit this. As an example, the first screening condition includes the similarity exceeding the preset similarity threshold. In this case, the case library can be sampled multiple times without replacement to obtain K case texts whose similarity meets the first screening condition as the K first case texts. As another example, the first screening condition includes the similarity being in the top K positions. In this case, the top K case texts with the highest similarity can be screened out from the case library as the K first case texts.

[0066] In practical applications, such as Figure 3 As shown, the text processing model may include a word segmenter, an embedding module, an encoding module, and an information collection module. Specifically, the word segmenter performs word segmentation on the first question text, maps the obtained multiple first units to corresponding digital IDs, and these digital IDs are converted into dense first vectors through the embedding layer; the embedding layer captures the semantic information of the first question text based on the first vector, and converts the first question text into a high-dimensional representation vector based on the semantic information. Similarly, for each case text in the case library, the word segmenter also performs word segmentation on the case text, maps the obtained multiple second units to corresponding digital IDs, and these digital IDs are converted into dense second vectors through the embedding layer; the embedding layer captures the semantic information of the case text based on the second vector, and converts the case text into a high-dimensional representation vector based on the semantic information.

[0067] The information collection module can be compatible with the changes in the scale of the case library, and adopts the cross-attention mechanism neural network structure to realize cross-field information collection, which specifically includes: performing cross-attention operations based on the representation vector of the first question text and the representation vector of each case text in the case library to obtain the similarity of each case text. As an example, the information collection module includes a first linear layer, a second linear layer, and a third linear layer, and all three linear layers can be trained. The first linear layer is also called the query layer, which is used to perform a first linear mapping on the representation vector of the first question text to obtain a query representation vector. The second linear layer is also called the key layer, which is used to perform a second linear mapping on the representation vectors of all case texts in the case library, and the mapped representation vectors are concatenated to form a first key representation vector. The third linear layer is also called the value layer, which is used to perform a third linear mapping on the representation vectors of all case texts in the case library, and the mapped representation vectors are concatenated to form a first value representation vector.

[0068] In the above S202, the first prompt word is used to indicate to perform case reasoning on the first question text with reference to the first case text. After selecting the first case text, the text processing model can generate the first prompt word in various ways, which can be selected according to actual needs, and the embodiment of the present application does not limit this.

[0069] In one embodiment, the first prompt word can be generated using a fixed prompt word template. Accordingly, based on the first question text and the first case text, generating the first prompt word includes the following steps: filling the first question text and the first case text into a preset fixed prompt word template to obtain the first prompt word.

[0070] Considering that the quality of the first prompt word will affect the accuracy of the case reasoning result in the case reasoning process, for example, when the first prompt word is input into the language model to use the ICL capability of the language model for case reasoning, the language model is not robust to the prompt word, and it is necessary to overcome the problem that the language model is sensitive to the prompt word. This problem is specifically manifested in that there will be a large difference between the two answer texts obtained by inputting the two end prompt words that are equivalent to humans into the language model for reasoning, while humans expect that the two answer texts should be highly consistent. In particular, the language model is sensitive to very simple changes in the prompt word (such as changing the separator ":" to "-"), which may cause the generated answer text to change from correct to wrong, or from wrong to correct. What is more complicated is that different language models, or the same language model for different case reasoning tasks, or the same language model for different question texts under the same case reasoning task, show different prompt word sensitivities, and there is no general rule. This makes the fixed prompt template not necessarily transferable, that is, it is not necessarily applicable to new problems to be solved, and then the prompt words generated based on the fixed prompt template are difficult to apply to various language models, case reasoning tasks, etc.

[0071] In view of this, in another embodiment, the influence of prompt words on the case reasoning results is taken into consideration, so that the text processing model learns how to select appropriate prompt word templates during the training process, ensuring that the improved prompt words can assist in improving the accuracy of case reasoning, and then the trained text processing model can automatically discover prompt templates that match the problem to be solved during the case reasoning process. Specifically, based on the first question text and the first case text, the first prompt word is generated, including the following steps: step A1, based on the first case text, determine the first prompt template that matches the first question text; step A2, fill the first question text and the first case text into the first prompt template to obtain the first prompt word.

[0072] In the above step A1, as an example, the prompt template used in the reasoning process of the first case text can be used as the first prompt template.

[0073] As another example, determine a set of selectable values ​​of multiple template elements for generating a first prompt template; and determine a first prompt template that matches the first question text based on the similarity of the first case text and the selectable value sets of multiple template elements.

[0074] As an example, the above-mentioned multiple template elements may include but are not limited to at least two of the following elements: task description, outer separator, input identifier, inner separator, output identifier, guide word, etc.

[0075] Each template element has a corresponding selectable value set, which includes at least one selectable value of the template element. For example, the following Table 1 shows an example of some template elements and their selectable value sets.

[0076] Table 1

[0077]

[0078] By combining the selectable values ​​of the multiple template elements shown in Table 1 above, multiple prompt templates can be obtained, and these prompt templates constitute the template solution space. In this case, through the text processing model, based on the similarity of the first case text, the matching degree of each prompt template and the first question text is predicted, and then the first prompt template matching the first question text is determined.

[0079] Therefore, in the process of training the text processing model to perform case reasoning, a fixed prompt template is not used. Instead, the influence of each template element on the case reasoning process is considered, and multiple template elements and their selectable value sets are pre-set, so that the text processing model can adaptively select appropriate values ​​for each template element to combine into the first prompt template, ensuring that the generated first prompt word can better assist the case reasoning process, thereby improving the accuracy of case reasoning.

[0080] As an example, based on the similarity of the first case text and the optional value set of multiple template elements, determining the first prompt template that matches the first question text includes the following steps: combining the optional values ​​of multiple template elements within the optional value range of multiple template elements to obtain multiple candidate prompt templates, each candidate prompt template including multiple template elements and an optional value for each template element; predicting the first matching degree of each candidate prompt template based on the similarity of the first case text; and selecting a candidate prompt template that meets the second filtering condition from the multiple candidate prompt templates as the first prompt template based on the first matching degree of each candidate prompt template.

[0081] The first matching degree of the candidate prompt template is used to indicate the degree to which the candidate prompt template matches the first question text.

[0082] For example, the selectable values ​​of multiple template elements are combined to obtain M candidate prompt templates. Then, the similarity of the first case text is aggregated through the text processing model, such as F = P*C V , get the collection vector, where F represents the collection vector, P represents the similarity of the first case text, C VRepresents the value representation vector obtained based on the representation vector of the first case text; then, the collection vector is subjected to a fourth linear mapping to obtain a tensor of M dimensions, where M is the number of candidate prompt templates; then, the tensor is operated using a normalized exponential function (i.e., a softmax function) to obtain the first matching degree of each candidate prompt template; finally, assuming that the second screening condition is that the first matching degree is the maximum, the candidate prompt template with the maximum first matching degree is screened out from the M candidate prompt templates as the first prompt template.

[0083] As another example, based on the similarity of the first case text and a set of optional values ​​of multiple template elements, determining a first prompt template that matches the first question text includes the following steps: based on the similarity of the first case text, predicting a second matching degree of each selectable value in the set of selectable values; based on the second matching degree of each selectable value, determining target values ​​of multiple template elements and combining them to obtain a first prompt template that matches the first question text.

[0084] For example, the similarity of the first case text is aggregated through the text processing model, such as F = P*C V , get the collection vector, where F represents the collection vector, P represents the similarity of the first case text, C V Represents a value representation vector obtained based on the representation vector of the first case text; then, for each template element, the collection vector is subjected to a fifth linear mapping to obtain a tensor of T dimension, wherein T is the number of selectable values ​​contained in the selectable value set of the template element; then, the tensor is operated using a normalized exponential function (i.e., a softmax function) to obtain a second matching degree of each selectable value of the template element; further, the selectable values ​​of all template elements are combined to obtain a plurality of candidate prompt templates, and the product of the second matching degrees of the selectable values ​​of each template element in the candidate prompt template is used as the first matching degree of the candidate prompt template; finally, assuming that the second screening condition is that the first matching degree is the maximum, the candidate prompt template with the maximum first matching degree is screened out from all candidate prompt templates as the first prompt template.

[0085] Compared with the template selection scheme described in the first example, the template selection scheme described in the second example can greatly reduce the number of matching degrees required to be predicted, thereby improving the efficiency of template selection. For example, there are 6 template elements in total, and the number of selectable values ​​of these template elements is 2, 3, 5, 3, 5, and 2 respectively. If the template selection scheme described in the first example is adopted, the number of matching degrees required to be predicted is 2*3*5*3*5*2=900; if the template selection scheme described in the second example is adopted, the number of matching degrees required to be predicted is 2+3+5+3+5+2=20. It can be seen that the number of matching degrees required to be predicted by the template selection scheme described in the second example is much smaller than the number of matching degrees required to be predicted by the template selection method described in the first example, and this difference in the number of matching degrees becomes more and more obvious as the number of template elements and the number of selectable values ​​of template elements increase, thereby significantly improving the efficiency of template selection.

[0086] In practical applications, such as Figure 3 As shown, the text processing model also includes a template prediction module. Each step of determining the first prompt template can be performed by the template prediction module.

[0087] In addition, the second screening condition is not limited to the maximum first match. For example, the first match is greater than a preset probability threshold, so that the candidate prompt template with a higher first match is more likely to be selected, but the candidate prompt template with a non-zero first match also has a chance to be selected, thereby encouraging the text processing model to explore other possibilities instead of sticking too much to the candidate prompt template with the maximum first match. This second screening condition helps the text processing model to strengthen learning and balance exploitation (i.e. sticking to the current optimal solution) and exploration (i.e. trying other better solutions).

[0088] In the above step A2, as an example, the first prompt template can also be a prompt template in the form of a few samples. By filling the first question text and the first case text into the first prompt template, a few samples can be obtained, and the first prompt word in the form of a few samples can be obtained, thereby guiding the model to refer to the reasoning process of the first case text, perform case reasoning on the first question text, and improve the accuracy of case reasoning.

[0089] For example, the following Table 2 is an example of a first prompt template in a few sample form. The first question text is filled into the {question text} position of the first prompt template, and the first case text is filled into the {case text} position of the first prompt template, and the first prompt word shown in the following Table 3 can be obtained.

[0090] Table 2

[0091]

[0092] Table 3

[0093]

[0094] As another example, the first prompt template can be a prompt template in the form of a thought chain. By filling the first question text and the first case text into the first prompt template, a first prompt word in the form of a thought chain can be obtained, thereby guiding the model to refer to the reasoning process of the first case text, perform case reasoning on the first question text, and output the case reasoning process in the output first answer text, thereby improving the accuracy of case reasoning.

[0095] For example, the following Table 4 is an example of a first prompt template in the form of a thought chain. The first question text is filled into the {question text} position of the first prompt template, and the first case text is filled into the {case text} position of the first prompt template, and the first prompt word shown in the following Table 5 can be obtained.

[0096] Table 4

[0097]

[0098]

[0099] Table 5

[0100]

[0101] S204: Determine a first answer text for the first question text based on the first prompt word.

[0102] In one embodiment, the first prompt word can be input into the language model, and the ICL capability of the language model can be used to perform case reasoning on the first question text to obtain the first answer text. The language model can adopt various large models that can generate text using generative language technology, such as a generative language model, etc., which is not limited in the embodiment of the present application.

[0103] For example, by inputting the first prompt word shown in Table 3 above into the language model, the first answer text shown in Table 6 below can be obtained, and by inputting the first prompt word shown in Table 5 above into the language model, the first answer text shown in Table 7 below can be obtained.

[0104] Table 6

[0105]

[0106] Table 7

[0107]

[0108]

[0109] In practical applications, the entire case reasoning process does not need to touch the language model parameters, so that the language model can use a privately deployed model or a model provided by other business parties. In this case, the first answer text can be obtained by calling the application programming interface (API) service of the language model through access mode.

[0110] Since the language model has a rich knowledge reserve and strong reasoning ability, it shows a strong problem-solving ability. In particular, the contextual learning ability of the language model is highly similar to the case reasoning system in form. Specifically, the language model uses the contextual learning ability to refer to the first case text in the first prompt word to generate the first answer text of the first question text, while the case reasoning system retrieves suitable existing cases, reuses and revises the solutions of existing cases to obtain the solution to the problem to be solved. The effects of both are highly dependent on the retrieved suitable cases. Based on this, the language model can be used to achieve accurate case reasoning for the first question text.

[0111] S206, optimizing the text processing model based on the first answer text to obtain a trained text processing model.

[0112] In one embodiment, the quality score of the first answer text can be evaluated based on the difference between the reference answer text of the first question text and the first answer text, and then the text processing model can be optimized based on the quality score. Figure 3 As shown, the text processing model may further include a quality assessment module, and the above S206 may be executed by the quality assessment module.

[0113] For example, a policy gradient algorithm of reinforcement learning can be used to optimize the text processing model. Specifically, the current model parameter θ of the text processing model and the first question text definition q i is a state, and given a state, the action of the text processing model is defined as sampling the first case text {e k} and the first prompt template t j , that is, {e k},t j =π θ ({e k},t j |q i ). Accordingly, the first prompt word generated based on the first case text and the first prompt template is p i =P(q i ,{e k},t j ), and then input the language model to get the first answer text Wherein, LLM represents language model. Further, the reference answer text of the first question text is compared with the first answer text. If the two are consistent, the reward score of the first question text is determined to be the first score (positive number); if the two are inconsistent, the reward score of the first question text is determined to be the second score (negative number). The reward score of the first question text is recorded as If the number of the first question texts is multiple, the total reward score for the text processing model is Where N represents the number of first question texts, {e k},t j ~π θ ({e k},t j |q i ). Furthermore, the gradient-based optimization algorithm (such as Adam optimizer) batch sampling (batch-wise) solves the problem and obtains a batch gradient approximation of Among them, B represents the number of first question texts in a batch. Finally, as Figure 3 As shown, the word segmenter and embedding layer of the text processing model are frozen, and the parameters of the encoding module, the parameters of the information collection module, and the parameters of the template prediction module are adjusted.

[0114] It is worth noting that the above steps S202 to S206 are only a training process for the text processing model. In practical applications, the text processing model can be trained multiple times until the training stop condition is met. Among them, the training stop condition can be set according to actual needs, for example, the total reward score of the text processing model exceeds the preset score, etc., which is not limited in the embodiment of the present application.

[0115] In another embodiment of the present application, after the above S206, the training method of the above text processing model may further include the following steps: generating a third case text based on the first question text and the first answer text, and adding the third case text to the case library.

[0116] In one implementation, if the first answer text is similar to the reference answer text of the first question text, the first question text and the first answer text are concatenated to obtain a third case text.

[0117] In another embodiment, a fourth case text similar to the case question text is selected from the case library through a trained text processing model; a fifth case text containing the first question text is determined from the fourth case text; if the number of the fifth case text is not less than a preset number threshold, a third case text is generated based on the first question text and the first answer text.

[0118] For example, the case question text is used as the problem text to be solved, and the text processing model is used to retrieve K fourth case texts similar to the case question text in the case library. If the question texts contained in half of the fourth case texts are similar to the first question text, the first question text and the first answer text are combined to obtain the third case text.

[0119] The above implementation adopts the cross-validation technology idea, which can accurately evaluate the quality of the first answer text. When the quality of the first answer text meets the requirements, the first answer text and the first question text are combined into the third case text and added to the case library, which realizes the retention link of the case reasoning system for subsequent reference, which is conducive to the continuous improvement and optimization of the case reasoning ability of the text processing model. In addition, the retention step in the case reasoning process is also closely linked with the steps of retrieval, reuse and revision process, further improving the overall case reasoning performance.

[0120] Considering that the case library plays a key role in the case-based reasoning process and is the basis of case-based reasoning, the quality and diversity of the case library have an important impact on the performance and effect of case-based reasoning, which directly affects the accuracy and efficiency when solving new problems. In view of this, in another embodiment of the present application, the construction process of the case library is also optimized to improve the quality of the case library, thereby improving the performance and effect of case-based reasoning.

[0121] Specifically, Figure 4 FIG. 1 is a flow chart of a method for training a text processing model provided by another embodiment of the present application, the method comprising the following steps:

[0122] S402, obtaining candidate case texts.

[0123] As an example, the candidate case text includes a second question text. The second question text may be a manually set question text (also called a native question text), or a question text obtained from other case-based reasoning systems (also called a derived question text).

[0124] As another example, the candidate case text may also include a candidate answer text for the second question text, wherein the candidate answer text may be an answer text obtained by other case reasoning systems performing case reasoning on the second question text, and such answer text is also called a derived question text.

[0125] S404, determining a second answer text for the second question text based on the candidate case text and the second prompt template.

[0126] In one embodiment, the second prompt template may include an answer prompt template. The answer prompt template is used to indicate an answer to the second question text. Accordingly, the above S404 includes the following steps: generating an answer prompt word based on the second question text and the answer prompt text; generating a second answer text for the second question text based on the answer prompt word.

[0127] For example, the answer prompt template may adopt a zero-sample prompt template, as shown in Table 8. The second question text is filled into the corresponding position of the answer prompt template to obtain the answer prompt words shown in Table 9. Then, the answer prompt words are input into the language model to obtain the second answer text as shown in Table 10.

[0128] Table 8

[0129]

[0130] Table 9

[0131]

[0132] Table 10

[0133]

[0134] For another example, the answer prompt template may adopt a zero-sample prompt template in the form of a thought chain, as shown in Table 11 below. The second question text is filled into the corresponding position of the answer prompt template to obtain the answer prompt words shown in Table 12 below. Then, the answer prompt words are input into the language model to obtain the second answer text as shown in Table 13 below.

[0135] Table 11

[0136]

[0137] Table 12

[0138]

[0139] Table 13

[0140]

[0141]

[0142] In practical applications, the second question text and the second answer text can be combined as the second case text and added to the case library. Alternatively, the second answer text can be evaluated for quality through manual inspection or a powerful language model. If the quality meets the standard, the second question text and the second answer text can be combined as the second case text and added to the case library.

[0143] As another example, the second prompt template includes a correction prompt template. The correction prompt template is used to indicate that the candidate answer text of the second question text is to be corrected. Accordingly, the above S404 may include the following steps: generating a correction prompt word based on the second question text, the candidate answer text and the correction prompt template; and correcting the candidate answer text based on the correction prompt word to obtain a second answer text for the second question text.

[0144] For example, the second question text and the candidate answer text are filled into the correction prompt template to obtain the correction prompt words; then, the correction prompt words are input into the language model, and the ICL capability of the language model is used to correct the candidate answer text so that the obtained second answer text is adapted to the ICL capability and style of the text processing model, thereby facilitating the use of the text processing model.

[0145] In the above implementation, by correcting the second answer text obtained by case reasoning of the second question text by other case reasoning systems, the migration processing of existing cases in other case reasoning systems can be achieved. The implementation is simple and efficient, and plays a role in rapidly expanding the case library.

[0146] S406: Based on the second question text and the second answer text, generate a second case text and add the second case text to the case library.

[0147] In one implementation, the second question text and the second answer text are combined to obtain a second case text and add it to the case library.

[0148] S408, selecting a first case text similar to the first question text from the case library through a text processing model, and generating a first prompt word based on the first question text and the first case text.

[0149] The specific implementation of S408 above is Figure 2 The specific implementation of S202 in the illustrated embodiment is similar and will not be described in detail.

[0150] S410: Determine a first answer text for the first question text based on the first prompt word.

[0151] The specific implementation of the above S410 is Figure 2 The specific implementation of S204 in the illustrated embodiment is similar and will not be described in detail.

[0152] S412, optimizing the text processing model based on the first answer text to obtain a trained text processing model.

[0153] The specific implementation of S412 is the same as Figure 2 The specific implementation of S206 in the illustrated embodiment is similar and will not be described in detail.

[0154] One or more embodiments of the present application provide a training method for a text processing model, which introduces a text processing model to implement the various steps of the above-mentioned case reasoning system, specifically including: first, using the text processing model to select the first case text that meets the similarity conditions with the first question from the case library to implement the retrieval step; second, generating a first prompt word based on the first question text and the first case text, and then performing case reasoning on the first question text based on the first prompt word to obtain the first answer text of the first question text, and implement the reuse and revision steps. On this basis, using the optimization algorithm in the field of machine learning, the first answer text will be made to meet the quality requirements as the goal orientation, and the text processing model will be optimized based on the first answer text to achieve end-to-end optimization training of the text processing model. This optimization process will closely link the above-mentioned retrieval, reuse and revision steps, that is, the processing result of the previous step will be used as the processing process that affects the next step, and then affect the entire case reasoning process. Therefore, the above-mentioned optimization process can achieve the overall optimization of the case reasoning process without the need to optimize each step separately, avoiding the problem of poor overall case reasoning performance caused by the separation of the implementation technologies of each step. As a result, the trained text processing model has a higher accuracy overall. Case reasoning based on the text processing model can not only improve the accuracy of the reasoning results, but also be adapted to various tasks and better adapt to technology iterations and frequently changing business scenarios.

[0155] The present application also provides a text processing method, which uses the text processing model trained by the above training method to perform case reasoning on the problem to be solved, and obtains a solution with high accuracy. Figure 5 , is a flowchart of a text processing method provided by an embodiment of the present application, the method comprising the following steps:

[0156] S502, selecting a target case text whose similarity with the target question text meets the conditions from the case library through the text processing model, and generating a target prompt word based on the target question text and the target case text.

[0157] The specific implementation of S502 is the same as Figure 2 The specific implementation of S202 in the illustrated embodiment is similar and will not be described in detail.

[0158] S504, determining a target answer text of the target question text based on the target prompt word.

[0159] The specific implementation of the above S504 is Figure 2 The specific implementation of S204 in the illustrated embodiment is similar and will not be described in detail.

[0160] The text processing method provided in the embodiment of the present application uses the trained text processing model to perform case reasoning on any problem text to obtain the corresponding solution. The entire reasoning process can avoid the problem of inaccurate reasoning results caused by the separation of the implementation technologies of each step in the case reasoning process. In addition, the case reasoning solution based on the text processing model can be adapted to various tasks and is more adaptable to technology iterations and frequently changing business scenarios.

[0161] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] Based on the same inventive concept, the present application embodiment also provides a training device for a text processing model. Figure 6 , is a structural diagram of a text processing model training device 600 provided in an embodiment of the present application, the device comprising: a first generating unit 610, a first determining unit 620 and an optimizing unit 630.

[0163] The first generating unit 610 is used to select a first case text whose similarity with the first question text meets the condition from the case library through a text processing model, and generate a first prompt word based on the first question text and the first case text.

[0164] The first determining unit 620 is configured to determine a first answer text for the first question text based on the first prompt word.

[0165] The optimization unit 630 is used to optimize the text processing model based on the first answer text to obtain a trained text processing model.

[0166] Obviously, the text processing model training device 600 provided in the embodiment of the present application can be used as the above Figure 2 The training method of the text processing model shown in the figure is an execution subject, so that the training device of the text processing model can be implemented in Figure 2 The functions realized are as follows. Since the principles are the same, they will not be described in detail.

[0167] Based on the same inventive concept, the present application embodiment also provides a text processing device. Figure 7 , is a structural diagram of a text processing device 700 provided in an embodiment of the present application, the device comprising: a second generating unit 710 and a second determining unit 720.

[0168] The second generating unit 710 is used to select a target case text whose similarity with the target question text meets the condition from the case library through the text processing model, and generate a target prompt word based on the target question text and the target case text.

[0169] The second determining unit 720 is configured to determine a target answer text of the target question text based on the target prompt word.

[0170] Obviously, the text processing device 700 provided in the embodiment of the present application can be used as the above Figure 5 The execution body of the text processing method shown in the figure can thus realize the text processing device in Figure 5 The functions realized are as follows. Since the principles are the same, they will not be described in detail.

[0171] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 8 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0172] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0173] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0174] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a training device for the text processing model at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0175] Selecting, from a case library, a first case text whose similarity with a first question text satisfies a condition through a text processing model, and generating a first prompt word based on the first question text and the first case text;

[0176] Based on the first prompt word, determining a first answer text for the first question text;

[0177] The text processing model is optimized based on the first answer text to obtain a trained text processing model.

[0178] Alternatively, the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a text processing device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0179] Selecting a target case text that satisfies a condition of similarity with a target question text from a case library through a text processing model, and generating a target prompt word based on the target question text and the target case text;

[0180] Based on the target prompt word, a target answer text of the target question text is determined.

[0181] The above application Figure 2 The training device or the text processing model disclosed in the embodiment shown Figure 5The method performed by the text processing device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0182] The electronic device may also perform Figure 2 The method and the training device of the text processing model are implemented in Figure 2 , Figure 3 Figure 4 Alternatively, the electronic device may also perform Figure 5 Method, and implement the text processing device in Figure 5 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0183] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0184] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 2 The method of the embodiment shown is specifically used to perform the following operations:

[0185] Selecting, from a case library, a first case text whose similarity with a first question text satisfies a condition through a text processing model, and generating a first prompt word based on the first question text and the first case text;

[0186] Based on the first prompt word, determining a first answer text for the first question text;

[0187] The text processing model is optimized based on the first answer text to obtain a trained text processing model.

[0188] Alternatively, when the instruction is executed by a portable electronic device including a plurality of application programs, the portable electronic device can execute Figure 5 The method of the embodiment shown is specifically used to perform the following operations:

[0189] Selecting a target case text that satisfies a condition of similarity with a target question text from a case library through a text processing model, and generating a target prompt word based on the target question text and the target case text;

[0190] Based on the target prompt word, a target answer text of the target question text is determined.

[0191] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the training method of the text processing model or the text processing method as described in the embodiment of the present application.

[0192] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0193] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0194] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0195] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0196] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A training method for a text processing model, characterized in that: include: Selecting, from a case library, a first case text whose similarity with a first question text satisfies a condition through a text processing model, and generating a first prompt word based on the first question text and the first case text; Based on the first prompt word, determining a first answer text for the first question text; The text processing model is optimized based on the first answer text to obtain a trained text processing model.

2. The method according to claim 1, characterized in that The step of selecting a first case text from the case library whose similarity to the first question text satisfies a condition includes: Performing a cross-attention operation on each of the case texts in the case library and the first question text to obtain a similarity of each of the case texts; Based on the similarity of each of the case texts, a case text that meets a first screening condition is selected from the case library as the first case text.

3. The method according to claim 2, characterized in that The performing a cross-attention operation on each case text in the case library and the first question text to obtain the similarity of each case text includes: Encoding the first question text to obtain a representation vector of the first question text, and encoding each of the case texts to obtain a representation vector of each of the case texts; Determine a query representation vector based on the representation vector of the first question text; Based on the representation vector of each case text, determine the key representation vector and the value representation vector of each case text; A cross-attention operation is performed based on the query representation vector, the key representation vector and the value representation vector of each of the case texts to obtain the similarity of each of the case texts.

4. The method according to claim 1, characterized in that The step of generating a first prompt word based on the first question text and the first case text includes: Based on the first case text, determining a first prompt template matching the first question text; The first question text and the first case text are filled into the first prompt template to obtain the first prompt word.

5. The method according to claim 4, characterized in that The determining, based on the first case text, a first prompt template matching the first question text includes: Determining a set of selectable values ​​for a plurality of template elements used to generate the first prompt template; Based on the similarity of the first case text and the selectable value set of the multiple template elements, a first prompt template matching the first question text is determined.

6. The method according to claim 5, characterized in that The determining, based on the similarity of the first case text and the selectable value set of the plurality of template elements, a first prompt template matching the first question text comprises: Combining the selectable values ​​of the multiple template elements within the selectable value range of the multiple template elements to obtain multiple candidate prompt templates, each of the candidate prompt templates including the multiple template elements and a selectable value of each template element; Predicting a first matching degree of each of the candidate prompt templates based on the similarity of the first case text; Based on the first matching degree of each of the candidate prompt templates, a candidate prompt template that meets a second screening condition is selected from the multiple candidate prompt templates as the first prompt template.

7. The method according to claim 5, characterized in that The determining, based on the similarity of the first case text and the selectable value set of the plurality of template elements, a first prompt template matching the first question text comprises: Predicting a second matching degree of each selectable value in the selectable value set based on the similarity of the first case text; Based on the second matching degree of each of the selectable values, target values ​​of the multiple template elements are determined and combined to obtain a first prompt template matching the first question text.

8. The method according to claim 1, characterized in that Before selecting the first case text whose similarity with the first question text satisfies the condition from the case library through the text processing model, the method further includes: Acquire a candidate case text, wherein the candidate case text includes a second question text; Determining a second answer text for the second question text based on the candidate case text and the second prompt template; Based on the second question text and the second answer text, a second case text is generated and the second case text is added to the case library.

9. The method according to claim 8, characterized in that The second prompt template includes a response prompt template; The step of determining a second answer text for the second question text based on the candidate case text and the second prompt template includes: Generate a response prompt word based on the second question text and the response prompt template; Based on the response prompt word, a second answer text of the second question text is generated.

10. The method according to claim 8, characterized in that The second prompt template includes a revised prompt template, and the candidate case text also includes a candidate answer text of the second question text; The step of determining a second answer text for the second question text based on the candidate case text and the second prompt template includes: Generate a correction prompt word based on the second question text, the candidate answer text and the correction prompt template; The candidate answer text is modified based on the modification prompt word to obtain a second answer text for the second question text.

11. The method according to any one of claims 1 to 10, characterized in that After optimizing the text processing model based on the first answer text to obtain a trained text processing model, the method further includes: Generate a third case text based on the first question text and the first answer text; The third case text is added to the case library.

12. The method according to claim 11, characterized in that The first case text includes case question text; The step of generating a third case text based on the first question text and the first answer text includes: Selecting a fourth case text similar to the case question text from the case library through the trained text processing model; Determining a fifth case text including the first question text from the fourth case text; If the number of the fifth case texts is not less than a preset number threshold, a third case text is generated based on the first question text and the first answer text.

13. A text processing method, characterized in that: include: Selecting a target case text that satisfies a condition of similarity with a target question text from a case library through a text processing model, and generating a target prompt word based on the target question text and the target case text; Based on the target prompt word, a target answer text of the target question text is determined.

14. A training device for a text processing model, characterized in that: include: A first generating unit is used to select a first case text whose similarity with the first question text satisfies a condition from the case library through a text processing model, and to generate a first prompt word based on the first question text and the first case text; A first determining unit, configured to determine a first answer text of the first question text based on the first prompt word; An optimization unit is used to optimize the text processing model based on the first answer text to obtain a trained text processing model.

15. A text processing device, characterized in that: include: A second generating unit is used to select a target case text whose similarity with the target question text satisfies a condition from the case library through a text processing model, and to generate a target prompt word based on the target question text and the target case text; The second determining unit is used to determine a target answer text of the target question text based on the target prompt word.

16. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method of the text processing model as described in any one of claims 1 to 12; or, the processor is configured to execute the instructions to implement the text processing method as described in claim 13.

17. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the text processing model as described in any one of claims 1 to 12; or, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the text processing method as described in claim 13.

18. A computer program product, characterized in that The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the training method for a text processing model as described in any one of claims 1 to 12 or the text processing method as described in claim 13.

Images