Text retrieval model training method and text retrieval method, device, and apparatus

Abstract

To provide a text retrieval model training method and a text processing method, a device, an electronic apparatus, a storage medium, and a program.SOLUTION: A method includes steps of inputting a sample text to be queried into a first text retrieval model, acquiring a first output text mark, training the first text retrieval model on the basis of a text mark tag of the sample text to be queried and the first output text mark, and acquiring a target retrieval cell. The sample text to be queried corresponds to a first target reference text. The text mark tag is a reference text mark of the first target reference text. A plurality of reference text marks are determined on the basis of a reference text sequence. The reference text sequence is determined on the basis of vocabularies of a plurality of reference texts. The first text retrieval model is obtained by training using the plurality of reference texts and the corresponding reference text marks to output a reference text mark that corresponds to a reference text.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as deep learning, large-scale language models, natural language understanding and search, etc. More specifically, the present disclosure provides a method for training a text search model, a text processing method, an apparatus, an electronic device, and a storage medium. [Background technology]

[0002] With the development of artificial intelligence technology, a large language model (LLM) may be used to perform search tasks. Summary of the Invention

[0003] The present disclosure provides a method for training a text search model, a text processing method, an apparatus, an electronic device, a storage medium, and a program.

[0004] According to an aspect of the present disclosure, there is provided a method for training a text retrieval model, the method including: inputting a sample text to be queried into a first text retrieval model to obtain a first output text mark; training the first text retrieval model based on text mark tags of the sample text to be queried and the first output text mark to obtain a target text retrieval model, where the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, the text mark tags are reference text marks of the first target reference text, the plurality of reference text marks are determined based on a reference text sequence, and the reference text sequence is determined based on a vocabulary of the plurality of reference texts; and the first text retrieval model is obtained by training an initial text retrieval model using the plurality of reference texts and the corresponding plurality of reference text marks, thereby outputting the reference text marks corresponding to the reference text.

[0005] According to another aspect of the present disclosure, there is provided a text retrieval method, the method including: inputting a text to be queried into a target text retrieval model to obtain a current output result; and in response to determining that the current output result hits a second target reference text in the plurality of reference texts, setting the second target reference text as a search result corresponding to the text to be queried, where the target text retrieval model is obtained by training a first text retrieval model using a sample text to be queried and a text mark tag, the sample text to be queried corresponds to the first target reference text in the plurality of reference texts, and the text mark tag is a reference text mark of the first target reference text.

[0006] According to another aspect of the present disclosure, there is provided an apparatus for training a text retrieval model, the apparatus including: a first obtaining module for inputting a sample text to be queried into a first text retrieval model to obtain a first output text mark; and a training module for training the first text retrieval model based on a text mark tag of the sample text to be queried and the first output text mark to obtain a target text retrieval model, where the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, the text mark tag is a reference text mark of the first target reference text, the plurality of reference text marks are determined based on a reference text sequence, and the reference text sequence is determined based on a vocabulary of the plurality of reference texts, and the first text retrieval model is obtained by training an initial text retrieval model using the plurality of reference texts and the corresponding plurality of reference text marks, thereby outputting the reference text mark corresponding to the reference text.

[0007] According to another aspect of the present disclosure, a text search device is provided, the text search device including: a fourth acquisition module for inputting a text to be queried into a target text search model to obtain a current output result; and a determination module for determining, in response to determining that the current output result hits a second target reference text in the plurality of reference texts, determining that the second target reference text is a search result corresponding to the text to be queried, where the target text search model is obtained by training a first text search model using a sample text to be queried and a text mark tag, the sample text to be queried corresponds to the first target reference text in the plurality of reference texts, and the text mark tag is a reference text mark of the first target reference text.

[0008] According to another aspect of the present disclosure, there is provided an electronic device including at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform a method provided in the present disclosure.

[0009] According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to perform the methods provided in the present disclosure.

[0010] According to another aspect of the present disclosure, there is provided a computer program product which, when executed by a processor, implements the methods provided in the present disclosure.

[0011] It should be understood that the subject matter described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily apparent from the following specification. [Brief description of the drawings]

[0012] The drawings are provided for a better understanding of the invention and are not intended to limit the disclosure. [Figure 1] FIG. 1 is a flowchart of a method for training a text retrieval model according to an embodiment of the present disclosure. [Figure 2A] FIG. 2A is a schematic diagram of training an initial text retrieval model according to one embodiment of the present disclosure. [Figure 2B] FIG. 2B is a schematic diagram of training a first text retrieval model according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a flowchart of a text search method according to another embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram of a text search method according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a block diagram of an apparatus for training a text retrieval model according to one embodiment of the present disclosure. [Figure 6] FIG. 13 is a block diagram of a text search device according to another embodiment of the present disclosure. [Figure 7] FIG. 7 is a block diagram of an electronic device to which a method for training a text search model and / or a method for text search according to an embodiment of the present disclosure can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings, and various details of the embodiments of the present disclosure will be included for ease of understanding, which are merely illustrative. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description will omit the description of known functions and structures.

[0014] With the widespread use of legal documents such as laws and policies, users have an increasingly strong need to inquire about legal provisions and policies. Corresponding legal provision search systems and policy content search systems have also been developed. Taking the legal provision search system as an example, the legal provision search system may perform precise or fuzzy searches based on the content of a part of a legal provision, or may provide the full text of a law.

[0015] However, some legal provision search systems are aimed primarily at legal experts such as lawyers. If an article of a law is known to these experts, they can use the search system to search for legal provisions with full authority. This search system is essentially a database that contains all laws, and it is difficult to assist non-experts with little or no legal knowledge in searching for relevant laws.

[0016] Also, the search system provides search results based on the lexical similarity between the problem and the legal provisions. However, if the problem is simple or has legal keywords, the search system can provide good search results, but it is difficult to mine fine-grained lexical relationships between the problem and the legal provisions. This search system has difficulty in understanding complex problems. If the problem is complex, ambiguous or unclear, the search system has difficulty in providing good search results.

[0017] Thus, to perform searches based on relatively complex problems, the present disclosure provides a method for training a text retrieval model, which is described below.

[0018] FIG. 1 is a flowchart of a method for training a text retrieval model according to an embodiment of the present disclosure.

[0019] As shown in FIG. 1, the method 100 may include operations S110 to S120.

[0020] In operation S110, a sample text to be queried is input into a first text retrieval model, and a first output text mark is obtained.

[0021] In the embodiment of the present disclosure, the sample text to be queried may be a complex problem or a simple problem. For example, the sample text to be queried may be "building a steel-clad building in the west to block the sunlight of the low-rise housing complex, and whether there is a legal provision in the country". The sample text to be queried may be a complex problem.

[0022] In an embodiment of the present disclosure, the first text search model may be a generative artificial intelligence model.

[0023] In the embodiment of the present disclosure, the first text search model may be obtained by training an initial search model using a plurality of reference texts and a corresponding plurality of reference text marks. For example, the reference text may be a legal provision. The legal provision of the first reference text in the plurality of reference texts may be, for example, "Article 293 of the Civil Code of the People's Republic of China: When constructing a building, it shall not violate the relevant engineering construction markings of the state and shall not obstruct the ventilation, lighting, and solar radiation of adjacent buildings." The reference text mark of the first reference text may be "55648."

[0024] In the embodiment of the present disclosure, the multiple reference text marks may be determined based on the reference text sequence. The reference text sequence may be determined based on the vocabulary of the multiple reference texts. For example, the reference text sequence may be multiple reference texts arranged in a certain order. The order may be determined based on the vocabulary of the reference texts. When the vocabulary similarity of two reference texts is large, the difference between the corresponding two reference text marks is small. The multiple reference texts may include current legal provisions and repealed legal provisions. The second reference text in the multiple reference texts may be a legal provision that states "Article 89 of the Property Law of the People's Republic of China [Regulations on ventilation, lighting and solar radiation] When constructing a building, it shall not violate the construction markings related to the state and shall not obstruct the ventilation, lighting and solar radiation of adjacent buildings", and the reference text mark of the second reference text may be "55942".

[0025] In an embodiment of the present disclosure, the first text retrieval module can output a reference text mark corresponding to the reference text. For example, the first reference text is input into the first text retrieval model to obtain a reference text mark "55648". The second reference text is input into the first text retrieval model to obtain a reference text mark "55942".

[0026] In an embodiment of the present disclosure, the first output text mark may be a string of characters that is similar or identical to the reference mark.

[0027] In operation S120, a first text retrieval model is trained based on the text mark tags of the sample text to be queried and the first output text marks to obtain a target retrieval model.

[0028] In an embodiment of the present disclosure, the sample text to be queried may correspond to a first target reference text in a plurality of reference texts. For example, the sample text to be queried may correspond to the first reference text. The first reference text may be the first target reference text.

[0029] In an embodiment of the present disclosure, the text mark tag may be the reference text mark of the first target reference text, for example, the text mark tag of the sample text to be queried may be the reference text mark "55648".

[0030] In the embodiments of the present disclosure, various loss functions can be used to determine the difference between the text mark tags and the first output text mark, and the first text retrieval model can be trained based on the difference.

[0031] According to the embodiment of the present disclosure, the reference text mark is determined based on the reference text sequence related to vocabulary height, which helps the model to efficiently understand the corresponding training task. The first text retrieval model is trained using the reference text and the corresponding text, so that the model parameters of the first text retrieval model and the reference text have a very strong correlation, so that the first text retrieval model has the ability to search the target reference text from multiple reference texts, effectively avoiding the generated text being different from the actual reference text caused by the text retrieval model directly generating the text, and fully improving the authority of the text retrieval model. In addition, the first text retrieval model is trained using the sample text to be queried and the corresponding text mark tag, which can realize sufficient interaction between the sample text to be queried and the model parameters related to the reference text, which helps the model to understand complex problems, so that the model can accurately determine the relationship between the complex problems and legal provisions, and reduce the computational complexity.

[0032] Having described the training method of the present disclosure, the reference text marks of the present disclosure will now be further described.

[0033] In some embodiments, the multiple reference texts are derived from N reference text sets. A reference text set may include at least one reference text. N may be an integer greater than 1. For example, if the reference text is a legal provision, the entire text of the law to which the legal provision belongs may be one reference text set. The Civil Code - Property Rights Chapter may be one reference text set. Each legal provision in the Civil Code - Property Rights Chapter may be one reference text.

[0034] In some embodiments, the reference text sequence is determined based on the vocabulary of the multiple reference texts by the following operations: perform clustering processing on N reference text sets respectively, adjust the order of the reference texts in the N reference text sets, and obtain N processed text sets. For example, the full text of the law corresponding to the N reference text sets includes "Civil Code - Property Rights", "Civil Code - Contract" and "Education Law". Many of the legal provisions in "Civil Code - Contract" are related to contracts. Many of the legal provisions in "Education Law" are related to education. Therefore, different reference texts in the same reference text set have a close vocabulary relationship. Based on the vocabulary of each reference text in the reference text set, perform clustering processing on one or more reference text sets respectively, form one or more clusters in each clustered reference text set, adjust the order of the reference texts in these reference text sets, and obtain one or more processed text sets.

[0035] In some embodiments, the reference text sequence is determined based on the vocabulary of the multiple reference texts by the following operation: obtain a fusion text set based on the N post-processing text sets. For example, after performing a clustering process on the N reference text sets, the obtained N post-processing text sets can be N subsets of the fusion text set, respectively. Each subset may include all the reference texts in one post-processing text set.

[0036] In some embodiments, the reference text sequence is determined based on the vocabulary of the multiple reference texts by the following operations: based on the vocabulary of each of the multiple reference texts, a clustering process is performed on the fusion text set, and the order of the multiple reference texts in the fusion text set is adjusted to obtain a reference text sequence. For example, different legal provisions from different laws may have a close vocabulary relationship. Based on the vocabulary of each of the multiple reference texts in the fusion text set, a clustering process is performed on the fusion text set, and the order of the multiple reference texts is adjusted again to obtain a reference text sequence, so that the distance between different legal provisions with close or similar vocabulary in different laws is close. Then, the marks of the multiple reference texts may be determined sequentially based on the order of the multiple reference texts in the reference text sequence.

[0037] It can be understood that in some other embodiments, random integer marking may be performed on all reference texts in the set of N reference texts. That is, the mark of each reference text may be a random integer. However, the generative artificial intelligence model adopts an output method generated by one element (token). If random integer marking is performed on the reference texts, it is difficult for the generative artificial intelligence model to learn complex relationships between multiple reference texts.

[0038] According to an embodiment of the present disclosure, a clustering process is performed on N reference text sets to fully utilize the lexical relationships between the reference texts in the text sets. Furthermore, a clustering process is performed again on the multiple post-clustering text sets to fully utilize the lexical relationships between the reference texts in different text sets, and multiple reference texts are sorted in a lexically related order so that different reference texts with high lexical similarity have the same or similar reference text mark prefixes, so that the model can fully utilize the lexical relationships between different reference texts.

[0039] Having described the reference text and reference textmarks of the present disclosure above, the first text retrieval model of the present disclosure will now be further described.

[0040] In some embodiments, the first text retrieval model may be a large-scale language model, such as a Wenmin word model, a ChatGPT model, or any other large-scale language model.

[0041] In the embodiment of the present disclosure, the first text retrieval model is obtained by training an initial text retrieval model. The initial text retrieval model may be a pre-trained model after training using a large amount of text data, and has strong vocabulary understanding ability. Hereinafter, the training method of the initial text retrieval model will be described with reference to FIG. 2A.

[0042] FIG. 2A is a schematic diagram of training an initial text retrieval model according to one embodiment of the present disclosure.

[0043] In some embodiments, the first text retrieval model is obtained by using a plurality of reference texts and a corresponding plurality of reference text marks to train an initial text retrieval model by the following operations: input the reference texts into the initial text retrieval model, and obtain the second output text marks of the reference texts. As shown in Fig. 2A, the reference texts 201, 202, 203, and 204 can be input into the initial text retrieval model M200 respectively, and obtain the output text mark 2011 of the reference text 201, the output text mark 2021 of the reference text 202, the output text mark 2031 of the reference text 203, and the output text mark 2041 of the reference text 204. The output text marks 2011 to 2041 can be the second output text marks respectively.

[0044] In some embodiments, the first text retrieval model may be obtained by using a plurality of reference texts and a corresponding plurality of reference text marks to train an initial text retrieval model by the following operation: fully fine-tune the initial text retrieval model according to the second output text marks and the reference text marks. For example, the reference text 201 may be the first reference text. Thus, the reference text mark of the reference text 201 may be "55648". Also, for example, the reference text 202 may be the second reference text. The reference text mark of the reference text 202 may be "55942".

[0045] In an embodiment of the present disclosure, the full-scale fine-tuning of the initial text retrieval model based on the second output text mark and the reference text mark includes determining a cross-entropy loss based on the second output text mark and the reference text mark. The full-scale fine-tuning of the initial text retrieval model is based on the cross-entropy loss. For example, the initial text retrieval model may output a plurality of character strings and a plurality of correspondence probabilities. The character string with the largest correspondence probability is the second output text mark. The training goal of the model may be maximization of the inter-target likelihood. That is, the model parameters may be adjusted so that the correspondence probability of the character string matching the reference text mark is maximized. The cross-entropy loss function may be utilized to determine the cross-entropy loss between the second output text mark and the reference text mark. The full-scale fine-tuning of the initial text retrieval model is based on the cross-entropy loss corresponding to the reference text, to obtain a first text retrieval model. Also, for example, the cross-entropy loss between the output text mark 2011 of the reference text 201 and the reference text mark "55648" may be determined, and the model may be full-scale fine-tuned. We can also determine the cross entropy loss between the output text mark 2021 of the reference text 202 and the reference text mark “55942” and fine-tune the model again.

[0046] According to the embodiment of the present disclosure, the reference text and the reference text mark are used to train an initial text retrieval model, and the trained first text retrieval model can output a text mark according to the text, and the target reference text can be searched from a plurality of reference texts according to the output text mark, which contributes to improving the authority of the model and avoids the deterioration of the model accuracy caused by the model "optical illusion".

[0047] Having described above how this disclosure trains an initial text retrieval model, we now turn to a description of how it trains a first text retrieval model.

[0048] FIG. 2B is a schematic diagram of training a first text retrieval model according to an embodiment of the present disclosure.

[0049] In some embodiments, the sample text to be queried can be input into the first text retrieval model to obtain a first output text mark. As shown in FIG. 2B, the sample text to be queried 210 can be the above sample text to be queried: "Building a steel-clad building in the west to block the sunlight of the low-rise residential area, whether there is a legal provision in the country". The sample text to be queried 210 can be input into the first text retrieval model M201 to obtain an output text mark 211 of the sample text to be queried 210. The output text mark 211 can be the first output text mark.

[0050] In the embodiment of the present disclosure, the sample text to be queried may correspond to a first target reference text. For example, a plurality of training text pairs may be obtained. Taking a legal provision as an example, the training text pair may include a sample text to be queried and a corresponding answer legal provision. The corresponding answer legal provision may be the first target reference text. The reference text mark of the corresponding answer legal provision may be the text mark tag of the sample text to be queried. The sample text 210 to be queried may correspond to the reference text 201. The reference text 201 may be the first target reference text corresponding to the sample text 210 to be queried. The reference text mark "55648" of the reference text 201 may be the text mark tag of the sample text 210 to be queried.

[0051] In some implementations, a first text retrieval model may be trained based on the text mark tags of the sample text to be queried and the first output text tags.

[0052] In the embodiment of the present disclosure, training the first text retrieval model may include full-scale fine-tuning the first text retrieval model. For example, based on the text mark tag "55648" of the sample text 210 to be queried and the output text mark 211 of the sample text 210 to be queried, various loss functions (e.g., the above-mentioned cross-entropy loss function) can be used to determine a loss value. The loss value can be used to full-scale fine-tune the first text retrieval model. After training the first text retrieval model using all the sample texts to be queried and a part of the corresponding text mark tags, a second file retrieval model can be obtained. After training the model using all the sample texts to be queried and the corresponding text mark tags, the obtained model can be the target text retrieval model.

[0053] According to the embodiment of the present disclosure, after training a model using a plurality of reference texts and a plurality of reference text marks, the model is further trained again using a sample text to be queried and corresponding text mark tags, which effectively improves the model's ability to process complex problems and helps the model accurately output text marks corresponding to the problems.

[0054] The above description of the present disclosure takes as an example that the sample text to be queried corresponds to one first target reference text, but the present disclosure is not limited thereto, and the sample text to be queried may correspond to multiple first target reference texts, which will be described below.

[0055] In some embodiments, a reference text mark may include a first reference text submark and a second reference text submark. For example, the reference text mark "55648" may include a first reference text submark "55" and a second reference text submark "648". The reference text mark "55942" may include a first reference text submark "55" and a second reference text submark "942". As can be appreciated, the first reference text submark may be a prefix of the reference text mark.

[0056] In some embodiments, the first reference text submarks of the first target reference texts corresponding to the sample texts to be queried are the same, for example, the reference text mark "55648" and the reference text mark "55942" have the same first reference text submark "55".

[0057] For ease of understanding, the disclosure has been described above in terms of examples in which the reference text is a legal document, but the disclosure is not limited thereto and will be described below in detail.

[0058] In some embodiments, the reference text may be rule text, which may include provisions in files created by various authorities, such as legal provisions, policy file provisions, and standards file provisions.

[0059] The above describes the method for training a text retrieval model of the present disclosure. Below, the text retrieval method of the present disclosure will be described.

[0060] FIG. 3 is a flowchart of a text search method according to another embodiment of the present disclosure.

[0061] As shown in FIG. 3, the method 300 may include operations S310 to S320.

[0062] In operation S310, the text to be queried is input into the target text retrieval model, and the current output result is obtained.

[0063] In an embodiment of the present disclosure, the text to be queried may be a question entered by a user.

[0064] In the embodiment of the present disclosure, the target text retrieval model is obtained by training a first text retrieval model using a sample text to be queried and a text mark tag. The first text retrieval model may be a generative artificial intelligence model. The sample text to be queried may be a complex problem or a simple problem.

[0065] In an embodiment of the present disclosure, the sample text to be queried corresponds to a first target reference text in the plurality of reference texts, and the text mark tag is a reference text mark of the first target reference text. The reference text mark may be a character string.

[0066] In the embodiment of the present disclosure, the current output result may be a string.

[0067] In operation S320, in response to determining that the current output result hits the second target reference text in the plurality of reference texts, the second target reference text is a search result corresponding to the text to be queried.

[0068] In the embodiment of the present disclosure, the current output result can be searched from the multiple reference texts to determine whether the current output result hits the second target reference text. If the current output result hits one of the multiple reference texts, the hit reference text can be the second target reference text. The second target reference text can be returned to the user as a search result.

[0069] According to the embodiment of the present disclosure, a target search model can be obtained by training using sample text to be queried and corresponding text mark tags, and the accuracy of search results for complex problems can be efficiently improved. Furthermore, by using the target text search model to determine search results, it can consume less resources and reduce the cost of non-professionals to accurately search texts in specialized fields.

[0070] It can be understood that the target text retrieval model according to the above method 300 is obtained by training the first text retrieval model using the method 100. The above detailed descriptions about the first text retrieval model, the reference text, the reference text mark, and the sample text to be queried can be similarly applied to the first text retrieval model, the reference text, the reference text mark, and the sample text to be queried in the method 300, and the disclosure will not be repeated here.

[0071] The text search method of the present disclosure has been described above. The text search method of the present disclosure will be further described below.

[0072] FIG. 4 is a schematic diagram of a text search method according to an embodiment of the present disclosure.

[0073] In the embodiment of the present disclosure, inputting the text to be queried into the target text retrieval model and obtaining the current output result may include inputting the text to be queried into an encoder of the target text retrieval model and obtaining an encoding result. The encoding result is input into a decoder of the target text retrieval model, and beam search decoding is performed on the encoding result to obtain a plurality of current output results. As shown in FIG. 4, the encoder of the target text retrieval model M403 can encode the text to be queried 420 and obtain the encoding result. Then, the decoder of the target text retrieval model M403 can perform beam search decoding on the encoding result to obtain the current output result 421 and the current output result 422. If the above sample text to be queried 210 is the text to be queried 420, the current output result 421 may be "55648", and the current output result 422 may be "55942". According to the embodiment of the present disclosure, beam search decoding is performed on the encoding result, so that the model has the ability to output multiple results and comprehensively searches for reference texts that are highly relevant to the text to be queried.

[0074] In an embodiment of the present disclosure, the current output result may be an integer string, for example, the output result of the target text retrieval model is arranged so as to reduce the likelihood probability of a non-integer string and sufficiently improve the probability of an integer string.

[0075] In the embodiment of the present disclosure, a search can be performed among multiple reference texts based on the current output result. For example, a search can be performed among multiple reference texts based on the current output result 421 and the current output result 422. It may be determined that the current output result 421 hits the first reference text, and the current output result 422 hits the second reference text. In this case, the first reference text and the second reference text can be search results of the text 420 to be queried.

[0076] It can be understood that the present disclosure has been described above by taking the example of performing beam search decoding on the encoding result. However, the present disclosure is not limited to this, and the encoding result may be decoded by other decoding methods.

[0077] It can be understood that the present disclosure has been described above by taking the example where the current output result hits one or more reference texts. In the following, a case where the current output result does not hit the reference text will be described as an example.

[0078] In some embodiments, the target text retrieval model may be a large language model, which may introduce a certain amount of randomness, for example, in the encoding and / or decoding processes.

[0079] In some embodiments, the method 300 may further include, in response to determining that the current output result does not hit any of the multiple reference texts, repeatedly performing an operation of inputting the text to be queried into the target text retrieval model until the post-output result of the current output result hits a second target reference text in the multiple reference texts. For example, if the current output result of the text to be queried does not hit any of the multiple reference texts, the text to be queried can be input again into the target text retrieval model to obtain one post-output result of the current output result. Then, it is determined whether the post-output result hits one or more of the multiple reference texts. If the post-output result hits the second target reference text in the multiple reference texts after one or more repetitions, the second target reference text is taken as a search result of the text to be queried. According to an embodiment of the present disclosure, if the output result does not hit the reference text, the robustness of the model can be further improved by using the model to perform re-inference.

[0080] It can be understood that the method of the present disclosure has been described above, and the apparatus of the present disclosure will now be described.

[0081] FIG. 5 is a block diagram of an apparatus for training a text retrieval model according to one embodiment of the present disclosure.

[0082] As shown in FIG. 5, the apparatus 500 may include a first acquisition module 510 and a training module 520 .

[0083] The first obtaining module 510 inputs the sample text to be queried into a first text retrieval model and obtains a first output text mark.

[0084] The training module 520 trains a first text retrieval model based on the text mark tags of the sample text to be queried and the first output text marks to obtain a target text retrieval model.

[0085] In an embodiment of the present disclosure, the sample text to be queried corresponds to a first target reference text in the plurality of reference texts, the text mark tag is a reference text mark of the first target reference text, the plurality of reference text marks are determined based on a reference text sequence, and the reference text sequence is determined based on a vocabulary of the plurality of reference texts.

[0086] In an embodiment of the present disclosure, the first text retrieval model is obtained by training an initial text retrieval model using a plurality of reference texts and a plurality of corresponding reference text marks, thereby outputting reference text marks corresponding to the reference texts.

[0087] In some embodiments, the first text retrieval model is obtained by training an initial text retrieval model using a plurality of reference texts and a corresponding plurality of reference text marks by the following modules: a second acquisition module, inputting the reference texts into the initial text retrieval model and obtaining a second output text mark of the reference text; a first fine-tuning module, fine-tuning the initial text retrieval model based on the second output text marks and the reference text marks to obtain the first text retrieval model.

[0088] In some embodiments, the first fine-tuning module includes a first determination sub-module for determining a cross-entropy loss based on the second output text mark and the reference text mark, and a first fine-tuning sub-module for globally fine-tuning the initial text retrieval model based on the cross-entropy loss.

[0089] In some embodiments, the multiple reference texts are from N reference text sets, the reference text sets including at least one reference text, and N is an integer greater than 1. The reference text sequence is determined based on the vocabulary of the multiple reference texts by the following modules: a first clustering processing module, which performs a clustering process on the N reference text sets respectively, and adjusts the order of the reference texts in the N reference text sets to obtain N processed text sets; a third obtaining module, which obtains a fused text set based on the N processed text sets; a second clustering processing module, which performs a clustering process on the fused text set based on the vocabulary of each of the multiple reference texts, and adjusts the order of the multiple reference texts in the fused text set to obtain a reference text sequence.

[0090] In some embodiments, the sample text to be queried corresponds to a plurality of first target reference texts, the reference text marks include a first reference text submark and a second reference text submark, and the first reference text submarks of the plurality of first target reference texts are the same.

[0091] In some embodiments, the training module includes a second fine-tuning sub-module for thoroughly fine-tuning the first text retrieval model.

[0092] In some embodiments, the reference text is a rule text and the first text retrieval model is a large-scale language model.

[0093] It can be understood that the training device of the present disclosure has been described above, and the text search device of the present disclosure will now be described.

[0094] FIG. 6 is a block diagram of a text search device according to another embodiment of the present disclosure.

[0095] As shown in FIG. 6, the apparatus 600 may include a fourth acquisition module 610 and a determination module 620.

[0096] The fourth obtaining module 610 inputs the text to be queried into the target text search model and obtains the current output result.

[0097] In response to determining that the current output result hits the second target reference text in the plurality of reference texts, the determining module 620 determines that the second target reference text is a search result corresponding to the text to be queried.

[0098] In an embodiment of the present disclosure, the target text retrieval model is obtained by training a first text retrieval model using a sample text to be queried and a text mark tag, where the sample text to be queried corresponds to a first target reference text in the multiple reference texts, and the text mark tag is a reference text mark of the first target reference text.

[0099] In an embodiment of the present disclosure, the target text retrieval model may be trained by the device 500, for example.

[0100] In some embodiments, the target text retrieval model is a large-scale language model. In response to determining that the current output result does not hit any of the multiple reference texts, the apparatus further includes an execution module for repeatedly performing an operation of inputting the text to be queried into the target text retrieval model until the output result after the current output result hits a second target reference text in the multiple reference texts.

[0101] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and other processing of such user personal information shall all comply with the provisions of relevant laws and shall not violate public order and morals.

[0102] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program.

[0103] 7 shows an exemplary block diagram for implementing an exemplary electronic device 700 according to an embodiment of the present disclosure. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure as described and / or claimed herein.

[0104] As shown in Fig. 7, the device 700 includes a computing unit 701, which can perform various appropriate operations and processes based on a computer program stored in a Read-Only Memory (ROM) 702 or loaded from a storage unit 708 into a Random Access Memory (RAM) 703. The RAM 703 can further store various programs and data required for the operation of the device 700. The computing unit 701, the ROM 702, and the RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0105] The components in the device 700 are connected to an I / O interface 705, including an input unit 706, e.g., a keyboard, a mouse, etc., an output unit 707, e.g., various types of displays, speakers, etc., a storage unit 708, e.g., a magnetic disk, an optical disk, etc., and a communication unit 709, e.g., a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network, such as the Internet, and / or various types of telecommunication networks.

[0106] The computing unit 701 may be various general-purpose and / or dedicated processing modules having processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for executing machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the methods and processes described above, such as the method for training a text search model and / or the method for text search. For example, in some embodiments, the method for training a text search model and / or the method for text search may be implemented as a computer software program tangibly included in a machine-readable medium, such as the storage unit 708. In some embodiments, some or all of the computer programs may be loaded and / or installed in the device 700 via the ROM 702 and / or the communication unit 709. The computer program, when loaded into RAM 703 and executed by the computing unit 701, may perform one or more steps of the above described method for training a text retrieval model and / or the text retrieval method. Alternatively, in another embodiment, the computing unit 701 may be configured to perform the method for training a text retrieval model and / or the text retrieval method in any other suitable form (e.g. via firmware).

[0107] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuit systems, integrated circuit systems, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs), Systems On Chips (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be embodied in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and may include a processor that is capable of receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

[0108] The program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer or other programmable data processing apparatus, so that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program codes may be fully executed on the device, partially executed on the device, partially executed on the device as a separate software package and partially executed on a remote device or fully executed on a remote device or server.

[0109] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may contain or store a program for use in or in combination with an instruction execution system, device, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or electronic device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections through one or more wires, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROMs) or flash memories, optical fibers, compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, a computer may implement the systems and techniques described herein and include a display device (e.g., a Cathode Ray Tube (CRT) display or a Liquid Crystal Display (LCD)) for displaying information to a user, and a keyboard and a pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices may also provide interaction with a user, for example, the feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and may receive input from the user in any form (including voice input, speech input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system including background components (e.g., a data server), or a computing system including middleware components (e.g., an application server), or a computing system including front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user may interact with the embodiments of the systems and techniques described herein), or a computing system including any combination of such background, middleware, or front-end components. The components of the system may be connected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks illustratively include a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0112] The computer system may include clients and servers. The clients and servers are generally remote and typically interact through a communication network. The relationship of client and server is created by computer programs running on the corresponding computers and having a client-server relationship.

[0113] It should be understood that various types of flows shown above may be used, and steps may be rearranged, added, or deleted. For example, each step described in the present invention may be performed in parallel, sequentially, or in a different order, and the present specification is not limited thereto as long as the desired results of the technical solution of the present disclosure can be achieved.

[0114] The specific embodiments described above do not limit the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. inputting a sample text to be queried into a first text retrieval model and obtaining a first output text mark; training the first text retrieval model based on the text mark tags of the sample text to be queried and the first output text marks to obtain a target text retrieval model; Wherein the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, the text mark tag is a reference text mark of the first target reference text, and the plurality of reference text marks are determined based on a reference text sequence, and the reference text sequence is determined based on a vocabulary of the plurality of reference texts; The first text retrieval model is obtained by training an initial text retrieval model using a plurality of the reference texts and a corresponding plurality of the reference text marks, thereby outputting the reference text marks corresponding to the reference texts. How to train a text search model.

2. The first text retrieval model is obtained by training an initial text retrieval model using a plurality of reference texts and a corresponding plurality of reference text marks by performing the following operations: The operation is inputting the reference text into an initial text retrieval model to obtain a second output text mark of the reference text; and fine-tuning the initial text retrieval model based on the second output text mark and the reference text mark to obtain the first text retrieval model. The method of claim 1.

3. fine-tuning the initial text retrieval model based on the second output text marks and the reference text marks includes: determining a cross-entropy loss based on the second output text mark and the reference text mark; and fine-tuning the initial text retrieval model based on the cross-entropy loss. The method of claim 2.

4. the plurality of reference texts are from N reference text sets, each of the reference text sets including at least one of the reference texts, N being an integer greater than 1; The reference text sequence is determined based on a vocabulary of a plurality of the reference texts by: The operation is Perform a clustering process on each of the N reference text sets, adjust the order of the reference texts in the N reference text sets, and obtain N processed text sets; Obtaining a fused text set based on the N processed text sets; performing a clustering process on the fused text set based on the vocabulary of each of the plurality of reference texts; and adjusting the order of the plurality of reference texts in the fused text set to obtain the reference text sequence. The method of claim 1.

5. The sample text to be queried corresponds to a plurality of the first target reference texts, the reference text marks include a first reference text submark and a second reference text submark, and the first reference text submarks of the plurality of the first target reference texts are the same. The method of claim 1.

6. Training the first text retrieval model comprises: and fine-tuning the first text retrieval model. The method of claim 1.

7. The reference text is a rule text and the first text retrieval model is a large-scale language model. The method of claim 1.

8. Inputting a text to be queried into a target text retrieval model to obtain a current output result; in response to determining that the current output result is a hit for a second target reference text in a plurality of reference texts, determining that the second target reference text is a search result corresponding to the text to be queried; Wherein, the target text retrieval model is obtained by training a first text retrieval model using a sample text to be queried and a text mark tag, the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, and the text mark tag is a reference text mark of the first target reference text. Text search methods.

9. The target text retrieval model is a large-scale language model; The method comprises: In response to determining that the current output result does not hit any of the plurality of reference texts, iteratively performing an operation of inputting the text to be queried into a target text retrieval model until an output result after the current output result hits a second target reference text in the plurality of reference texts. The method according to claim 8.

10. a first obtaining module for inputting a sample text to be queried into a first text retrieval model to obtain a first output text mark; a training module for training the first text retrieval model based on the text mark tags of the sample text to be queried and the first output text marks to obtain a target text retrieval model; Wherein the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, the text mark tag is a reference text mark of the first target reference text, and the plurality of reference text marks are determined based on a reference text sequence, and the reference text sequence is determined based on a vocabulary of the plurality of reference texts; The first text retrieval model is obtained by training an initial text retrieval model using a plurality of the reference texts and a corresponding plurality of the reference text marks, thereby outputting the reference text marks corresponding to the reference texts. A training apparatus for text retrieval models.

11. The first text retrieval model is a second obtaining module for inputting the reference text into an initial text retrieval model to obtain a second output text mark of the reference text; a first fine-tuning module for fine-tuning the initial text retrieval model based on the second output text mark and the reference text mark to obtain the first text retrieval model; The method is obtained by training an initial text retrieval model using a plurality of reference texts and a corresponding plurality of reference text marks.

11. The apparatus of claim 10.

12. The first fine-tuning module includes: a first determining sub-module for determining a cross-entropy loss based on the second output text mark and the reference text mark; and a first fine-tuning submodule that globally fine-tunes the initial text retrieval model based on the cross-entropy loss.

12. The apparatus of claim 11.

13. the plurality of reference texts are from N reference text sets, each of the reference text sets including at least one of the reference texts, N being an integer greater than 1; The reference text sequence is a first clustering processing module for performing a clustering process on each of the N reference text sets, adjusting the order of the reference texts in the N reference text sets, and obtaining N processed text sets; a third acquisition module for acquiring a fused text set based on the N processed text sets; a second clustering processing module for performing a clustering process on the fused text set based on the vocabulary of each of the plurality of reference texts, and adjusting the order of the plurality of reference texts in the fused text set to obtain the reference text sequence, determined based on the vocabulary of a plurality of said reference texts 11. The apparatus of claim 10.

14. The training module includes: a second fine-tuning submodule for fully fine-tuning the first text retrieval model.

11. The apparatus of claim 10.

15. The reference text is a rule text and the first text retrieval model is a large-scale language model.

11. The apparatus of claim 10.

16. A fourth acquisition module inputs the query text into the target text retrieval model to obtain a current output result; and a determining module for determining, in response to determining that the current output result is a hit for a second target reference text in a plurality of reference texts, that the second target reference text is a search result corresponding to the text to be queried; Wherein, the target text retrieval model is obtained by training a first text retrieval model using a sample text to be queried and a text mark tag, the sample text to be queried corresponds to a first target reference text in a plurality of reference texts, and the text mark tag is a reference text mark of the first target reference text; Text search device.

17. The target text retrieval model is a large-scale language model; The apparatus comprises: and an execution module for repeatedly executing an operation of inputting the text to be queried into a target text retrieval model until an output result after the current output result hits a second target reference text in the multiple reference texts in response to determining that the current output result does not hit any of the multiple reference texts.

17. The apparatus of claim 16.

18. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform the method of any one of claims 1 to 7. electronic equipment.

19. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause the computer to carry out a method according to any one of claims 1 to 7. A non-transitory computer-readable storage medium.

20. When executed by a processor, the method according to any one of claims 1 to 7 is implemented. Computer program.

Images