Text retrieval method, apparatus, device, and medium

Abstract

The application is suitable for the field of digital medical treatment, and discloses a text retrieval method. The method comprises the following steps: acquiring a plurality of searched texts, adding a plurality of preset classification labels to each searched text respectively to obtain modified searched texts; acquiring a search text, adding a preset classification label to the search text to obtain a modified search text; inputting the modified searched texts and the modified search text into a preset text retrieval model; processing the modified searched texts by a first preset encoder to obtain a searched vector set; processing the modified search text by a second preset encoder to obtain a search vector set; calculating a matching score between the search text and each searched text according to the search vector and the searched vector set by a preset full-connection neural network; and determining a target text in the plurality of searched texts according to the matching score. The method of the application solves the problem that the existing text retrieval method cannot guarantee the search time length and accuracy at the same time.

Description

Technical Field

[0001] This application relates to the fields of natural language processing technology and digital healthcare, and in particular to a text retrieval method, apparatus, device, and medium. Background Technology

[0002] In the field of Natural Language Processing (NLP), with the rapid development of pre-trained models such as BERT, pre-trained models are widely used in NLP scenarios. In text retrieval scenarios, this is mainly achieved by encoding the user's input search text (query) and a massive amount of searched text (documents) into vectors. During the inference stage, the similarity between the query vector and the document vector is used to obtain the most matching result.

[0003] Text retrieval can be applied to a variety of technical fields. Natural language text can be medical text, which includes electronic healthcare records—digitalized personal health records such as medical records, electrocardiograms (ECGs), and medical images—all valuable for archiving and future reference. For example, in the medical field, a user enters "coronary heart disease" into a search box. The search engine matches relevant text content based on the user's input and sends the results to the user. This content could include medical records, ECGs, and maintenance guidelines related to coronary heart disease. Doctors can refer to the received medical records and ECGs to diagnose and treat coronary heart disease, while patients can refer to the maintenance guidelines to adjust their lifestyles and perform daily care.

[0004] In current text retrieval tasks, pre-trained models such as BERT are primarily used, mainly employing two approaches: The first approach concatenates the query and document text into the same pre-trained model, then obtains the final matching score by acquiring the [CLS] tag output vector. This approach requires matching the query against a massive number of documents one by one during the inference phase, a computationally intensive process that consumes significant time. The second approach inputs the query and document text into different pre-trained models, then obtains the matching score by performing an inner product of the [CLS] input vectors of the two models. In this approach, the query and document are input into different models, preventing sufficient data interaction between the texts and potentially leading to a loss of semantic information, thus compromising the accuracy of search results. Summary of the Invention

[0005] In view of this, this application provides a text retrieval method, apparatus, medium and device to solve the problem that existing text retrieval methods cannot simultaneously guarantee search time and accuracy.

[0006] A first aspect of this application provides a text retrieval method, the method comprising:

[0007] Multiple searched texts are obtained from a preset corpus, and multiple preset category tags are added to each of the searched texts to obtain the modified searched texts.

[0008] Obtain the search text and add a preset category tag to the search text to obtain the modified search text;

[0009] The modified searched text and the modified search text are input into a preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder, and a preset fully connected neural network.

[0010] The first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes the searched vectors corresponding to each preset category label;

[0011] The second preset encoder processes the modified search text to obtain the search vector corresponding to the search text;

[0012] The preset fully connected neural network calculates the matching score between the search text and each of the searched texts based on the search vector and each set of searched vectors.

[0013] Based on the matching score, a target text that matches the search text is determined from among the multiple searched texts.

[0014] A second aspect of this application provides a text retrieval device, the device comprising:

[0015] The modification module is used to obtain multiple searched texts from a preset corpus, add multiple preset category tags to each of the searched texts to obtain modified searched texts; and to obtain searched texts and add one of the preset category tags to the searched texts to obtain modified searched texts.

[0016] The processing module is used to input the modified searched text and the modified search text into a preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder, and a preset fully connected neural network; the first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes a searched vector corresponding to each preset category label; and the second preset encoder processes the modified search text to obtain a search vector corresponding to the search text; the preset fully connected neural network calculates a matching score between the search text and each set of searched vectors, respectively.

[0017] The matching module is used to determine the target text that matches the search text among a plurality of searched texts based on the matching score.

[0018] A third aspect of this application provides an electronic device, including a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor, wherein the processor implements the above-described text retrieval method when executing the instructions or code.

[0019] In a fourth aspect, this application provides a storage medium having instructions or code stored thereon, which, when executed by a processor, implement the above-described text retrieval method.

[0020] The aforementioned text retrieval method, apparatus, device, and medium solve the problems of high computational load and long retrieval time caused by concatenating the search text and the searched text into the same pre-trained model in existing technologies. Furthermore, by adding multiple pre-defined classification labels to the searched text, multiple searched vectors are obtained, representing the searched text from multiple perspectives. This effectively improves the interaction between the search text and the searched text, compensating for the drawback of insufficient interaction when the search text and the searched text are not in the same pre-defined encoder. Therefore, the above solution improves retrieval accuracy. Moreover, after the pre-defined text retrieval model is trained, the above solution can still generate corresponding searched vectors offline for all searched texts. During the inference phase, only the search vector is obtained, and it can be matched with a massive number of searched vectors to obtain the final result. This process effectively saves matching time and improves retrieval efficiency.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic diagram of a text retrieval method provided in an embodiment of this application is shown;

[0024] Figure 2 This paper shows a schematic diagram of the structure of a preset text retrieval model of a text retrieval method provided in an embodiment of this application;

[0025] Figure 3 A flowchart illustrating another text retrieval method provided in an embodiment of this application is shown;

[0026] Figure 4 This paper shows a structural block diagram of a text retrieval device provided in an embodiment of this application;

[0027] Figure 5 A structural block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The text retrieval method provided in this application can be applied to electronic devices with instruction or program execution capabilities. These electronic devices can be, but are not limited to, various servers, workstations, personal computers, laptops, etc. The difference in execution between different computing devices is merely a difference in the execution entity of the solution; those skilled in the art will foresee that running the solution on different computing devices will produce the same technical effect. The invention will now be described in detail through specific embodiments.

[0030] Please see Figure 1 As shown, Figure 1A flowchart illustrating a text retrieval method provided in an embodiment of the present invention includes the following steps:

[0031] S10: Obtain multiple searched texts from the preset corpus, add multiple preset category tags to each searched text, and obtain the modified searched text.

[0032] The method provided by this invention is used in text retrieval scenarios. Specifically, in a text retrieval scenario, there are multiple documents to be searched. The search text query is obtained through user input or other means, and the result matching the search text is retrieved from the multiple documents to be searched.

[0033] In specific application scenarios, for example, a pre-built corpus contains a massive amount of diagnostic records. Each diagnostic record serves as a searchable text. When a user enters a search term, the system automatically retrieves one or more diagnostic records that match the user's search term from the vast database. Specifically, for example, if a user enters "pneumothorax," the system automatically retrieves diagnostic records with "pneumothorax" as the diagnosis. This allows for statistical analysis of hospital diagnoses of pneumothorax and the development of new plans based on the statistical results, such as equipment procurement and departmental adjustments.

[0034] For example, the preset corpus includes information on multiple hospitals and their departments. When a user enters "Chaoyang" and "psoriasis," the system will automatically search the preset corpus and display information on hospitals located in Chaoyang and doctors within those hospitals who specialize in treating psoriasis. It can also further display the consultation hours of each doctor to facilitate appointment booking for the user.

[0035] Based on this, multiple search texts are first obtained from a pre-defined corpus to retrieve target texts that match the search text. After obtaining the search texts, pre-defined category tags are used to modify them so that matching operations can be performed using the modified search texts.

[0036] Preferably, multiple preset category tags are added to each searched text, including adding a preset number of CLS category tags before the beginning of each searched text.

[0037] It is understandable that CLS (classification) is a classification label. Placing the CLS classification label at the beginning of a sentence and processing it yields the sentence's representation vector, which can then be used for subsequent tasks. Preferably, the preset number can be 64, meaning 64 CLS classification labels are added before the beginning of the searched text.

[0038] In addition, a SEP tag can be added after the end of each searched text. The SEP tag separates two input sentences and can be used here to indicate the end of a searched text.

[0039] For example, for a searched text (document text), adding a preset number of CLS category tags before the first character and a SEP tag after the last character results in the modified searched text shown below:

[0040] [CLS];[CLS];…;[CLS];document;[SEP]

[0041] In this step, a preset number of CLS category tags are added before the first and last characters of each searched text. After processing, a preset number of representation vectors corresponding to the searched text can be obtained, thereby realizing text retrieval based on multiple angles (i.e., a preset number of angles) and making the retrieval results more accurate.

[0042] For example, for coronary heart disease, stents can be used as a treatment method. In a searched text, if there is information about stents, such as which diseases stents can be used for, the materials, price, and precautions of stents, then [CLS] tags can be set for different angles. Each [CLS] tag is used to represent a representation vector from different angles. For example, one [CLS] tag is used to represent whether the searched text is applicable to the treatment of coronary heart disease, another [CLS] tag is used to represent whether the searched text is related to medical implants, and yet another [CLS] tag is used to represent the unsuitable population for the searched text, such as patients with multiple severe stenosis of multiple coronary arteries.

[0043] Prior to step S10, the following steps are also included:

[0044] S01: Obtain at least one user's search records, wherein the search records include historical search text, searched text, and the user's reading records based on the searched text;

[0045] S02: Determine the original tag corresponding to each searched text based on the reading history;

[0046] S03: Build a preset corpus based on historical search text, searched text, and original tags.

[0047] In steps S01-S03, before retrieving multiple searched texts from the preset corpus, a preset corpus is first constructed based on historical information. Specifically, at least one user's search records are first retrieved. These records include the user's historical search text, the searched text corresponding to the historical search text, and the user's reading records after retrieving the searched text. After retrieving the search records, the original tags corresponding to each searched text are determined based on the reading records. Then, the historical search text, the searched text corresponding to the historical search text, and the original tags corresponding to the searched text are used as a set of samples. The collection of multiple sets of samples is used as the preset corpus.

[0048] For example, in everyday news text retrieval scenarios, a large number of users' historical news retrieval records are collected to construct a pre-defined corpus. The dataset of the pre-defined corpus is D = {(q1, d1, c1), ..., (q i d i c i ), ..., (q n d n c n )).

[0049] Where, q i For historical search text, d i This means that D is related to q. i Matching searched text, c i For the original tag, (q i d i c i ) represents a sample in the preset corpus, and n represents the number of samples in the preset corpus.

[0050] This embodiment utilizes the search records of multiple users to construct a preset corpus, which contains information from a massive amount of search records, laying the foundation for subsequent search operations.

[0051] In step S02, the original tags corresponding to each searched text are determined based on the reading records, including the following steps:

[0052] S021: Based on the reading history, determine whether the user has read the searched text after searching the historical search text;

[0053] S022: If so, set the original tag corresponding to the searched text to the first tag value;

[0054] S023: If not, set the original tag to the second tag value.

[0055] In steps S021-S023, based on the user's reading history after retrieving the searched text, corresponding original tags are set for the searched text. It is understood that when a user inputs historical search text and retrieves the searched text, if the user believes the searched text is what they want to find, they may read it; conversely, if the user believes the searched text is not what they want to find, they may not read it. Therefore, the reading history may show reading the searched text, or it may show not reading it. Based on this, the original tag can be set to a first tag value or a second tag value based on whether the user has read the searched text after retrieving it.

[0056] For example, regarding the aforementioned original label c i c i ∈{0, 1}, where 1 is the first tag value, indicating that the user has read the searched text; and 0 is the second tag value, indicating that the user has not read the searched text. Of course, other values ​​can be assigned to the first and second tag values ​​respectively, as long as the two values ​​are different.

[0057] For example, in medical text retrieval scenarios, a pre-defined corpus is constructed by collecting a large number of users' historical medical search records. Specifically, for example, a user may have previously searched for "coronary heart disease," i.e., historical search text q. i For coronary heart disease, the searched text d matched i This is information related to stents. If a user searches for coronary heart disease and then views information related to stents, it will be the original tag c. i Assigning the value 1, thus constructing a sample (q) i d i c i ).

[0058] This embodiment introduces original tags, which can reflect whether historical users agree with the retrieved searched text. Therefore, the original tags can be used in the subsequent model training process to improve the retrieval accuracy.

[0059] S20: Obtain the search text and add a preset category label to the search text to obtain the modified search text.

[0060] In this step, the search text query is obtained through user input, and the search text is modified using preset category tags so that the modified search text can be used for matching operations.

[0061] Preferably, a preset category tag is added to the search text, including adding a CLS category tag before the beginning of the search text.

[0062] In addition, a SEP tag can be added after the end of the search text.

[0063] For example, for a search text, adding a CLS category tag before the first character and a SEP tag after the last character results in the following modified search text:

[0064] [CLS]; query; [SEP]

[0065] In this step, the concept of CLS is as described above. By adding a CLS classification label before the first word of the search text, the representation vector corresponding to the search text can be obtained after processing. Then, by using the representation vector corresponding to the searched text and the representation vector corresponding to the search text, the target text corresponding to the search text can be matched among multiple searched texts.

[0066] S30: Input the modified searched text and the modified search text into the preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder and a preset fully connected neural network.

[0067] In this step, both the modified search text and the modified search text are input into a preset text retrieval model. The preset text retrieval model is then used to perform text retrieval, retrieving the target text that matches the search text from among multiple search texts.

[0068] Specifically, the preset retrieval model includes a first preset encoder, a second preset encoder, and a preset fully connected neural network. The first preset encoder can be used to encode the modified searched text, the second preset encoder can be used to encode the modified searched text, and the preset fully connected neural network can be used to match the searched text and the searched text based on the encoding results.

[0069] Figure 2 The figure shows a schematic diagram of the structure of a preset text retrieval model according to an embodiment of this application. As shown in the figure, the document encoder is a first preset encoder used to encode the searched text (document text); the query encoder is a second preset encoder used to encode the searched text (query text); the MLP (Multilayer Perceptron) in the figure is a preset fully connected neural network, also called a multilayer perceptron, which is a type of feedforward neural network used to match the searched text and the searched text, and to determine the target text that matches the searched text in the searched text.

[0070] The modified search text [CLS]; [CLS]; ...; [CLS]; document; [SEP] is input into the first preset encoder, and the modified search text [CLS]; query; [SEP] is input into the second preset encoder. The output results of the two preset encoders are processed and then input into a preset fully connected neural network to output the matching score between the search text and the searched text. The target text that matches the search text is then determined based on the matching score.

[0071] S40: The first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes the searched vectors corresponding to each preset category label.

[0072] In this step, the modified search text is input into the first preset encoder of the preset text retrieval model. The first preset encoder processes the modified search text. Since the modified search text includes multiple preset category labels, a search vector can be output for each preset category label, as shown in the attached figure. Figure 2 The e shown d1 e d2 , ...e dm , where m is the number of preset category tags, that is, m is the preset number defined above. In other words, for this search text, a set of search vectors consisting of a preset number of search vectors can be obtained.

[0073] Preferably, after obtaining the set of search vectors corresponding to the searched text, the set of search vectors can be saved. Then, in subsequent matching stages, only the search vectors corresponding to the searched text need to be generated. This method avoids regenerating the search vectors for each match, improving matching efficiency.

[0074] S50: The second preset encoder processes the modified search text to obtain the search vector corresponding to the search text.

[0075] In this step, based on the same principle, the modified search text is input into the second preset encoder in the preset text retrieval model. The second preset encoder processes the modified search text. Since the modified search text includes a preset category label, a search vector can be output for this preset category label, as shown in the attached figure. Figure 2 The e shown q .

[0076] The first preset encoder and the second preset encoder are each a pre-trained language representation model.

[0077] The first and second pre-programmed encoders are each a BERT (Bidirectional Encoder Representations from Transformers) pre-trained model. The main input of the BERT model is the original word vectors of each character / word in the text. These vectors can be randomly initialized or pre-trained using algorithms such as Word2Vector as initial values. The output is the vector representation of each character / word in the text after incorporating the semantic information of the entire text. In this embodiment, the modified search text and the searched text can be processed using the BERT pre-trained model to obtain the corresponding vector representations, namely the search vector and the searched vector. The search vector and the searched vector respectively contain the semantic information of the search text and the searched text.

[0078] S60: The preset fully connected neural network calculates the matching score between the search text and each searched text based on the search vector and each set of searched vectors.

[0079] In this step, the search vectors and the searched vectors output by the two preset encoders are processed, and the results are input into a preset fully connected neural network to output a matching score between the search text and the searched text.

[0080] In step S60, the preset fully connected neural network calculates the matching score between the search text and each searched text based on the search vector and each set of searched vectors, including the following steps:

[0081] S61: Take the inner product of the search vector with each searched vector in the set of searched vectors to obtain the sub-matching score corresponding to the searched vector;

[0082] S62: A pre-defined fully connected neural network processes multiple sub-matching scores to obtain the matching score between the search vector and the set of searched vectors, which is used as the matching score between the search text and the searched text.

[0083] In steps S61-S62, the search vector e is respectively... q With each search vector e in the set of search vectors di Perform the inner product to obtain the sub-matching score s of the search vector and each searched vector. i =e q *e di Among them, e di Let represent the search vector corresponding to the i-th preset category label. After obtaining the sub-matching score for each search vector, all sub-matching scores s1, s2...s are processed. m Input the search vector into a pre-defined fully connected neural network and obtain the matching score between the search vector and the entire set of searched vectors. This score is then used as the final match score between the search text and the text being searched.

[0084] S70: Based on the matching score, determine the target text that matches the search text among multiple searched texts.

[0085] In this step, based on the matching score of each searched text in the search text field, the searched text that best matches the search text can be determined. Specifically, the searched text with the highest matching score can be considered the most matching searched text and used as the target text.

[0086] Prior to step S70, the method further includes the following steps:

[0087] S701: Obtain the original label corresponding to each searched text in the preset corpus, and calculate the loss function value based on the matching score and the original label. The original label is used to identify the reading record of the searched text.

[0088] S702: If the loss function value does not reach the preset convergence condition, adjust the model parameters of the preset text retrieval model and return to the step of inputting the modified searched text and the modified search text into the preset text retrieval model.

[0089] In steps S701-S702, if the preset text retrieval model has not been trained, its retrieval accuracy cannot be guaranteed. Therefore, if the aforementioned preset text retrieval model has not been trained, it can be trained based on the matching score and the original tags to improve the accuracy of the model.

[0090] Specifically, the first step is to construct a cross-entropy loss function. The original labels corresponding to the searched text are obtained from a pre-defined corpus. The original labels and the matching scores of the searched text are then input into this cross-entropy loss function to obtain the corresponding loss function value. The convergence of the loss function value is then determined. If the loss function value converges, the optimization of the pre-defined text retrieval model is considered complete, and the model can be used for matching. If the loss model does not converge, it is considered that the pre-defined text retrieval model can be further optimized. Therefore, the parameters in the model are adjusted, and the process returns to the step of inputting the modified searched text and the modified search text into the pre-defined text retrieval model. This process is repeated until the model's loss function value reaches the pre-defined convergence condition.

[0091] Figure 3 A flowchart illustrating another embodiment of the text retrieval method provided in this application is shown, as illustrated, including the following steps:

[0092] S001: Obtain at least one user's search records, wherein the search records include historical search text, searched text, and the user's reading records based on the searched text;

[0093] S002: Based on the reading history, determine whether the user has read the searched text after searching the historical search text;

[0094] S003: If so, set the original tag corresponding to the searched text to the first tag value;

[0095] S004: If not, set the original tag to the second tag value;

[0096] S005: Construct a pre-defined corpus based on historical search text, searched text, and original tags;

[0097] S06: Obtain multiple searched texts from the preset corpus, add multiple preset category tags to each searched text, and obtain the modified searched text;

[0098] S007: Obtain the search text and add a preset category label to the search text to obtain the modified search text;

[0099] S008: Input the modified searched text and the modified search text into the preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder and a preset fully connected neural network;

[0100] S009: The first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes the searched vectors corresponding to each preset category label;

[0101] S010: The second preset encoder processes the modified search text to obtain the search vector corresponding to the search text;

[0102] S011: Take the inner product of the search vector with each searched vector in the set of searched vectors to obtain the sub-matching score corresponding to the searched vector;

[0103] S012: The preset fully connected neural network processes multiple sub-matching scores to obtain the matching score between the search vector and the set of searched vectors, which is used as the matching score between the search text and the searched text.

[0104] S013: Obtain the original label corresponding to each searched text in the preset corpus, and calculate the loss function value based on the matching score and the original label;

[0105] S014: Determine whether the loss function value has reached the preset convergence condition; if yes, proceed to step 015; if no, proceed to step 016.

[0106] S015: Based on the matching score, determine the target text that matches the search text among multiple searched texts;

[0107] S016: Adjust the model parameters of the preset text retrieval model and proceed to step S008.

[0108] The principles and effects of steps S001-S016 are similar to those of the aforementioned embodiments, and can be found in the relevant descriptions in the method embodiments. To avoid repetition, they will not be repeated here.

[0109] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0110] As can be seen, compared to existing text retrieval methods, the above scheme inputs the search text and the searched text into different pre-defined encoders, solving the problems of high computational load and long retrieval time caused by concatenating the search text and the searched text into the same pre-trained model in existing technologies. Furthermore, by adding multiple pre-defined classification labels to the searched text, the above scheme obtains multiple searched vectors, representing the searched text from multiple perspectives. This effectively improves the interaction between the search text and the searched text, compensating for the drawback of insufficient interaction when the search text and the searched text are not in the same pre-defined encoder. Therefore, the above scheme can improve retrieval accuracy. In addition, after the pre-defined text retrieval model is trained, the above scheme can still generate corresponding searched vectors offline for all searched texts. During the inference phase, only the search vector is obtained, and it can be matched with a massive number of searched vectors to obtain the final result. This process can effectively save matching time and improve retrieval efficiency.

[0111] In one embodiment, a text retrieval device is provided, which corresponds one-to-one with the text retrieval methods described in the above embodiments. For example... Figure 4 As shown, the text retrieval device includes: a modification module, a calculation module, and a matching module. Detailed descriptions of each functional module are as follows:

[0112] The modification module is used to obtain multiple searched texts from a preset corpus, add multiple preset category tags to each searched text to obtain modified searched texts; and to obtain searched texts and add a preset category tag to the searched texts to obtain modified searched texts.

[0113] The processing module is used to input the modified searched text and the modified search text into a preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder, and a preset fully connected neural network; the first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes searched vectors corresponding to each preset category label; and the second preset encoder processes the modified search text to obtain search vectors corresponding to the search text; the preset fully connected neural network calculates the matching score between the search text and each searched text based on the search vectors and each set of searched vectors respectively.

[0114] The matching module is used to determine the target text that matches the search text among multiple searched texts based on the matching score.

[0115] In one embodiment, the modification module is used for:

[0116] The original labels corresponding to each searched text are obtained from the preset corpus, and the loss function value is calculated based on the matching score and the original labels. The original labels are used to identify the reading records of the searched text.

[0117] If the loss function value does not reach the preset convergence condition, the model parameters of the preset text retrieval model are adjusted, and the process returns to the step of inputting the modified searched text and the modified search text into the preset text retrieval model.

[0118] In one embodiment, the arithmetic module is used for:

[0119] The sub-matching score corresponding to the search vector is obtained by taking the inner product of the search vector with each search vector in the set of search vectors.

[0120] A pre-defined fully connected neural network processes multiple sub-matching scores to obtain the matching score between the search vector and the set of searched vectors, which is used as the matching score between the search text and the searched text.

[0121] In one embodiment, the modification module is used for:

[0122] Add a preset number of CLS category tags before the beginning of each searched text; and,

[0123] Add a CLS category tag before the first part of the search text.

[0124] In one embodiment, the apparatus further includes a building module for:

[0125] Obtain at least one user's search records, wherein the search records include historical search text, searched text, and the user's reading records based on the searched text;

[0126] Determine the original tags for each searched text based on the reading history;

[0127] A pre-defined corpus is built based on historical search text, the text being searched, and the original tags.

[0128] In one embodiment, the building module is used for:

[0129] Based on the reading history, determine whether the user has read the searched text after searching the historical search text;

[0130] If so, set the original tag corresponding to the searched text to the first tag value;

[0131] If not, set the original tag to the second tag value.

[0132] In one embodiment, the first preset encoder and the second preset encoder are each a pre-trained language representation model.

[0133] In one embodiment, an electronic device is provided. This electronic device can be a mobile device such as a mobile phone, tablet, or in-vehicle mobile terminal, or other device with program execution capabilities. The internal structure diagram of this electronic device can be as follows: Figure 5 As shown, the electronic device includes a processor, memory, and a network module. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, instructions, or code. The internal memory provides an environment for the operation of the operating system and instructions or code in the non-volatile storage media. When the instructions or code are executed by the processor, they implement the functions or steps of the aforementioned text retrieval method. The network module may include a network interface and / or a wireless network module, allowing the electronic device to communicate with other devices or service platforms. Furthermore, the electronic device may also include a display screen and input devices.

[0134] In one embodiment, an electronic device is provided, including a memory, a processor, and instructions or code stored in the memory and executable on the processor, wherein the processor executes the instructions or code to perform the following steps:

[0135] Multiple searched texts are obtained from a preset corpus, and multiple preset category tags are added to each searched text to obtain the modified searched text.

[0136] Obtain the search text and add a preset category tag to the search text to obtain the modified search text;

[0137] The modified searched text and the modified search text are input into a preset text retrieval model, which includes a first preset encoder, a second preset encoder, and a preset fully connected neural network.

[0138] The first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes the searched vectors corresponding to each preset category label;

[0139] The second preset encoder processes the modified search text to obtain the search vector corresponding to the search text; the preset fully connected neural network calculates the matching score between the search text and each searched text based on the search vector and each set of searched vectors respectively.

[0140] Based on the matching score, the target text that matches the search text is determined from multiple searched texts.

[0141] In one embodiment, a storage medium is provided on which instructions or code are stored, which, when executed by a processor, perform the following steps:

[0142] Multiple searched texts are obtained from a preset corpus, and multiple preset category tags are added to each searched text to obtain the modified searched text.

[0143] Obtain the search text and add a preset category tag to the search text to obtain the modified search text;

[0144] The modified searched text and the modified search text are input into a preset text retrieval model, which includes a first preset encoder, a second preset encoder, and a preset fully connected neural network.

[0145] The first preset encoder processes the modified searched text to obtain a set of searched vectors corresponding to the searched text, wherein the set of searched vectors includes the searched vectors corresponding to each preset category label;

[0146] The second preset encoder processes the modified search text to obtain the search vector corresponding to the search text;

[0147] The pre-defined fully connected neural network calculates the matching score between the search text and each set of searched texts based on the search vector and each set of searched vectors.

[0148] Based on the matching score, the target text that matches the search text is determined from multiple searched texts.

[0149] It should be noted that the functions or steps that the storage medium or electronic device can achieve are described in the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through instructions or code. These instructions or code can be stored in a non-volatile readable storage medium. When executed, the instructions or code can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0152] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the units or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the units in the system within the embodiment can be distributed throughout the system as described in the embodiment, or they can be modified to reside in one or more systems different from this embodiment. The units in the above-described embodiment can be combined into one unit, or further divided into multiple sub-units.

[0153] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method of text retrieval, characterized by, The method comprises: obtaining a plurality of searched texts in a preset corpus, adding a plurality of preset classification labels to each of the searched texts to obtain modified searched texts; obtaining a search text and adding a preset classification label to the search text to obtain a modified search text; inputting the modified searched texts and the modified search text into a preset text retrieval model, wherein the preset text retrieval model comprises a first preset encoder, a second preset encoder and a preset fully connected neural network; the first preset encoder processes the modified searched texts to obtain a searched vector set corresponding to the searched texts, wherein the searched vector set comprises a searched vector corresponding to each of the preset classification labels; the second preset encoder processes the modified search text to obtain a search vector corresponding to the search text; the preset fully connected neural network calculates a matching score between the search text and each of the searched texts according to the search vector and each of the searched vector set; determining a target text matching the search text from the searched texts according to the matching score; before determining the target text from the searched texts according to the matching score, the method further comprises: obtaining an original label corresponding to each of the searched texts in the preset corpus, and calculating a loss function value according to the matching score and the original label, wherein the original label is used to identify the reading record of the searched text; if the loss function value does not reach a preset convergence condition, adjusting the model parameters of the preset text retrieval model, and returning to the step of inputting the modified searched texts and the modified search text into the preset text retrieval model.

2. The method of claim 1, wherein, The preset fully connected neural network calculates a matching score between the search text and each of the searched texts according to the search vector and each of the searched vector set, comprising: respectively performing inner product of the search vector and each of the searched vector set to obtain a sub-matching score corresponding to the searched vector; the preset fully connected neural network processes a plurality of sub-matching scores to obtain a matching score between the search vector and the searched vector set as a matching score between the search text and the searched text.

3. The method of claim 1, wherein, The method further comprises: respectively adding a preset number of CLS classification labels before the beginning of each of the searched texts; correspondingly, adding a preset classification label to each of the search texts, comprising: adding a CLS classification label before the beginning of the search text.

4. The method of claim 1, wherein, Before obtaining a plurality of searched texts in a preset corpus, the method further comprises: obtaining at least one user's retrieval record, wherein the retrieval record comprises a historical search text, a searched text and a reading record of the user based on the searched text; determine a corresponding original label of each of the searched texts according to the reading records respectively; construct the preset corpus according to the historical search texts, the searched texts and the original labels.

5. The method of claim 4, wherein, The method further includes: determine whether the user has read the searched text after searching the historical search text according to the reading records; if yes, set the original label corresponding to the searched text as a first label value; if no, set the original label as a second label value.

6. The method of claim 1, wherein the first preset encoder and the second preset encoder are each a pre-trained language representation model. The device includes:

7. A text retrieval apparatus characterized by comprising: a modification module configured to obtain a plurality of searched texts from a preset corpus, add a plurality of preset classification labels to each of the searched texts to obtain modified searched texts, and obtain a search text and add a preset classification label to the search text to obtain a modified search text; an operation module configured to input the modified searched texts and the modified search text into a preset text retrieval model, wherein the preset text retrieval model includes a first preset encoder, a second preset encoder and a preset fully connected neural network; the first preset encoder processes the modified searched texts to obtain searched vector sets corresponding to the searched texts, wherein each of the searched vector sets includes a searched vector corresponding to each of the preset classification labels; the second preset encoder processes the modified search text to obtain a search vector corresponding to the search text; and the preset fully connected neural network calculates a matching score between the search text and each of the searched texts according to the search vector and each of the searched vector sets respectively; a matching module configured to determine a target text matching the search text from the searched texts according to the matching score; The modification module is configured to obtain an original label corresponding to each of the searched texts from a preset corpus, and calculate a loss function value according to the matching score and the original label, wherein the original label is used to identify a reading record of the searched text; if the loss function value does not reach a preset convergence condition, adjust model parameters of the preset text retrieval model, and return to the step of inputting the modified searched texts and the modified search text into the preset text retrieval model. The program or instructions, when executed by the processor, implement the method of any one of claims 1 to 6.

8. A storage medium having stored thereon a program or instructions, characterized in that, The processor, when executing the program, implements the method of any one of claims 1 to 6.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, ​

Images