Semantic vector-based model training method and device for information retrieval

Abstract

The invention discloses a method and a device for training a model for performing information retrieval based on semantic vectors in the field of artificial intelligence. The method comprises the following steps: acquiring a first training data set and a second training data set; wherein the first training data is any piece of training data in a first training data set, and the first training data is composed of a first query text, a first retrieval text corresponding to the first query text and provided with a positive tag, and a second retrieval text corresponding to the first query text and provided with a negative tag; the second training data is any piece of training data in a second training data set, and the second training data is composed of a second query text, a third retrieval text corresponding to the second query text and a mark of the correlation degree between the second query text and the third retrieval text; updating the first information retrieval model based on the first training data set to obtain a second information retrieval model; and updating the second information retrieval model based on the second training data set to obtain a third information retrieval model. The performance of the information retrieval model can be improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method and device for training a model for information retrieval based on semantic vectors. Background Art

[0002] Information retrieval (IR) technology is a technology that finds information that meets user needs from a large amount of information. The information retrieval model based on semantic vectors is an important branch of information retrieval technology. The dual-tower model (also known as the dual-encoder model) is a common information retrieval model in retrieval technology based on semantic vectors, which has the characteristics of high retrieval efficiency. The principle of the dual-tower model can be briefly summarized as follows: the query text and the retrieval text are encoded by different encoders to obtain the semantic vector of the query text and the semantic vector of the retrieval text respectively; further, the similarity between the semantic vector of the query text and the semantic vector of the text to be retrieved is calculated, and based on the similarity, the text to be retrieved in the recommendation database is sorted.

[0003] In the process of training an information retrieval model, a large amount of high-quality training data is usually needed to train the information retrieval model to improve the performance of the information retrieval model; wherein, high-quality training data can generally be understood as training data consisting of a query text, a retrieval text corresponding to the query text, and annotations indicating the degree of relevance between the query text and the retrieval text. In the training process, if there is too little high-quality training data, the performance of the information retrieval model will be affected. Summary of the invention

[0004] The present application provides a method and device for training a model for information retrieval based on semantic vectors, which is conducive to reducing the dependence on a large amount of high-quality training data during the training process, and is also conducive to improving the performance of the information retrieval model in retrieving information.

[0005] It should be understood that the method provided in the embodiment of the present application can be executed by a training device for a model for information retrieval based on semantic vectors. The training device can be a terminal device, or a part of the device in the terminal device, such as a chip or processor used in the terminal device, or a server, such as a local server or a cloud server, or a part of the device in the server, such as a chip or processor used in the server, without limitation.

[0006] In a first aspect, the present application provides a training method for a model for information retrieval based on semantic vectors, the method comprising: obtaining training data of a first training data set and training data of a second training data set; wherein the first training data is any training data in the first training data set, the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; further, based on the training data of the first training data set, updating the first information retrieval model to obtain a second information retrieval model; and then based on the training data of the second training data set, updating the second information retrieval model to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output information retrieval results based on the semantic vector of the third query text.

[0007] Based on the method provided in the first aspect, when the amount of high-quality training data (i.e., the training data of the second training data set mentioned in the present application) is small, the information retrieval model can be first trained with training data whose quality is slightly lower than that of the high-quality training data (i.e., the training data of the first training data set mentioned in the present application) to increase the scale of the training data; and then the information retrieval model is trained with the high-quality training data to improve the quality of the semantic vectors represented in the information retrieval process, thereby reducing the dependence on a large amount of high-quality training data during the training process, and is also beneficial to improving the performance of the information retrieval model in retrieving information.

[0008] In a possible implementation, a first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked are obtained; the correlation between the first query text and the fourth search text, and the correlation between the first query text and the fifth search text are determined; further, the aforementioned first search text is determined from the fourth search text, and the first search text is a search text whose correlation with the first query text is greater than or equal to a first threshold; and the second search text is determined from the fifth search text, and the second search text is a search text whose correlation with the first query text is less than or equal to a second threshold, and whose correlation with the first query text is greater than or equal to a third threshold. By implementing this possible implementation, positive examples (i.e., search texts with positive labels) and negative examples (i.e., search texts with negative labels) are annotated according to the correlation between the query text and each search text, which is conducive to improving the quality of the training data in the first training data set obtained.

[0009] In a possible implementation, the first information retrieval model includes an encoder, in which case, a training text of a third training data set is obtained; and the training text of the third training data set is input into the encoder to obtain a first semantic vector corresponding to the training text, the first semantic vector being generated based on the word vectors of the words included in the training text and the position vectors of the words; the training text of the third training data set and the first semantic vector are input into the decoder to obtain a second semantic vector, the second semantic vector being generated based on the first semantic vector, the word vectors of the words included in the training text and the position vectors of the words; the text is reproduced based on the second semantic vector to obtain a reproduced text; further, the model parameters of the encoder are updated according to the reproduced text and the training text. By implementing this possible implementation, compared to the method of representing the semantic vector by the average value of the word vectors of all words included in the text, the present application represents the semantic vector of the text based on the word vectors of each word included in the text and the position vectors of each word, and the context information of each word in the text can be combined when generating the semantic vector of the text, which is conducive to improving the accuracy of the semantic vector of the text represented by the information retrieval model, thereby helping to improve the performance of the information retrieval model.

[0010] In a possible implementation, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text. By implementing this possible implementation, it is beneficial to reduce the vector distance between the query text and the positive example (i.e., the retrieval text with a positive label), and increase the vector distance between the query text and the negative example (i.e., the retrieval text with a negative label), thereby helping to improve the performance of the information retrieval model.

[0011] In a possible implementation, the loss function of the first information retrieval model satisfies the following conditions:

[0012]

[0013] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0014] In a second aspect, the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising: a transceiver unit 702, used to obtain training data of a first training data set and training data of a second training data set; wherein the first training data is any training data in the first training data set, and the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; a processing unit, used to update the first information retrieval model based on the training data of the first training data set to obtain a second information retrieval model; the processing unit is also used to update the second information retrieval model based on the training data of the second training data set to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output information retrieval results based on the semantic vector of the third query text.

[0015] Among them, the beneficial effects achieved by the device provided by the second aspect can refer to the beneficial effects corresponding to the method provided by the first aspect, and will not be described in detail here.

[0016] In combination with the second aspect, in a possible implementation, the transceiver unit 702 is further used to obtain a first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked; the processing unit is further used to determine the degree of correlation between the first query text and the fourth search text, and the degree of correlation between the first query text and the fifth search text; the processing unit is further used to determine the aforementioned first search text from the fourth search text, the first search text being a search text whose degree of correlation with the first query text is greater than or equal to a first threshold; the processing unit is further used to determine a second search text from the fifth search text, the second search text being a search text whose degree of correlation with the first query text is less than or equal to a second threshold, and whose degree of correlation with the first query text is greater than or equal to a third threshold.

[0017] In combination with the second aspect, in a possible implementation, the first information retrieval model includes an encoder. In this case, the transceiver unit 702 is also used to obtain training text of a third training data set; the processing unit is also used to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; the processing unit is also used to input the training text of the third training data set and the first semantic vector into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the processing unit is also used to reproduce the text based on the second semantic vector to obtain reproduced text; the processing unit is also used to update the model parameters of the encoder based on the reproduced text and the training text.

[0018] In combination with the second aspect, in one possible implementation, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

[0019] In conjunction with the second aspect, in a possible implementation manner, the loss function of the first information retrieval model satisfies the following conditions:

[0020]

[0021] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0022] In the third aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising a processor. When the processor calls a computer program in a memory, the method described in the first aspect is executed, thereby also achieving the beneficial effects of the method provided in the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising a processor and a memory, the processor and the memory being coupled; the processor is used to implement the method as described in the first aspect, and thus can also achieve the beneficial effects of the method provided in the first aspect.

[0024] In the fifth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising a processor, a memory and a transceiver, the processor and the memory being coupled; the transceiver is used to send and receive data, and the processor is used to implement the method described in the first aspect, thereby also achieving the beneficial effects of the method provided in the first aspect.

[0025] In the sixth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising a processor and an interface, the interface being used to receive or output signals, the processor being used to implement the method as described in the first aspect, thereby also being able to achieve the beneficial effects of the method provided in the first aspect.

[0026] In the seventh aspect, an embodiment of the present application provides a chip (or a group of chip systems), which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface to execute the method in the first aspect.

[0027] Optionally, as an implementation method, the chip (or chip system) may also include a memory, in which instructions are stored, and a processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the first aspect.

[0028] In the eighth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the training method of a model for information retrieval based on semantic vectors provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect, and can also achieve the beneficial effects of the method provided in the first aspect.

[0029] In the ninth aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on a device, the device executes the method provided in the first aspect above, and can achieve the beneficial effects of the method provided in the first aspect.

[0030] In a tenth aspect, a computer device is provided, which includes a training device for a model for information retrieval based on semantic vectors in any one of the second to sixth aspects above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1A This is a schematic diagram of an application scenario of information retrieval provided by this application;

[0032] Figure 1B This is another schematic diagram of an application scenario of information retrieval provided by this application;

[0033] Figure 1C This is another schematic diagram of an application scenario of information retrieval provided by this application;

[0034] Figure 2 It is a schematic diagram of a system architecture provided by this application;

[0035] Figure 3 It is a flowchart of a training method for a model for information retrieval based on semantic vectors provided in this application;

[0036] Figure 4 is a schematic diagram of an operation of obtaining training data of a first training data set provided by the present application;

[0037] Figure 5 It is a flowchart of an update information retrieval model provided by this application;

[0038] Figure 6 It is a schematic diagram of a flow chart of updating model parameters of an encoder provided by the present application;

[0039] Figure 7 It is a structural schematic diagram of a training device for a model of information retrieval based on semantic vectors provided by the present application;

[0040] Figure 8 It is a structural schematic diagram of another training device for a model for information retrieval based on semantic vectors provided in the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0042] The training method of the model for information retrieval based on semantic vectors provided in the embodiment of the present application can be applied to intelligent question-answering scenarios, natural language processing scenarios, and scenarios where information needs to be retrieved (or analyzed), or applied to human-computer interaction, information retrieval, information recommendation, information matching, and scenarios where information needs to be managed. Specifically, the training method of the model for information retrieval based on semantic vectors provided in the embodiment of the present application can be applied in information retrieval scenarios, and the following is a brief introduction to information retrieval scenarios.

[0043] As the name implies, information retrieval is to find information that meets user needs from a large amount of information. For example, a user enters a query text through a user device, and a data processing device analyzes the query text, retrieves a search text that matches the query text from a database that has a communication connection with the data processing device, and outputs the search text to the user device.

[0044] In order to facilitate understanding of the information retrieval scenarios mentioned in this application, this application provides several schematic diagrams of information retrieval application scenarios below.

[0045] See also Figure 1A As shown, it is a schematic diagram of an application scenario of information retrieval provided by the present application. The information retrieval system includes user devices and data processing devices. Among them, user devices include smart terminals such as mobile phones, personal computers or information processing centers. The user device is the initiator of information retrieval, and as the initiator of language questions and answers, retrieval or query requests, the user usually initiates the request through the user device. The data processing device can be a device or server with data processing functions such as a cloud server, a network server, an application server and a management server. The data processing device receives query statements / voice / text and other questions from the smart terminal through an interactive interface, and then performs language data processing in the form of machine learning, deep learning, search, reasoning, decision-making, etc. through a memory for storing data and a processor for data processing. The memory can be a general term, including local storage and a database for storing historical data. The database can be on a data processing device or on other network servers.

[0046] See also Figure 1B As shown in FIG. 1 , another application scenario of information retrieval provided by the present application is shown. The information retrieval system includes an intelligent terminal (or user device). The intelligent terminal can be directly used as a data processing device, that is, the intelligent terminal can directly receive input from the user, and can also directly process language data by the hardware of the intelligent terminal itself. The specific process is similar to Figure 1A Similarly, please refer to the above description and will not repeat them here.

[0047] See also Figure 1C As shown, it is a schematic diagram of another application scenario of information retrieval provided by the present application. The information retrieval system includes a local device (e.g. Figure 1C The local device 301 and the local device 302 in the embodiment of the present invention are provided, the execution device 210 and the data storage system 250. It is understood that the data storage system 250 can be integrated on the execution device 210, or can be set on the cloud or other network servers. Figure 1C The local device in can be understood as Figure 1A The user equipment in the embodiment, the execution device 210 can be understood as Figure 1A The data processing equipment in the Figure 1A Similarly, please refer to the above description and will not repeat them here.

[0048] It can be understood that the accuracy (or performance) of the information retrieval model in the information retrieval scenario will directly affect whether the output retrieval text meets the user's needs. The training method of the model for information retrieval based on semantic vectors provided in this application can improve the performance of the information retrieval model. In order to better understand the solution provided in this application, the system architecture of this application is first introduced below.

[0049] See also Figure 2 , Figure 2 A system architecture 200 is provided for the present application. As shown in the system architecture 200, the data acquisition device 260 is used to collect training data. The training data in the embodiment of the present application includes: question and answering (QA) data (i.e., question text and answer text), query text, and the search text corresponding to the query text that has been clicked by the user (which can be understood as the search text adopted, accepted or consulted by the user) and the search text that has not been clicked by the user (which can be understood as the search text that has not been adopted, accepted or consulted by the user), etc.; and the training data is stored in the database 230; the training device 220 obtains the target model / rule 201 based on the training data maintained in the database 230. The target model / rule 201 can be used to implement the information retrieval method provided in the embodiment of the present application, that is, the user inputs the query requirement (such as the query text) into the target model / rule 201, and the search text (or understood as the recommended text) that meets the query requirement can be obtained. The target model / rule 201 in the embodiment of the present application can specifically be an information retrieval model (or information retrieval network). In the embodiment provided in the present application, the information retrieval model can be obtained by training one or more combinations or deformations of neural networks (NN), convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN).

[0050] It should be noted that, in actual applications, the training data maintained in the database 230 may not all come from the data acquisition device 260, but may also be received from other devices. It should also be noted that the training device 220 may not train the target model / rule 201 entirely based on the training data maintained by the database 230, but may also obtain training data from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0051] Figure 2 The following is a functional module diagram of the data processing process, corresponding to Figure 1A to Figure 1C In the actual application scenario diagram in FIG. 1 , the client device 240 may be Figure 1A to Figure 1C The user device, the execution device 210 and the data storage system 250 are Figure 1A to Figure 1C When the data processing capability of the user equipment in the system is relatively strong, it can be integrated into the user equipment. In some embodiments, the execution device 210 and the data storage system 250 can also be integrated into Figure 1A to Figure 1C The database 230, the training device 220 and the data acquisition device 260 can be integrated on Figure 1A to Figure 1C It can be set on the data processing device in the system, or on other servers in the cloud or on the network.

[0052] In one possible implementation, Figure 2 The data collection device 260 may be a user device or an input / output interface of the aforementioned server or cloud, which is an interactive layer (interface) for obtaining query text (or query statement) and returning search text (or reply statement).

[0053] The target model / rule 201 trained by the training device 220 can be applied to different systems or devices, such as Figure 2 The execution device 210 shown. Figure 2 In the embodiment, the execution device 210 is configured with an I / O interface 212 for data interaction with an external device, and a user can input data (such as a query text in this application) into the I / O interface 212 through a client device 240 .

[0054] The association function module 213 is used to pre-process the input data (such as query text) received by the I / O interface 212. In the embodiment of the present application, the association function module 213 includes but is not limited to one or more of the following processing functions: text segmentation, basic error correction, blank removal, capitalization unification, punctuation removal, stop word removal, and special character retention for the query text.

[0055] When the execution device 210 pre-processes the input data through the associated function module 213, or when the calculation module 211 of the execution device 210 performs calculations and other related processing, the execution device 210 can call the data, code, etc. in the data storage system 250 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 250.

[0056] Finally, the I / O interface 212 returns the processing result, such as the retrieved text obtained above, to the client device 240 for providing to the user.

[0057] It is worth noting that the training device 220 can generate corresponding target models / rules 201 based on different data for different goals to provide users with better results.

[0058] In the attached Figure 2 In the case shown in , the user can manually specify the data to be input into the execution device 210, for example, by operating in the interface provided by the I / O interface 212. In another case, the client device 240 can automatically input data into the I / O interface 212 and obtain the result. If the client device 240 needs to obtain the user's authorization to automatically input data, the user can set the corresponding authority in the client device 240. The user can view the results output by the execution device 210 on the client device 240, and the specific presentation form can be a specific method such as display, sound, action, etc. The client device 240 can also serve as a data collection end to store the collected query text and search text into the database 230.

[0059] It is worth noting that Figure 2 This is only a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. Figure 2 In the embodiment, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0060] To facilitate understanding, the relevant terms / concepts involved in the embodiments of the present application are introduced below.

[0061] 1. Word vector and semantic vector

[0062] Word embedding technology is a general term for a class of language modeling and feature learning technologies in natural language processing (NLP). Through word embedding technology, words / phrases from the vocabulary can be mapped to the vector space of real numbers to obtain the vector representation of the word / phrase, that is, the word vector of the word / phrase. It should be noted that the word embedding technology mentioned in this application includes but is not limited to the word vector (Word2Vec) algorithm or the global word vector (Global Vectors for Word Representation, GloVe) algorithm, etc.

[0063] The semantic vector mentioned in this application can be understood as a vector of a text (or referred to as text) composed of multiple words / phrases.

[0064] 2. Loss Function

[0065] During the training process, because we hope that the output of the information retrieval model is as close as possible to the value we really want to predict, we can compare the output value corresponding to the current information retrieval model with the target value we really want, and then update the parameters of the information retrieval model according to the difference between the two. For example, if the output value corresponding to the information retrieval model is high, adjust the weight vector to make it lower, and continue to adjust until the output value corresponding to the information retrieval model is equal to the target value we really want, or the output value corresponding to the information retrieval model is very close to the target value we really want. Therefore, it is necessary to predefine "how to compare the difference between the output value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the output value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the information retrieval model becomes a process of minimizing this loss as much as possible.

[0066] 3. Dual-Encoder

[0067] Also known as the dual encoder neural network model, it refers to the independent encoding of the query text and the retrieval text by different encoders, and the output of the encoder is the semantic vector of the query text and the semantic vector of the retrieval text. Furthermore, the correlation between the semantic vector of the query text and the semantic vector of the retrieval text is calculated, and each retrieval text is sorted based on the correlation between each retrieval text and the query text. Among them, the correlation can be understood as the similarity between the two semantic vectors, and the similarity can be calculated by the cosine similarity calculation method or the vector dot product method; or, the correlation can be understood as the vector distance between the two semantic vectors, and the vector distance can be any one of the Euclidean distance, the vector distance calculated by cosine similarity, or the vector distance calculated by dot matrix.

[0068] In the training process of the information retrieval model, it is usually necessary to train the information retrieval model with a large amount of high-quality training data. If there is too little high-quality training data during the training process, it may affect the performance of the information retrieval model. In order to solve the problem of poor performance of the information retrieval model obtained when there is less high-quality training data, the present application provides a training method for a model for information retrieval based on semantic vectors, which can reduce the dependence of the information retrieval model on a large amount of high-quality training data during the training process, and is conducive to improving the performance of the obtained information retrieval model when there is less high-quality training data.

[0069] The following will be combined Figure 3 , which describes the technical solution in this application in detail. Figure 3 , Figure 3 : is a flowchart of a method for training a model for information retrieval based on semantic vectors provided in an embodiment of the present application. It is understandable that the method for training a model for information retrieval based on semantic vectors provided in the present application can be executed by a terminal device, or a chip in a terminal device, or a chip in a server, etc., and is not limited here. For the convenience of description, the following embodiments of the present application are described using a terminal device as an example. Figure 3 As shown, the training method of the above-mentioned model for information retrieval based on semantic vectors may include the following steps:

[0070] S301: Obtain training data of a first training data set and training data of a second training data set.

[0071] The first training data is any piece of training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text. The second training data is any piece of training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and a label of the degree of correlation (also called relevance, correlation, or similarity, etc.) between the second query text and the third search text.

[0072] The method for the terminal device to obtain the training data of the training data set (including the first training data set or the second training data set) includes but is not limited to one or more of the following methods: method 1, the terminal device obtains the training data of the training data set from its own storage; method 2, the terminal device obtains the training data of the training data set from a database to which it has access rights; method 3, the terminal device obtains the training data of the training data set from a data acquisition device with which it has a communication connection (the function of the data acquisition device can be found in Figure 2 The data acquisition device 260 (introduction in FIG. 2 ) collects training data to form a training data set.

[0073] It should be noted that the manner in which the terminal device obtains the training data of the first training data set and the manner in which the terminal device obtains the training data of the second training data set in the present application may be the same or different, and the present application does not specifically limit this. The acquisition source of the first training data set and the acquisition source of the second training data set may be the same or different, and the present application does not specifically limit this. The degree of correlation between the two texts mentioned in the present application can be understood as the vector distance between the semantic vectors corresponding to the two texts. The closer the vector distance is, the higher the degree of correlation is, and the farther the vector distance is, the lower the degree of correlation is. Among them, the vector distance can be any one of the Euclidean distance, the vector distance calculated by cosine similarity, or the vector distance calculated by dot matrix. Alternatively, the degree of correlation between the two texts mentioned in the present application can be understood as the similarity between the semantic vectors corresponding to the two texts, and the similarity can be calculated by methods such as cosine similarity calculation method or vector dot product.

[0074] For example, see the database 1 shown in Table 1, in which the first training data set and the second training data set are stored. The first training data set includes M training data, and the second training data set includes N training data, where M and N are both positive integers greater than 1, and M and N may be equal or unequal, which is not limited in this application. Any training data in the first training data set includes a query text q, a search text d with a positive label corresponding to the query text q, and a search text d with a positive label corresponding to the query text q. + and the retrieved text d with negative labels -For example, the training data 01 of the first training data set in Table 1 includes the query text q0, the search text with a positive label corresponding to the query text q0 and i (a positive integer greater than 1) search texts with negative labels Any training data in the second training data set includes a query text q', a retrieval text d' corresponding to the query text q', and a relevance score (or understood as a label) score between the query text q' and the retrieval text d'; for example, training data 02 of the second training data set in Table 1 includes a query text q'0, a retrieval text d'0 corresponding to the query text q'0, and a relevance score score 0 between the query text q'0 and the retrieval text d'0.

[0075] Table 1

[0076]

[0077] In this case, if the terminal device has access rights to the database 1 , the terminal device can obtain the training data of the first training data set and the training data of the second training data set from the database 1 .

[0078] In a possible implementation, the terminal device may obtain the first training data set by obtaining multiple training data from at least one data source (or database), and then processing the multiple training data to obtain the training data in the first training data set. For ease of understanding, the following is an example of obtaining any training data in the first training data set, and the first training data (including the first query text, the first search text, and the second search text) is used to refer to any training data in the first training data set. The terminal device obtains the first query text, the fourth search text that has been clicked corresponding to the first query text, and the fifth search text that has not been clicked corresponding to the first query text from a database, and determines the correlation between the first query text and the fourth search text, and the correlation between the first query text and the fifth search text. Further, the terminal device determines the first search text from the fourth search text, the first search text is a search text whose correlation with the first query text is greater than or equal to the first threshold, and determines the second search text from the fifth search text, the second search text is a search text whose correlation with the first query text is less than or equal to the second threshold, and whose correlation with the first query text is greater than or equal to the third threshold.

[0079] Among them, the specific values ​​of the first threshold, the second threshold and the third threshold can be set according to the specific application scenario, and this application does not make specific restrictions on this. It can be understood that when the correlation between the fourth search text and the query text is less than the first threshold, the fourth search text is a search text that should have a negative label but is mistakenly clicked by the user; when the correlation between the fifth search text and the query text is greater than the second threshold, the fifth search text may be a search text that should have a positive label but is not clicked by the user; when the correlation between the fifth search text and the query text is less than the third threshold, the fifth search text may be a search text that is irrelevant to the query text.

[0080] For example, see Figure 4 As shown in FIG. 2 , the terminal device obtains the query text q1 and multiple search texts d1 to d6, wherein search text d1 and search text d5 are search texts that have been clicked by the user, and search text d2, search text d3, search text d4 and search text d5 are search texts that have not been clicked by the user. The terminal device calculates the relevance between the query text q1 and each search text, and the calculation results are shown in Table 2.

[0081] Table 2

[0082]

[0083] Since the clicked search text may be mistakenly clicked, in order to remove the mistakenly clicked search text to improve the quality of the first training data set. The terminal device obtains the first threshold, and compares the correlation between each clicked search text (recorded as the fourth search text) and the query text q1 with the first threshold, and deletes the fourth search text whose correlation is less than the first threshold. For example, the score in Table 2 11 is greater than or equal to the first threshold, score 15 is less than the first threshold, then Figure 4 The terminal device shown labels the search text d1 with a positive label and deletes the search text d5. In order to improve the ability of the search model to distinguish between search texts with positive labels and search texts with negative labels, the terminal device selects the search text that can be labeled with a negative label from the multiple search texts that have not been clicked (recorded as the fifth search text). The terminal device obtains the second threshold and the third threshold, first compares the correlation between each search text that has not been clicked and the query text q1 with the third threshold, deletes the fifth search text whose correlation is less than the third threshold, and then compares the correlation between each search text that has not been clicked and the query text q1 with the second threshold, and deletes the fifth search text whose correlation is greater than the second threshold. For example, the score in Table 2 12 Greater than the second threshold, score 13and score 14 are both less than or equal to the second threshold and greater than or equal to the third threshold, score 16 is less than the third threshold, then Figure 4 The terminal device shown deletes the retrieved text d2 and the retrieved text d6, and marks the retrieved text d3 and the retrieved text d4 with negative labels.

[0084] That is to say, after the terminal device obtains a first query text and multiple search texts corresponding to the first query text (including a fourth search text clicked by a user and a fifth search text not clicked by the user) from a database, the terminal device determines the fourth search text whose correlation with the first query text is greater than or equal to a first threshold as the first search text (i.e., labels the fourth search text with a positive label), and determines the fifth search text whose correlation with the first query text is greater than or equal to a third threshold and less than or equal to a second threshold as the second search text (i.e., labels the fifth search text with a negative label).

[0085] In a possible implementation of determining the first threshold, the terminal device obtains a random text database containing multiple random texts, calculates the correlation between the first query text and each random text; further, sorts the random texts in the random text database in descending order of the correlation, and determines the correlation between the kth random text and the first query text after sorting as the first threshold. In a possible implementation of determining the third threshold, the terminal device determines the correlation between the first query text and each fifth search text; and sorts each fifth search text in descending order of the correlation; and determines the correlation between the nth fifth search text and the first query text as the third threshold. Among them, the specific values ​​of k and n can be set according to the specific application scenario, and this application does not specifically limit this.

[0086] In a possible implementation, after the terminal device obtains multiple training data from at least one database, there may be training data with repeated query text or repeated search text in the multiple training data. In this case, the terminal device may perform data deduplication processing on the multiple training data and then refer to Figure 4 The processing procedure shown processes the deduplicated training data to obtain the training data in the first training data set.

[0087] Exemplarily, the terminal device obtains training data 1 to training data 5, and performs precise string matching processing, fuzzy matching processing, etc. on each training data including the query text and the search text in training data 1 to training data 5, and determines that the query text included in training data 1 and training data 2 is the same, and the search text included in training data 3 and training data 4 is the same. Then, the terminal device performs data deduplication processing on training data 1 to training data 5, and the deduplication training data obtained are training data 1, training data 3, and training data 5. Further, referring to Figure 4 The processing flow provided processes the search text included in the deduplicated training data.

[0088] S302: Based on the training data of the first training data set, update the first information retrieval model to obtain a second information retrieval model.

[0089] The terminal device updates the first information retrieval model based on the various training data included in the first training data set to obtain the second information retrieval model. The updating of the first information retrieval model based on the first training data (i.e., a synonym for any training data in the first training data set) is used as an example for explanation. The terminal device inputs the first query text, the first retrieval text, and the second retrieval text included in the first training data into the first information retrieval model, and the first information retrieval model outputs the degree of correlation between the first query text and the first retrieval text (or can be understood as the degree of correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text), and the degree of correlation between the first query text and each second retrieval text (or can be understood as the degree of correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text). Taking the first information retrieval model as a double-tower model as an example, please refer to Figure 5 As shown, the terminal device inputs the first query text, the first search text and the second search text into the first information retrieval model, and the first information retrieval model obtains the semantic vector of the first query text through encoder 1, obtains the semantic vector of the first search text and the semantic vector of the second search text through encoder 2, and calculates the correlation between the semantic vector of the first query text and the semantic vector of the first search text, and the correlation between the semantic vector of the first query text and the semantic vector of the second search text. Furthermore, the terminal device updates the model parameters of the first information retrieval model according to the correlation between the first query text and the first search text, the correlation between the first query text and each second search text, and the loss function of the first information retrieval model.

[0090] In a possible implementation, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text. In a specific example, the loss function of the first information retrieval model satisfies the condition shown in formula (1).

[0091]

[0092] Where P(.) is the loss function of the first information retrieval model, q is the semantic vector of the first query text, and d + is the semantic vector of the first search text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0093] In a possible implementation, the first information retrieval model includes an encoder, and before updating the first information retrieval model based on the training data of the first training data set to obtain the second information retrieval model, the encoder obtains the training text of the third training data set; the training text of the third training data set is input into the encoder to obtain the first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; further, the training text of the third training data set and the first semantic vector are input into the decoder to obtain the second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the text is reproduced based on the second semantic vector to obtain the reproduced text; the model parameters of the encoder are updated according to the reproduced text and the training text. By implementing this possible implementation, the ability of the encoder of the first information retrieval model to represent the semantic vector of the text can be optimized.

[0094] For example, see Figure 6As shown, the training text "Science and technology are the first productive forces" is randomly masked to obtain masked text 1 and masked text 2; the masked text 1 is input into the encoder to obtain the semantic vector of the masked text 1 (i.e., the aforementioned first semantic vector); the masked text 2 and the semantic vector of the masked text 1 are input into the decoder to obtain the second semantic vector. The second semantic vector satisfies the condition shown in formula (2).

[0095]

[0096] in, is the semantic vector of the masked text 1 (i.e., the first semantic vector mentioned above), N is the number of words contained in the training text, is the word vector of the i-th word, p i is the position vector of the i-th word, is the second semantic vector.

[0097] Further, the text is reproduced based on the second semantic vector to obtain the reproduced text; and the model parameters of the encoder are updated according to the reproduced text and the training text. It should be understood that Figure 6 In the process of randomly masking the training text to obtain masked text 1 and masked text 2, and then obtaining the first semantic vector according to masked text 1, the process is performed by a masked language model (MLM). The MLM can update the model parameters according to the MLM loss function.

[0098] S303: Based on the training data of the second training data set, update the second information retrieval model to obtain a third information retrieval model.

[0099] The terminal device updates the second information retrieval model based on the training data included in the second training data set to obtain a third information retrieval model. Afterwards, the third information retrieval model can receive a third query text (i.e., a query text input into the third information retrieval model), determine a semantic vector of the third query text, and output an information retrieval result (or an information retrieval text corresponding to the third query text) based on the semantic vector of the third query text.

[0100] The following is an example of updating the second information retrieval model based on the second training data (i.e., a synonym for any training data in the second training data set). The terminal device inputs the second query text and the third search text into the second information retrieval model, and the second information retrieval model outputs the degree of correlation between the second query text and the third search text (or can be understood as the degree of correlation between the semantic vector of the second query text and the semantic vector of the third search text). Take the second information retrieval model as a double tower model as an example, see Figure 5 As shown, the terminal device inputs the second query text and the third search text into the second information retrieval model, and the second information retrieval model obtains the semantic vector of the second query text through encoder 1 and the semantic vector of the third search text through encoder 2, and calculates the correlation between the semantic vector of the second query text and the semantic vector of the third search text. Furthermore, the terminal device updates the model parameters of the second information retrieval model according to the correlation between the semantic vector of the second query text and the semantic vector of the third search text, the annotation of the correlation between the second query text and the third search text, and the loss function of the second information retrieval model.

[0101] In a possible implementation, the loss function of the second information retrieval model is set based on the correlation between the semantic vector of the second query text and the semantic vector of the third retrieval text, and the annotation of the correlation between the second query text and the third retrieval text.

[0102] pass Figure 3 In the embodiment shown, when the amount of high-quality training data (i.e., the training data of the second training data set mentioned in the present application, or the second training data mentioned in the present application) is small, the information retrieval model can be first trained with training data whose quality is slightly lower than that of the high-quality training data (e.g., training data without annotations of the degree of correlation between the query text and the retrieval text, i.e., the training data of the first training data set mentioned in the present application) to increase the scale of the training data; and then the information retrieval model is trained with the high-quality training data to improve the quality of the semantic vector represented by the information retrieval model, thereby facilitating improving the overall performance of the information retrieval model.

[0103] The training device for the information retrieval model based on semantic vectors in this application will be described below.

[0104] In the case of integrated units, see Figure 7 , Figure 7 It is a structural diagram of a training device for a model for information retrieval based on semantic vectors provided in an embodiment of the present application. Figure 7 The training device 700 of the model for information retrieval based on semantic vectors is Figure 2 The training device 220 in the embodiment may be a terminal device, a chip in a terminal device, a server (including but not limited to a cloud server) or a chip in a server, etc., and is not limited here. Figure 7As shown, the training device 700 for the model for information retrieval based on semantic vectors includes a processing unit 701 and a transceiver unit 702. The transceiver unit 702 may be a transceiver or a communication interface, and the processing unit 701 may be one or more processors. The training device for the model for information retrieval based on semantic vectors may be used to implement the functions of the terminal device, chip or server involved in the above method embodiments.

[0105] Exemplarily, the training device for the model for information retrieval based on semantic vectors can be a terminal device. The terminal device can be a network element in a hardware device, a software function running on dedicated hardware, or a virtualized function instantiated on a platform (e.g., a cloud platform). Optionally, the training device for the model for information retrieval based on semantic vectors can also include a storage unit (not shown in the figure) for storing program code and data of the semantic understanding device.

[0106] Exemplarily, when the training device of the model for information retrieval based on semantic vectors is a chip, the transceiver unit 702 can be an interface, a pin or a circuit, etc. The interface can be used to input data to be processed to the processor, and can output the processing results of the processor to the outside. In a specific implementation, the interface can be a general purpose input output (GPIO) interface, which can be connected to multiple peripheral devices (such as a display (LCD), a camera (camera), a radio frequency (RF) module, an antenna, etc.). The interface is connected to the processor through a bus.

[0107] The processing unit 701 may be a processor that can execute computer-executable instructions stored in the storage unit to enable the chip to execute Figure 3 The methods involved.

[0108] Furthermore, the processor may include a controller, an arithmetic unit and a register. Exemplarily, the controller is mainly responsible for instruction decoding and issuing control signals for operations corresponding to the instructions. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, etc., and may also perform address operations and conversions. The register is mainly responsible for storing register operands and intermediate operation results temporarily stored during the execution of instructions. In a specific implementation, the hardware architecture of the processor may be an application specific integrated circuit (ASIC) architecture, a microprocessor without interlocked piped stages architecture (MIPS) architecture, an advanced RISC machines (ARM) architecture, or a network processor (NP) architecture, etc. The processor may be single-core or multi-core.

[0109] The storage unit may be a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0110] It should be noted that the functions corresponding to the processor and the interface can be implemented through hardware design, software design, or a combination of hardware and software, and there is no limitation here.

[0111] Specifically, in one design, the training device for the information retrieval model based on semantic vectors can be used to process the acquired text data, specifically:

[0112] The transceiver unit 702 is used to obtain the training data of the first training data set and the training data of the second training data set; wherein the first training data is any training data in the first training data set, and the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; the processing unit 701 is used to update the first information retrieval model based on the training data of the first training data set to obtain a second information retrieval model; the processing unit 701 is also used to update the second information retrieval model based on the training data of the second training data set to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

[0113] Among them, the beneficial effects achieved by the device provided by the second aspect can refer to the beneficial effects corresponding to the method provided by the first aspect, and will not be described in detail here.

[0114] In combination with the second aspect, in a possible implementation, the transceiver unit 702 is further used to obtain the first query text, the fourth search text corresponding to the first query text that has been clicked, and the fifth search text corresponding to the first query text that has not been clicked; the processing unit 701 is further used to determine the degree of correlation between the first query text and the fourth search text, and the degree of correlation between the first query text and the fifth search text; the processing unit 701 is further used to determine the aforementioned first search text from the fourth search text, the first search text being a search text whose degree of correlation with the first query text is greater than or equal to a first threshold; the processing unit 701 is further used to determine the second search text from the fifth search text, the second search text being a search text whose degree of correlation with the first query text is less than or equal to a second threshold, and whose degree of correlation with the first query text is greater than or equal to a third threshold.

[0115] In combination with the second aspect, in a possible implementation, the first information retrieval model includes an encoder. In this case, the transceiver unit 702 is also used to obtain training text of a third training data set; the processing unit 701 is also used to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; the processing unit 701 is also used to input the training text of the third training data set and the first semantic vector into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the processing unit 701 is also used to reproduce the text based on the second semantic vector to obtain reproduced text; the processing unit 701 is also used to update the model parameters of the encoder based on the reproduced text and the training text.

[0116] In combination with the second aspect, in one possible implementation, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

[0117] In conjunction with the second aspect, in a possible implementation manner, the loss function of the first information retrieval model satisfies the following conditions:

[0118]

[0119] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0120] It should be understood that the above-mentioned training device for the model for information retrieval based on semantic vectors can correspond to the steps of the above-mentioned method embodiment, and the above-mentioned operations or functions of each unit in the training device for the model for information retrieval based on semantic vectors are respectively for implementing the corresponding operations performed by the terminal device in the above-mentioned method embodiment, wherein the corresponding beneficial effects can be referred to the method embodiment, and for the sake of brevity, they will not be repeated here.

[0121] The above describes the training device for the model of information retrieval based on semantic vectors in the embodiment of the present application. The following describes the possible product forms of the training device for the model of information retrieval based on semantic vectors. It should be understood that any device having the above Figure 7 Any product of the function of the training device for information retrieval model based on semantic vectors mentioned above falls within the protection scope of the embodiments of the present application. It should also be understood that the following introduction is only an example and does not limit the product form of the training device for information retrieval model based on semantic vectors of the embodiments of the present application to this.

[0122] As a possible product form, see Figure 8 As shown, Figure 8 It is a schematic diagram of the hardware structure of another training device for a model for information retrieval based on semantic vectors provided in an embodiment of the present application. Figure 8 The training device 800 for information retrieval model based on semantic vectors (the device 800 may be a computer device) includes a memory 801, a processor 802, a communication interface 803 and a bus 804. The memory 801, the processor 802 and the communication interface 803 are connected to each other through the bus 804.

[0123] The memory 801 may be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 801 may store a program. When the program stored in the memory 801 is executed by the processor 802, the processor 802 and the communication interface 803 are used to execute the functions required to be executed by the unit in the segmentation network model training device of the embodiment of the present application, or to execute the training method of the model for information retrieval based on semantic vectors of the method embodiment of the present application.

[0124] The processor 802 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the training method of the model for information retrieval based on semantic vectors of the present application may be completed by the hardware integrated logic circuit or software instructions in the processor 802. The above-mentioned processor 802 may also be a general-purpose processor, a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as a hardware decoding processor for execution, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 802 reads the information in the memory 801, and combines its hardware to complete the functions required to be performed by the units included in the training device of the model for information retrieval based on semantic vectors in the embodiment of the present application, or executes the training method of the model for information retrieval based on semantic vectors in the method embodiment of the present application.

[0125] The communication interface 803 uses a transceiver such as, but not limited to, a transceiver to implement communication between the device 800 and other devices or a communication network. For example, training data (such as the training data of the first training data set and the training data of the second training data set in S301 of the present application) can be obtained through the communication interface 803.

[0126] The bus 804 may include a path for transmitting information between various components of the device 800 (eg, the memory 801 , the processor 802 , and the communication interface 803 ).

[0127] It should be understood that the transceiver unit 702 in the training device 700 for a model for information retrieval based on semantic vectors is equivalent to the communication interface 803 in the training device 800 for a model for information retrieval based on semantic vectors, and the processing unit 701 in the training device 700 for a model for information retrieval based on semantic vectors can be equivalent to the processor 802 in the training device 800 for a model for information retrieval based on semantic vectors.

[0128] It should be understood that the training devices for the models for information retrieval based on semantic vectors in the above-mentioned various product forms have any functions of the terminal device in the above-mentioned method embodiments, can implement the steps in the above-mentioned method embodiments accordingly, and achieve corresponding technical effects. For the sake of brevity, they will not be repeated here.

[0129] It should be noted that although Figure 8 The device 800 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the device 800 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the device 800 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the device 800 may also only include the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 8 All devices shown in .

[0130] It can be understood that the device 800 is equivalent to Figure 2 The training device 220 in the embodiment disclosed herein can be appreciated by a person skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for training a model for information retrieval based on semantic vectors, characterized in that: The method comprises: Acquire training data of a first training data set and training data of a second training data set; The first training data is any training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and a label of the degree of relevance between the second query text and the third search text; Based on the training data of the first training data set, updating the first information retrieval model to obtain a second information retrieval model; Based on the training data of the second training data set, the second information retrieval model is updated to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

2. The method according to claim 1, characterized in that: The step of obtaining training data of the first training data set includes: Acquire the first query text, a fourth search text corresponding to the first query text and clicked, and a fifth search text corresponding to the first query text and not clicked; Determining a correlation between the first query text and the fourth search text, and a correlation between the first query text and the fifth search text; Determine the first search text from the fourth search text, the first search text being a search text having a relevance to the first query text greater than or equal to a first threshold; The second search text is determined from the fifth search text, where the second search text is a search text whose relevance to the first query text is less than or equal to a second threshold and whose relevance to the first query text is greater than or equal to a third threshold.

3. The method according to claim 1 or 2, characterized in that: The first information retrieval model includes an encoder. Before updating the first information retrieval model to obtain a second information retrieval model based on the training data of the first training data set, the method further includes: Obtaining training text of a third training data set; Inputting the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, wherein the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; Inputting the training text of the third training data set and the first semantic vector into a decoder to obtain a second semantic vector, wherein the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; Reproducing the text based on the second semantic vector to obtain a reproduced text; The model parameters of the encoder are updated according to the reproduced text and the training text.

4. The method according to any one of claims 1 to 3, characterized in that: The loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

5. The method according to claim 4, characterized in that: The loss function of the first information retrieval model satisfies the following conditions: Where q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - The semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation degree between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

6. A training device for a model for information retrieval based on semantic vectors, characterized in that: The device comprises: A transceiver unit, used to obtain a first training data set and the training data set; The first training data is any training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and a label of the degree of relevance between the second query text and the third search text; A processing unit, configured to update the first information retrieval model to obtain a second information retrieval model based on the training data of the first training data set; The processing unit is also used to update the second information retrieval model to obtain a third information retrieval model based on the training data of the second training data set; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

7. The device according to claim 6, characterized in that: The transceiver unit is further used to obtain the first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked; The processing unit is further used to determine the correlation between the first query text and the fourth search text, and the correlation between the first query text and the fifth search text; The processing unit is further configured to determine the first search text from the fourth search text, wherein the first search text is a search text having a correlation with the first query text that is greater than or equal to a first threshold; The processing unit is further used to determine the second search text from the fifth search text, where the second search text is a search text whose correlation with the first query text is less than or equal to a second threshold and whose correlation with the first query text is greater than or equal to a third threshold.

8. The device according to claim 6 or 7, characterized in that: The first information retrieval model includes an encoder, The transceiver unit is further used to obtain training text of a third training data set; The processing unit is further configured to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, wherein the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; The processing unit is further used to input the training text of the third training data set and the first semantic vector into a decoder to obtain a second semantic vector, where the second semantic vector is generated based on the first semantic vector, the word vector of the word included in the training text, and the position vector of the word; The processing unit is further used to reproduce the text based on the second semantic vector to obtain a reproduced text; The processing unit is further used to update the model parameters of the encoder according to the reproduced text and the training text.

9. The device according to any one of claims 6 to 8, characterized in that: The loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

10. The device according to claim 9, characterized in that: The loss function of the first information retrieval model satisfies the following conditions: Where q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - The semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation degree between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

11. A computer device, characterized in that: The device includes a processor, a transceiver, and a memory; The processor and the transceiver are used to couple with the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1-5.

12. A chip, characterized in that: It comprises a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 5 is executed.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 5 is executed.

Images