Method and apparatus for training model for semantic vector-based information retrieval

Abstract

The present application discloses a method and apparatus for training a model for semantic vector-based information retrieval in the field of artificial intelligence. The method comprises: obtaining a first training data set and a second training data set, wherein first training data is any training data in the first training data set, the first training data comprises a first query text, a first retrieval text corresponding to the first query text and having a positive label, and a second retrieval text corresponding to the first query text and having a negative label, the second training data is any training data in the second training data set, and the second training data comprises a second query text, a third retrieval text corresponding to the second query text, and a tag indicating the degree of correlation between the second query text and the third retrieval text; updating a first information retrieval model on the basis of the first training data set to obtain a second information retrieval model; and updating the second information retrieval model on the basis of the second training data set to obtain a third information retrieval model. The present application can improve the performance of an information retrieval model.

Description

A training method and device for a model for information retrieval based on semantic vectors

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 8, 2023, with application number 202311159877.0 and application name “A training method and device for a model for information retrieval based on semantic vectors”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of natural language processing, and in particular to a method and device for training a model for information retrieval based on semantic vectors. Background Art

[0003] Information retrieval (IR) technology is the art of finding information that meets user needs from a vast amount of information. Information retrieval models based on semantic vectors are an important branch of IR technology. The dual-tower model (also known as the dual-encoder model) is a common information retrieval model in semantic vector-based retrieval technology and features high retrieval efficiency. The dual-tower model's principles can be summarized as follows: The query and retrieval texts are encoded using different encoders to obtain semantic vectors for the query and retrieval texts, respectively. Furthermore, the similarity between the semantic vectors of the query and retrieval texts is calculated, and based on this similarity, the retrieval texts in the recommendation database are ranked.

[0004] The training process of an information retrieval model typically requires a large amount of high-quality training data to improve its performance. High-quality training data can generally be understood as consisting of a query text, the corresponding search text, and annotations indicating the degree of relevance between the query and search text. Too little high-quality training data can negatively impact the performance of the information retrieval model.

[0005] Summary of the Invention

[0006] The present application provides a method and device for training a model for information retrieval based on semantic vectors, which is conducive to reducing the dependence on a large amount of high-quality training data during the training process, and is also conducive to improving the performance of the information retrieval model in retrieving information.

[0007] It should be understood that the methods provided in the embodiments of the present application can be performed by a training device for a model for information retrieval based on semantic vectors. The training device can be a terminal device, or a component of the terminal device, such as a chip or processor used in the terminal device, or a server, such as a local server or a cloud server, or a component of a server, such as a chip or processor used in the server, without limitation.

[0008] In a first aspect, the present application provides a training method for a model for information retrieval based on semantic vectors, the method comprising: obtaining training data of a first training data set and training data of a second training data set; wherein the first training data is any training data in the first training data set, and the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; further, based on the training data of the first training data set, the first information retrieval model is updated to obtain a second information retrieval model; and then based on the training data of the second training data set, the second information retrieval model is updated to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output information retrieval results based on the semantic vector of the third query text.

[0009] Based on the method provided in the first aspect, when the amount of high-quality training data (i.e., the training data of the second training data set mentioned in this application) is small, the information retrieval model can be first trained with training data whose quality is slightly lower than that of the high-quality training data (i.e., the training data of the first training data set mentioned in this application) to increase the scale of the training data; and then the information retrieval model is trained with high-quality training data to improve the quality of the semantic vectors represented in the information retrieval process, thereby reducing the dependence on a large amount of high-quality training data during the training process, and at the same time, it is beneficial to improve the performance of the information retrieval model in retrieving information.

[0010] In one possible implementation, a first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked are obtained; the degree of correlation between the first query text and the fourth search text, as well as the degree of correlation between the first query text and the fifth search text, are determined; further, the first search text is determined from the fourth search text, where the first search text is a search text whose degree of correlation with the first query text is greater than or equal to a first threshold; and a second search text is determined from the fifth search text, where the second search text is a search text whose degree of correlation with the first query text is less than or equal to a second threshold and whose degree of correlation with the first query text is greater than or equal to a third threshold. By implementing this possible implementation, positive examples (i.e., search texts with positive labels) and negative examples (i.e., search texts with negative labels) are labeled according to the degree of correlation between the query text and each search text, which is conducive to improving the quality of the training data in the obtained first training data set.

[0011] In one possible embodiment, the first information retrieval model includes an encoder. In this case, the training text of the third training data set is obtained; the training text of the third training data set is input into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; the training text of the third training data set and the first semantic vector are input into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text and the position vectors of the words; the text is reproduced based on the second semantic vector to obtain a reproduced text; further, the model parameters of the encoder are updated according to the reproduced text and the training text. By implementing this possible embodiment, compared to the method of representing the semantic vector by the average value of the word vectors of all words included in the text, the present application represents the semantic vector of the text based on the word vectors of each word included in the text and the position vectors of each word. When generating the semantic vector of the text, the context information of each word in the text can be combined, which is conducive to improving the accuracy of the semantic vector of the text represented by the information retrieval model, thereby improving the performance of the information retrieval model.

[0012] In one possible embodiment, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first search text, as well as the correlation between the semantic vector of the first query text and the semantic vector of the second search text. Implementing this possible embodiment helps reduce the vector distance between the query text and positive examples (i.e., search texts with positive labels) and increases the vector distance between the query text and negative examples (i.e., search texts with negative labels), thereby improving the performance of the information retrieval model.

[0013] In one possible implementation, the loss function of the first information retrieval model satisfies the following conditions:

[0014] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first retrieved text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0015] In a second aspect, the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising: a transceiver unit 702, for obtaining training data of a first training data set and training data of a second training data set; wherein the first training data is any training data in the first training data set, and the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; a processing unit, for updating the first information retrieval model based on the training data of the first training data set to obtain a second information retrieval model; the processing unit, further for updating the second information retrieval model based on the training data of the second training data set to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

[0016] Among them, the beneficial effects achieved by the device provided by the second aspect can refer to the beneficial effects corresponding to the method provided by the first aspect, and will not be described in detail here.

[0017] In combination with the second aspect, in a possible implementation, the transceiver unit 702 is further used to obtain a first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked; the processing unit is further used to determine the degree of correlation between the first query text and the fourth search text, and the degree of correlation between the first query text and the fifth search text; the processing unit is further used to determine the aforementioned first search text from the fourth search text, the first search text being a search text whose degree of correlation with the first query text is greater than or equal to a first threshold; the processing unit is further used to determine a second search text from the fifth search text, the second search text being a search text whose degree of correlation with the first query text is less than or equal to a second threshold, and whose degree of correlation with the first query text is greater than or equal to a third threshold.

[0018] In combination with the second aspect, in one possible embodiment, the first information retrieval model includes an encoder. In this case, the transceiver unit 702 is also used to obtain training text of the third training data set; the processing unit is also used to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; the processing unit is also used to input the training text of the third training data set and the first semantic vector into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the processing unit is also used to reproduce the text based on the second semantic vector to obtain reproduced text; the processing unit is also used to update the model parameters of the encoder based on the reproduced text and the training text.

[0019] In combination with the second aspect, in one possible embodiment, the loss function of the first information retrieval model is set based on the degree of correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the degree of correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

[0020] In conjunction with the second aspect, in one possible implementation, the loss function of the first information retrieval model satisfies the following conditions:

[0021] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first retrieved text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d +) is the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0022] In a third aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device including a processor. When the processor calls a computer program in a memory, the method described in the first aspect is executed, thereby also achieving the beneficial effects of the method provided in the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device comprising a processor and a memory, the processor and the memory being coupled; the processor is used to implement the method as described in the first aspect, and thus can also achieve the beneficial effects of the method provided in the first aspect.

[0024] In the fifth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, which includes a processor, a memory and a transceiver, and the processor and the memory are coupled; the transceiver is used to send and receive data, and the processor is used to implement the method described in the first aspect, thereby also achieving the beneficial effects of the method provided in the first aspect.

[0025] In the sixth aspect, an embodiment of the present application provides a training device for a model for information retrieval based on semantic vectors, the device including a processor and an interface, the interface being used to receive or output signals, and the processor being used to implement the method as described in the first aspect, thereby also being able to achieve the beneficial effects of the method provided in the first aspect.

[0026] In the seventh aspect, an embodiment of the present application provides a chip (or a group of chip systems), which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface and executes the method in the first aspect.

[0027] Optionally, as an implementation method, the chip (or chip system) may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored on the memory. When the instructions are executed, the processor is used to execute the method in the first aspect.

[0028] In the eighth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the training method of a model for information retrieval based on semantic vectors provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect, and can also achieve the beneficial effects of the method provided by the first aspect.

[0029] In the ninth aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on a device, the device executes the method provided in the first aspect above, thereby achieving the beneficial effects of the method provided in the first aspect.

[0030] In a tenth aspect, a computer device is provided, which includes a training device for a model for information retrieval based on semantic vectors in any one of the second to sixth aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1A is a schematic diagram of an application scenario of information retrieval provided by this application;

[0032] FIG1B is a schematic diagram of another application scenario of information retrieval provided by the present application;

[0033] FIG1C is a schematic diagram of another application scenario of information retrieval provided by the present application;

[0034] FIG2 is a schematic diagram of a system architecture provided by the present application;

[0035] FIG3 is a flow chart of a method for training a model for information retrieval based on semantic vectors provided in this application;

[0036] FIG4 is a schematic diagram of an operation for obtaining training data of a first training data set provided by the present application;

[0037] FIG5 is a flow chart of an update information retrieval model provided by the present application;

[0038] FIG6 is a schematic diagram of a flow chart of updating model parameters of an encoder provided by the present application;

[0039] FIG7 is a schematic structural diagram of a training device for a model for information retrieval based on semantic vectors provided by the present application;

[0040] FIG8 is a schematic structural diagram of another training device for a model for information retrieval based on semantic vectors provided in the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0042] The training method for the model for information retrieval based on semantic vectors provided in the embodiments of the present application can be applied in intelligent question-answering scenarios, natural language processing scenarios, and scenarios where information retrieval (or analysis) is required, or in human-computer interaction, information retrieval, information recommendation, information matching, and scenarios where information management is required. Specifically, the training method for the model for information retrieval based on semantic vectors provided in the embodiments of the present application can be applied in information retrieval scenarios, which are briefly introduced below.

[0043] As the name implies, information retrieval involves finding information that meets user needs from a wide range of available information. For example, a user enters a query into a user device. A data processing device analyzes the query and retrieves matching text from a database connected to the data processing device. The matching text is then sent to the user device.

[0044] To facilitate understanding of the information retrieval scenarios mentioned in this application, this application provides several schematic diagrams of information retrieval application scenarios below.

[0045] Please refer to Figure 1A, which is a schematic diagram of an application scenario of information retrieval provided by this application. The information retrieval system includes user devices and data processing devices. Among them, user devices include smart terminals such as mobile phones, personal computers or information processing centers. The user device is the initiator of information retrieval, and as the initiator of language question and answer, retrieval or query requests, the user usually initiates the request through the user device. The data processing device can be a device or server with data processing functions such as a cloud server, a network server, an application server and a management server. The data processing device receives query statements / voice / text and other questions from the smart terminal through an interactive interface, and then performs language data processing such as machine learning, deep learning, search, reasoning, and decision-making through a memory that stores data and a processor link for data processing. The memory can be a general term, including local storage and a database that stores historical data. The database can be on the data processing device or on other network servers.

[0046] Please refer to FIG1B , which is a schematic diagram of another information retrieval application scenario provided by this application. The information retrieval system includes an intelligent terminal (or user device). The intelligent terminal can directly serve as a data processing device, that is, the intelligent terminal can directly receive input from the user, or the intelligent terminal itself can directly process language data through its own hardware. The specific process is similar to FIG1A , and the above description can be referred to, and no further details will be given here.

[0047] Please refer to Figure 1C, which is a schematic diagram of another application scenario of information retrieval provided by this application. The information retrieval system includes a local device (such as the local device 301 and the local device 302 in Figure 1C), an execution device 210, and a data storage system 250. It can be understood that the data storage system 250 can be integrated on the execution device 210, or it can be set on the cloud or other network servers. The local device in Figure 1C can be understood as the user device in Figure 1A, and the execution device 210 can be understood as the data processing device in Figure 1A. The specific information retrieval process is similar to that in Figure 1A. Please refer to the description above and will not be repeated here.

[0048] It is understandable that the accuracy (or performance) of the information retrieval model in the information retrieval scenario will directly affect whether the output retrieval text meets the user's needs. The training method of the information retrieval model based on semantic vectors provided in this application can improve the performance of the information retrieval model. In order to better understand the solution provided by this application, the system architecture of this application is first introduced below.

[0049] Please refer to Figure 2, which is a system architecture 200 provided by the present application. As shown in the system architecture 200, the data acquisition device 260 is used to collect training data. The training data in the embodiment of the present application includes: question and answering (QA) data (i.e., question text and answer text), query text and the corresponding search text that has been clicked by the user (which can be understood as the search text adopted, accepted or viewed by the user) and the search text that has not been clicked by the user (which can be understood as the search text that has not been adopted, accepted or viewed by the user), etc.; and the training data is stored in the database 230; the training device 220 obtains the target model / rule 201 based on the training data maintained in the database 230. The target model / rule 201 can be used to implement the information retrieval method provided by the embodiment of the present application, that is, the user inputs the query requirement (such as the query text) into the target model / rule 201, and the search text that meets the query requirement (or can be understood as the recommended text) can be obtained. The target model / rule 201 in the embodiment of the present application can specifically be an information retrieval model (or information retrieval network). In the embodiment provided in the present application, the information retrieval model can be obtained by training one or more combinations or deformations of neural networks (NN), convolutional neural networks (CNN), recurrent neural networks (RNN) or generative adversarial networks (GAN).

[0050] It should be noted that, in actual applications, the training data maintained in the database 230 may not all be collected by the data acquisition device 260, but may also be received from other devices. It should also be noted that the training device 220 may not train the target model / rule 201 entirely based on the training data maintained by the database 230, but may also obtain training data from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0051] FIG2 shows a functional module diagram of the data processing process. When corresponding to the actual application scenario diagrams in FIG1A to FIG1C , the client device 240 can be the user device in FIG1A to FIG1C . When the data processing capabilities of the user device in FIG1A to FIG1C are relatively strong, the execution device 210 and the data storage system 250 can be integrated into the user device. In some embodiments, the execution device 210 and the data storage system 250 can also be integrated into the data processing device in FIG1A to FIG1C . The database 230, training device 220, and data acquisition device 260 can be integrated into the data processing device in FIG1A to FIG1C , or can be set up on the cloud or other servers on the network.

[0052] In one possible implementation, the data acquisition device 260 in FIG2 may be a user device, or an input / output interface of the aforementioned server or cloud, an interactive layer (interface) for obtaining query text (or query statements) and returning retrieved text (or reply statements).

[0053] The target model / rule 201 trained by the training device 220 can be applied to different systems or devices, such as the execution device 210 shown in FIG2 . In FIG2 , the execution device 210 is configured with an I / O interface 212 for data interaction with external devices. A user can input data (such as query text in this application) into the I / O interface 212 through a client device 240.

[0054] The association function module 213 is used to pre-process the input data (such as query text) received by the I / O interface 212. In an embodiment of the present application, the association function module 213 includes but is not limited to one or more of the following processing functions: text segmentation, basic error correction, blank removal, case unification, punctuation removal, stop word removal, and special character retention for the query text.

[0055] When the execution device 210 pre-processes the input data through the associated function module 213, or when the calculation module 211 of the execution device 210 performs calculations and other related processing, the execution device 210 can call the data, code, etc. in the data storage system 250 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 250.

[0056] Finally, the I / O interface 212 returns the processing result, such as the retrieved text obtained above, to the client device 240 for providing to the user.

[0057] It is worth noting that the training device 220 can generate corresponding target models / rules 201 based on different data for different goals to provide users with better results.

[0058] In the scenario shown in FIG. 2 , a user can manually specify data input into execution device 210, for example, by operating within the interface provided by I / O interface 212. Alternatively, client device 240 can automatically input data into I / O interface 212 and obtain results. If automatic data input by client device 240 requires user authorization, the user can set the corresponding permissions within client device 240. The user can view the results output by execution device 210 on client device 240, which can be presented via a display, sound, action, or other specific means. Client device 240 can also serve as a data collection terminal, storing the collected query text and search text in database 230.

[0059] It is worth noting that Figure 2 is only a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 2, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0060] To facilitate understanding, the relevant terms / concepts involved in the embodiments of this application are introduced below.

[0061] 1. Word vectors and semantic vectors

[0062] Word embedding technology is a general term for a class of language modeling and feature learning technologies in natural language processing (NLP). Through word embedding technology, words / phrases from a vocabulary can be mapped to a vector space of real numbers to obtain a vector representation of the word / phrase, that is, the word vector of the word / phrase. It should be noted that the word embedding technology mentioned in this application includes but is not limited to the word vector (Word2Vec) algorithm or the global word vector (GloVe) algorithm.

[0063] The semantic vector mentioned in this application can be understood as a vector of a text (or referred to as text) composed of multiple words / phrases.

[0064] 2. Loss Function

[0065] During the training process, because we want the output of the information retrieval model to be as close as possible to the value we actually want to predict, we can compare the output value corresponding to the current information retrieval model with the target value we actually want, and then update the parameters of the information retrieval model based on the difference between the two. For example, if the output value corresponding to the information retrieval model is too high, we adjust the weight vector to make it lower, and continue to adjust until the output value corresponding to the information retrieval model is equal to the target value we actually want, or the output value corresponding to the information retrieval model is very close to the target value we actually want. Therefore, it is necessary to predefine "how to compare the difference between the output value and the target value." This is the loss function or objective function, which are important equations used to measure the difference between the output value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Then, training the information retrieval model becomes a process of minimizing this loss as much as possible.

[0066] 3. Dual-Encoder

[0067] Also known as the dual-encoder neural network model, it refers to the independent encoding of the query text and the retrieval text by different encoders, and the output of the encoder is the semantic vector of the query text and the semantic vector of the retrieval text. Furthermore, the correlation between the semantic vector of the query text and the semantic vector of the retrieval text is calculated, and each retrieval text is sorted based on the correlation between each retrieval text and the query text. The correlation can be understood as the similarity between the two semantic vectors, which can be calculated by methods such as cosine similarity calculation method or vector dot product; or, the correlation can be understood as the vector distance between the two semantic vectors, which can be any one of the Euclidean distance, the vector distance calculated by cosine similarity, or the vector distance calculated by dot matrix.

[0068] During the training process of the information retrieval model, it is usually necessary to train the information retrieval model with a large amount of high-quality training data. If there is too little high-quality training data during the training process, it may affect the performance of the information retrieval model. In order to solve the problem of poor performance of the information retrieval model obtained when there is less high-quality training data, the present application provides a training method for a model for information retrieval based on semantic vectors, which can reduce the dependence of the information retrieval model on a large amount of high-quality training data during the training process, and is conducive to improving the performance of the obtained information retrieval model when there is less high-quality training data.

[0069] The technical solution in the present application will be described in detail below in conjunction with Figure 3. Please refer to Figure 3, which is a flow chart of a method for training a model for information retrieval based on semantic vectors provided in an embodiment of the present application. It can be understood that the method for training a model for information retrieval based on semantic vectors provided in the present application can be executed by a terminal device, or a chip in a terminal device or a chip in a server, etc., and is not limited here. For the convenience of description, the following embodiments of the present application are explained using a terminal device as an example. As shown in Figure 3, the above-mentioned method for training a model for information retrieval based on semantic vectors may include the following steps:

[0070] S301: Obtain training data of a first training data set and training data of a second training data set.

[0071] The first training data is any piece of training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text. The second training data is any piece of training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and an annotation of the degree of relevance (also known as relevance, correlation, or similarity, etc.) between the second query text and the third search text.

[0072] The manner in which the terminal device obtains the training data of the training data set (including the first training data set or the second training data set) includes but is not limited to one or more of the following manners: manner 1, the terminal device obtains the training data of the training data set from its own memory; manner 2, the terminal device obtains the training data of the training data set from a database to which it has access rights; manner 3, the terminal device collects training data from a data acquisition device with which it has a communication connection (the function of the data acquisition device can be found in the introduction of the data acquisition device 260 in Figure 2) to form a training data set.

[0073] It should be noted that, in the present application, the manner in which the terminal device obtains the training data of the first training data set and the manner in which the terminal device obtains the training data of the second training data set may be the same or different, and this application does not impose any specific restrictions on this. The acquisition source of the first training data set and the acquisition source of the second training data set may be the same or different, and this application does not impose any specific restrictions on this. The degree of correlation between the two texts mentioned in the present application can be understood as the vector distance between the semantic vectors corresponding to the two texts. The closer the vector distance is, the higher the degree of correlation can be understood, and the farther the vector distance is, the lower the degree of correlation can be understood. Among them, the vector distance can be any one of the Euclidean distance, the vector distance calculated by cosine similarity, or the vector distance calculated by dot matrix. Alternatively, the degree of correlation between the two texts mentioned in the present application can be understood as the similarity between the semantic vectors corresponding to the two texts, and the similarity can be calculated by methods such as the cosine similarity calculation method or the vector dot product method.

[0074] For example, refer to the database 1 shown in Table 1, in which the first training data set and the second training data set are stored. The first training data set includes M training data, and the second training data set includes N training data, where M and N are both positive integers greater than 1, and M and N may be equal or unequal, which is not limited in this application. Any training data in the first training data set includes a query text q, a retrieval text d with a positive label corresponding to the query text q, and a query text d with a positive label corresponding to the query text q. + and the retrieved text d with negative labels -For example, the training data 01 of the first training data set in Table 1 includes the query text q0, the search text with a positive label corresponding to the query text q0 and i (a positive integer greater than 1) search texts with negative labels Any training data in the second training data set includes the query text q', the retrieval text d' corresponding to the query text q', and the relevance score (or understood as a label) score between the query text q' and the retrieval text d'; for example, the training data 02 of the second training data set in Table 1 includes the query text q'0, the retrieval text d'0 corresponding to the query text q'0, and the relevance score score 0 between the query text q'0 and the retrieval text d'0.

[0075] Table 1

[0076] In this case, if the terminal device has access rights to the database 1 , the terminal device can obtain the training data of the first training data set and the training data of the second training data set from the database 1 .

[0077] In one possible embodiment, the terminal device obtains the first training data set by obtaining multiple training data from at least one data source (or database), and then processing the multiple training data to obtain the training data in the first training data set. For ease of understanding, the following is an example of obtaining any training data in the first training data set. Hereinafter, the first training data (including the first query text, the first search text, and the second search text) will be referred to as any training data in the first training data set. The terminal device obtains the first query text, the fourth search text corresponding to the first query text that has been clicked, and the fifth search text corresponding to the first query text that has not been clicked from a database, and determines the correlation between the first query text and the fourth search text, and the correlation between the first query text and the fifth search text. Furthermore, the terminal device determines the first search text from the fourth search text, the first search text being a search text whose correlation with the first query text is greater than or equal to a first threshold, and determines the second search text from the fifth search text, the second search text being a search text whose correlation with the first query text is less than or equal to a second threshold, and whose correlation with the first query text is greater than or equal to a third threshold.

[0078] The specific values ​​of the first threshold, the second threshold, and the third threshold can be set according to the specific application scenario, and this application does not impose specific restrictions on this. It can be understood that when the correlation between the fourth search text and the query text is less than the first threshold, the fourth search text is a search text that should have a negative label but was mistakenly clicked by the user; when the correlation between the fifth search text and the query text is greater than the second threshold, the fifth search text may be a search text that should have a positive label but was not clicked by the user; when the correlation between the fifth search text and the query text is less than the third threshold, the fifth search text may be a search text that is irrelevant to the query text.

[0079] For example, as shown in Figure 4, a terminal device obtains query text q1 and multiple search texts d1 to d6. Search text d1 and search text d5 are search texts that the user has clicked on, while search texts d2, d3, d4, and d5 are search texts that the user has not clicked on. The terminal device calculates the correlation between query text q1 and each search text, and the results are shown in Table 2.

[0080] Table 2

[0081] Since the clicked search texts may be mistakenly clicked, in order to remove the mistakenly clicked search texts and improve the quality of the first training data set, the terminal device obtains the first threshold, and compares the correlation between each clicked search text (recorded as the fourth search text) and the query text q1 with the first threshold, and deletes the fourth search text whose correlation is less than the first threshold. For example, the score in Table 2 11 is greater than or equal to the first threshold, score 15 If it is less than the first threshold, the terminal device will label the search text d1 with a positive label and delete the search text d5 as shown in Figure 4. In order to improve the ability of the retrieval model to distinguish between search texts with positive labels and search texts with negative labels, the terminal device selects the search text that can be labeled with a negative label from the multiple search texts that have not been clicked (recorded as the fifth search text). The terminal device obtains the second threshold and the third threshold, first compares the correlation between each search text that has not been clicked and the query text q1 with the third threshold, deletes the fifth search text with a correlation less than the third threshold, and then compares the correlation between each search text that has not been clicked and the query text q1 with the second threshold, and deletes the fifth search text with a correlation greater than the second threshold. For example, the score in Table 2 12 Greater than the second threshold, score 13 and score 14 are all less than or equal to the second threshold and greater than or equal to the third threshold, score16 If the value is less than the third threshold, the terminal device deletes the search text d2 and the search text d6 and marks the search text d3 and the search text d4 with negative labels, as shown in FIG4 .

[0082] That is to say, after the terminal device obtains the first query text and multiple search texts corresponding to the first query text (including the fourth search text clicked by the user and the fifth search text not clicked by the user) from a database, the terminal device will determine the fourth search text whose correlation with the first query text is greater than or equal to the first threshold as the first search text (that is, the fourth search text is labeled with a positive label), and determine the fifth search text whose correlation with the first query text is greater than or equal to the third threshold and less than or equal to the second threshold as the second search text (that is, the fifth search text is labeled with a negative label).

[0083] In one possible implementation of determining the first threshold, the terminal device obtains a random text database containing multiple random texts, calculates the degree of correlation between the first query text and each random text; further, sorts the random texts in the random text database in descending order of the degree of correlation, and determines the degree of correlation between the kth random text and the first query text as the first threshold. In one possible implementation of determining the third threshold, the terminal device determines the degree of correlation between the first query text and each fifth search text; and sorts the fifth search texts in descending order of the degree of correlation; and determines the degree of correlation between the nth fifth search text and the first query text as the third threshold. Specifically, the values ​​of k and n can be set according to the specific application scenario, and are not specifically limited in this application.

[0084] In one possible implementation, after a terminal device obtains multiple training data from at least one database, some of the multiple training data may contain duplicate query text or duplicate search text. In this case, the terminal device may perform data deduplication on the multiple training data and then process the deduplicated training data according to the process shown in FIG. 4 to obtain the training data in the first training data set.

[0085] Exemplarily, a terminal device obtains training data 1 through 5 and, by performing precise string matching, fuzzy matching, or other methods on the query text and search text in each of the training data 1 through 5, determines that the query text included in training data 1 and training data 2 is identical, and the search text included in training data 3 and training data 4 is identical. The terminal device then performs data deduplication on training data 1 through 5, obtaining the deduplicated training data as training data 1, training data 3, and training data 5. Furthermore, referring to the processing flow shown in FIG4 , the search text included in the deduplicated training data is processed.

[0086] S302: Based on the training data of the first training data set, update the first information retrieval model to obtain a second information retrieval model.

[0087] The terminal device updates the first information retrieval model based on the various training data included in the first training data set to obtain the second information retrieval model. The updating of the first information retrieval model based on the first training data (i.e., a synonym for any training data in the first training data set) is used as an example for explanation. The terminal device inputs the first query text, the first retrieval text, and the second retrieval text included in the first training data into the first information retrieval model, and the first information retrieval model outputs the degree of correlation between the first query text and the first retrieval text (or can be understood as the degree of correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text), and the degree of correlation between the first query text and each second retrieval text (or can be understood as the degree of correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text). Taking the first information retrieval model as a double-tower model as an example, as shown in FIG5 , the terminal device inputs the first query text, the first retrieval text, and the second retrieval text into the first information retrieval model. The first information retrieval model obtains the semantic vector of the first query text through encoder 1, and obtains the semantic vector of the first retrieval text and the semantic vector of the second retrieval text through encoder 2, and calculates the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text. Furthermore, the terminal device updates the model parameters of the first information retrieval model based on the correlation between the first query text and the first retrieval text, the correlation between the first query text and each second retrieval text, and the loss function of the first information retrieval model.

[0088] In one possible implementation, the loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first search text, and the correlation between the semantic vector of the first query text and the semantic vector of the second search text. In a specific example, the loss function of the first information retrieval model satisfies the condition shown in formula (1).

[0089] Where P(.) is the loss function of the first information retrieval model, q is the semantic vector of the first query text, and d + is the semantic vector of the first retrieved text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0090] In one possible implementation, the first information retrieval model includes an encoder. Before updating the first information retrieval model based on the training data of the first training data set to obtain the second information retrieval model, the encoder obtains training text of a third training data set; the training text of the third training data set is input into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; further, the training text of the third training data set and the first semantic vector are input into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the text is reproduced based on the second semantic vector to obtain reproduced text; and the model parameters of the encoder are updated based on the reproduced text and the training text. By implementing this possible implementation, the ability of the encoder of the first information retrieval model to represent the semantic vectors of text can be optimized.

[0091] For example, as shown in FIG6 , the training text “Science and technology are the primary productive forces” is randomly masked to obtain masked text 1 and masked text 2. Masked text 1 is input into the encoder to obtain the semantic vector of masked text 1 (i.e., the aforementioned first semantic vector). Masked text 2 and the semantic vector of masked text 1 are input into the decoder to obtain a second semantic vector. The second semantic vector satisfies the conditions shown in formula (2).

[0092] in, is the semantic vector of the masked text 1 (i.e., the first semantic vector mentioned above), N is the number of words contained in the training text, is the word vector of the i-th word, p i is the position vector of the i-th word, is the second semantic vector.

[0093] Furthermore, the text is reproduced based on the second semantic vector to obtain the reproduced text; and the encoder model parameters are updated based on the reproduced text and the training text. It should be understood that the process of randomly masking the training text in Figure 6 to obtain masked text 1 and masked text 2, and then obtaining the first semantic vector based on masked text 1, is performed using a masked language model (MLM). This MLM can update model parameters based on the MLM loss function.

[0094] S303: Based on the training data of the second training data set, update the second information retrieval model to obtain a third information retrieval model.

[0095] The terminal device updates the second information retrieval model based on the training data included in the second training data set to obtain a third information retrieval model. The third information retrieval model can then receive a third query text (i.e., a query text input into the third information retrieval model), determine a semantic vector for the third query text, and output an information retrieval result (or, an information retrieval text corresponding to the third query text) based on the semantic vector for the third query text.

[0096] The following is an example of updating the second information retrieval model based on the second training data (i.e., a synonym for any training data in the second training data set). The terminal device inputs the second query text and the third retrieval text into the second information retrieval model, and the second information retrieval model outputs the degree of correlation between the second query text and the third retrieval text (or can be understood as the degree of correlation between the semantic vector of the second query text and the semantic vector of the third retrieval text). Taking the second information retrieval model as a double-tower model as an example, as shown in Figure 5, the terminal device inputs the second query text and the third retrieval text into the second information retrieval model, and the second information retrieval model obtains the semantic vector of the second query text through encoder 1 and the semantic vector of the third retrieval text through encoder 2, and calculates the degree of correlation between the semantic vector of the second query text and the semantic vector of the third retrieval text. Furthermore, the terminal device updates the model parameters of the second information retrieval model based on the degree of correlation between the semantic vector of the second query text and the semantic vector of the third retrieval text, the annotation of the degree of correlation between the second query text and the third retrieval text, and the loss function of the second information retrieval model.

[0097] In one possible implementation, the loss function of the second information retrieval model is set based on the correlation between the semantic vector of the second query text and the semantic vector of the third retrieval text, and the annotation of the correlation between the second query text and the third retrieval text.

[0098] Through the implementation shown in Figure 3, when the amount of high-quality training data (i.e., the training data of the second training data set mentioned in this application, or the second training data mentioned in this application) is small, the information retrieval model can be first trained with training data of slightly lower quality than the high-quality training data (for example, training data without annotations of the degree of correlation between the query text and the retrieval text, i.e., the training data of the first training data set mentioned in this application) to increase the scale of the training data; and then the information retrieval model is trained with the high-quality training data to improve the quality of the semantic vector represented by the information retrieval model, which is conducive to improving the overall performance of the information retrieval model.

[0099] The following describes the training device for the information retrieval model based on semantic vectors in this application.

[0100] In the case of adopting an integrated unit, refer to Figure 7, which is a structural diagram of a training device for a model for information retrieval based on semantic vectors provided in an embodiment of the present application. The training device 700 for the model for information retrieval based on semantic vectors in Figure 7 is the training device 220 in Figure 2, which can be a terminal device, a chip in a terminal device, a server (including but not limited to a cloud server) or a chip in a server, etc., and is not limited here. As shown in Figure 7, the training device 700 for the model for information retrieval based on semantic vectors includes a processing unit 701 and a transceiver unit 702. Among them, the transceiver unit 702 can be a transceiver or a communication interface, and the processing unit 701 can be one or more processors. The training device for the model for information retrieval based on semantic vectors can be used to implement the functions of the terminal device, chip or server involved in the above-mentioned method embodiments.

[0101] Exemplarily, the training device for the model for information retrieval based on semantic vectors can be a terminal device. The terminal device can be a network element in a hardware device, a software function running on dedicated hardware, or a virtualized function instantiated on a platform (e.g., a cloud platform). Optionally, the training device for the model for information retrieval based on semantic vectors can also include a storage unit (not shown in the figure) for storing program code and data of the semantic understanding device.

[0102] Exemplarily, when the training device for the model for information retrieval based on semantic vectors is a chip, the transceiver unit 702 can be an interface, a pin, or a circuit. The interface can be used to input data to be processed into the processor and can output the processing results of the processor. In a specific implementation, the interface can be a general purpose input and output (GPIO) interface that can be connected to multiple peripheral devices (such as a display (LCD), a camera, a radio frequency (RF) module, an antenna, etc.). The interface is connected to the processor via a bus.

[0103] The processing unit 701 may be a processor that may execute computer-executable instructions stored in the storage unit to enable the chip to execute the method involved in FIG. 3 .

[0104] Furthermore, the processor may include a controller, an arithmetic unit and a register. For example, the controller is mainly responsible for decoding instructions and issuing control signals for the operations corresponding to the instructions. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations and logical operations, etc., and can also perform address operations and conversions. The register is mainly responsible for storing register operands and intermediate operation results temporarily stored during the execution of instructions. In a specific implementation, the hardware architecture of the processor can be an application-specific integrated circuit (ASIC) architecture, a microprocessor without interlocked piped stages architecture (MIPS) architecture, an advanced RISC machines (ARM) architecture or a network processor (NP) architecture, etc. The processor can be single-core or multi-core.

[0105] The storage unit may be a storage unit within the chip, such as a register, cache, etc. The storage unit may also be a storage unit located outside the chip, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0106] It should be noted that the functions corresponding to the processor and the interface can be implemented through hardware design, software design, or a combination of hardware and software, and there is no limitation here.

[0107] Specifically, in one design, the training device for the semantic vector-based information retrieval model can be used to process the acquired text data, specifically:

[0108] The transceiver unit 702 is used to obtain training data of the first training data set and training data of the second training data set; wherein the first training data is any training data in the first training data set, and the first training data consists of a first query text, a first retrieval text with a positive label corresponding to the first query text, and a second retrieval text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third retrieval text corresponding to the second query text, and an annotation of the degree of correlation between the second query text and the third retrieval text; the processing unit 701 is used to update the first information retrieval model based on the training data of the first training data set to obtain a second information retrieval model; the processing unit 701 is also used to update the second information retrieval model based on the training data of the second training data set to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

[0109] Among them, the beneficial effects achieved by the device provided by the second aspect can refer to the beneficial effects corresponding to the method provided by the first aspect, and will not be described in detail here.

[0110] In combination with the second aspect, in a possible implementation, the transceiver unit 702 is further used to obtain the first query text, the fourth search text corresponding to the first query text that has been clicked, and the fifth search text corresponding to the first query text that has not been clicked; the processing unit 701 is further used to determine the degree of correlation between the first query text and the fourth search text, and the degree of correlation between the first query text and the fifth search text; the processing unit 701 is further used to determine the aforementioned first search text from the fourth search text, the first search text being a search text whose degree of correlation with the first query text is greater than or equal to a first threshold; the processing unit 701 is further used to determine the second search text from the fifth search text, the second search text being a search text whose degree of correlation with the first query text is less than or equal to a second threshold, and whose degree of correlation with the first query text is greater than or equal to a third threshold.

[0111] In combination with the second aspect, in one possible embodiment, the first information retrieval model includes an encoder. In this case, the transceiver unit 702 is also used to obtain training text of the third training data set; the processing unit 701 is also used to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, and the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; the processing unit 701 is also used to input the training text of the third training data set and the first semantic vector into the decoder to obtain a second semantic vector, and the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; the processing unit 701 is also used to reproduce the text based on the second semantic vector to obtain reproduced text; the processing unit 701 is also used to update the model parameters of the encoder based on the reproduced text and the training text.

[0112] In combination with the second aspect, in one possible embodiment, the loss function of the first information retrieval model is set based on the degree of correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the degree of correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

[0113] In conjunction with the second aspect, in one possible implementation, the loss function of the first information retrieval model satisfies the following conditions:

[0114] Among them, q is the semantic vector of the first query text, d + is the semantic vector of the first retrieved text, d - is the semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, f(q,d - ) is the correlation between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

[0115] It should be understood that the above-mentioned training device for the model for information retrieval based on semantic vectors can correspond to the steps of the above-mentioned method embodiment, and the above-mentioned operations or functions of each unit in the training device for the model for information retrieval based on semantic vectors are respectively for realizing the corresponding operations performed by the terminal device in the above-mentioned method embodiment, wherein the corresponding beneficial effects can be referred to the method embodiment, and for the sake of brevity, they will not be repeated here.

[0116] The above describes the training device for a model for information retrieval based on semantic vectors according to an embodiment of the present application. The following describes possible product forms of the training device for a model for information retrieval based on semantic vectors. It should be understood that any product having the functions of the training device for a model for information retrieval based on semantic vectors as described above in FIG. 7 falls within the scope of protection of the embodiment of the present application. It should also be understood that the following description is for illustrative purposes only and does not limit the product form of the training device for a model for information retrieval based on semantic vectors according to the embodiment of the present application to this.

[0117] As a possible product form, please refer to Figure 8, which is a schematic diagram of the hardware structure of another training device for a model for information retrieval based on semantic vectors provided in an embodiment of the present application. The training device 800 for a model for information retrieval based on semantic vectors shown in Figure 8 (the device 800 can specifically be a computer device) includes a memory 801, a processor 802, a communication interface 803, and a bus 804. Among them, the memory 801, the processor 802, and the communication interface 803 are connected to each other through the bus 804.

[0118] The memory 801 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 801 can store programs. When the program stored in the memory 801 is executed by the processor 802, the processor 802 and the communication interface 803 are used to perform the functions required to be performed by the units in the segmentation network model training device of the embodiment of the present application, or to perform the training method of the model for information retrieval based on semantic vectors of the method embodiment of the present application.

[0119] The processor 802 may also be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the training method for the semantic vector-based information retrieval model of the present application may be completed by hardware integrated logic circuits or software instructions in the processor 802. The aforementioned processor 802 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 801, and the processor 802 reads the information in the memory 801, and combines its hardware to complete the functions required to be performed by the units included in the training device of the model for information retrieval based on semantic vectors in the embodiment of the present application, or executes the training method of the model for information retrieval based on semantic vectors in the method embodiment of the present application.

[0120] The communication interface 803 uses a transceiver device such as, but not limited to, a transceiver to enable communication between the apparatus 800 and other devices or a communication network. For example, training data (such as the training data of the first training data set and the training data of the second training data set in S301 of the present application) can be obtained through the communication interface 803.

[0121] The bus 804 may include a path for transmitting information between various components of the device 800 (eg, the memory 801 , the processor 802 , and the communication interface 803 ).

[0122] It should be understood that the transceiver unit 702 in the training device 700 for a model for information retrieval based on semantic vectors is equivalent to the communication interface 803 in the training device 800 for a model for information retrieval based on semantic vectors, and the processing unit 701 in the training device 700 for a model for information retrieval based on semantic vectors can be equivalent to the processor 802 in the training device 800 for a model for information retrieval based on semantic vectors.

[0123] It should be understood that the training devices for the models of information retrieval based on semantic vectors in the above-mentioned various product forms have any functions of the terminal equipment in the above-mentioned method embodiments, can implement the steps in the above-mentioned method embodiments accordingly, and achieve corresponding technical effects. For the sake of brevity, they will not be repeated here.

[0124] It should be noted that although the device 800 shown in FIG8 only shows a memory, a processor, and a communication interface, during the specific implementation process, those skilled in the art will understand that the device 800 also includes other components necessary for normal operation. At the same time, according to specific needs, those skilled in the art will understand that the device 800 may also include hardware components that implement other additional functions. In addition, those skilled in the art will understand that the device 800 may also include only the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in FIG8.

[0125] It is understood that the apparatus 800 is equivalent to the training device 220 in FIG2 . Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0130] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for training a model for information retrieval based on semantic vectors, characterized in that: The method comprises: Acquire training data of a first training data set and training data of a second training data set; The first training data is any training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and a label of the degree of relevance between the second query text and the third search text; Based on the training data of the first training data set, updating the first information retrieval model to obtain a second information retrieval model; Based on the training data of the second training data set, the second information retrieval model is updated to obtain a third information retrieval model; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

2. The method according to claim 1, characterized in that: The step of obtaining training data of the first training data set includes: Acquire the first query text, a fourth search text corresponding to the first query text and clicked, and a fifth search text corresponding to the first query text and not clicked; Determining a correlation between the first query text and the fourth search text, and a correlation between the first query text and the fifth search text; Determine the first search text from the fourth search text, the first search text being a search text having a relevance to the first query text greater than or equal to a first threshold; The second search text is determined from the fifth search text, where the second search text is a search text whose relevance to the first query text is less than or equal to a second threshold and whose relevance to the first query text is greater than or equal to a third threshold.

3. The method according to claim 1 or 2, characterized in that: The first information retrieval model includes an encoder, and before updating the first information retrieval model to obtain a second information retrieval model based on the training data of the first training data set, the method further includes: Obtaining training text of a third training data set; Inputting the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, wherein the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; Inputting the training text of the third training data set and the first semantic vector into a decoder to obtain a second semantic vector, wherein the second semantic vector is generated based on the first semantic vector, the word vectors of the words included in the training text, and the position vectors of the words; Reproducing the text based on the second semantic vector to obtain a reproduced text; The model parameters of the encoder are updated according to the reproduced text and the training text.

4. The method according to any one of claims 1 to 3, characterized in that: The loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

5. The method according to claim 4, characterized in that: The loss function of the first information retrieval model satisfies the following conditions: Where q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - The semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation degree between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

6. A training device for a model for information retrieval based on semantic vectors, characterized in that: The device comprises: A transceiver unit, used to obtain a first training data set and the training data set; The first training data is any training data in the first training data set, and the first training data consists of a first query text, a first search text with a positive label corresponding to the first query text, and a second search text with a negative label corresponding to the first query text; the second training data is any training data in the second training data set, and the second training data consists of a second query text, a third search text corresponding to the second query text, and a label of the degree of relevance between the second query text and the third search text; A processing unit, configured to update the first information retrieval model to obtain a second information retrieval model based on the training data of the first training data set; The processing unit is also used to update the second information retrieval model to obtain a third information retrieval model based on the training data of the second training data set; the third information retrieval model is used to receive a third query text, determine the semantic vector of the third query text, and output an information retrieval result based on the semantic vector of the third query text.

7. The device according to claim 6, characterized in that: The transceiver unit is further used to obtain the first query text, a fourth search text corresponding to the first query text that has been clicked, and a fifth search text corresponding to the first query text that has not been clicked; The processing unit is further used to determine the correlation between the first query text and the fourth search text, and the correlation between the first query text and the fifth search text; The processing unit is further configured to determine the first search text from the fourth search text, wherein the first search text is a search text having a correlation with the first query text that is greater than or equal to a first threshold; The processing unit is further used to determine the second search text from the fifth search text, where the second search text is a search text whose correlation with the first query text is less than or equal to a second threshold and whose correlation with the first query text is greater than or equal to a third threshold.

8. The device according to claim 6 or 7, characterized in that: The first information retrieval model includes an encoder, The transceiver unit is further used to obtain training text of a third training data set; The processing unit is further configured to input the training text of the third training data set into the encoder to obtain a first semantic vector corresponding to the training text, wherein the first semantic vector is generated based on the word vectors of the words included in the training text and the position vectors of the words; The processing unit is further used to input the training text of the third training data set and the first semantic vector into a decoder to obtain a second semantic vector, where the second semantic vector is generated based on the first semantic vector, the word vector of the word included in the training text, and the position vector of the word; The processing unit is further used to reproduce the text based on the second semantic vector to obtain a reproduced text; The processing unit is further used to update the model parameters of the encoder according to the reproduced text and the training text.

9. The device according to any one of claims 6 to 8, characterized in that: The loss function of the first information retrieval model is set based on the correlation between the semantic vector of the first query text and the semantic vector of the first retrieval text, and the correlation between the semantic vector of the first query text and the semantic vector of the second retrieval text.

10. The device according to claim 9, characterized in that: The loss function of the first information retrieval model satisfies the following conditions: Where q is the semantic vector of the first query text, d + is the semantic vector of the first search text, d - The semantic vector of the second search text, is the semantic vector set to which the semantic vector of the second search text belongs, f(q,d + ) is the correlation between the semantic vector of the first query text and the semantic vector of the first search text, f(q,d - ) is the correlation degree between the semantic vector of the first query text and the semantic vector of the second search text, and τ is the temperature parameter corresponding to the first information retrieval model.

11. A computer device, characterized in that: The device includes a processor, a transceiver, and a memory; The processor and the transceiver are used to couple with the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1-5.

12. A chip, characterized in that: It comprises a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 5 is executed.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 5 is executed.

Images