Information retrieval method, related system, and storage medium

Abstract

The present application provides an information retrieval method and related system and storage medium, including: S1, obtaining Mi candidate documents of the i+1th hop based on a search text Query and Ki candidate documents of the i-th hop; S2, obtaining a score for each candidate document in the Mi candidate documents of the i+1th hop; S3, obtaining the probability of the path corresponding to the candidate document Pjy(i+1) based on the score of the candidate document Pjy(i+1) and the probability of the path L; S4, obtaining K(i+1) candidate documents of the i+1th hop based on the probabilities of the paths corresponding to the Mi candidate documents of the i+1th hop; and obtaining a retrieval result corresponding to the Query based on the K(i+1) candidate documents of the i+1th hop. This solution can perform sorting from the perspective of global optimality of the global retrieval path, thereby improving the accuracy and efficiency of information retrieval.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an information retrieval method and related systems and storage media. Background Art

[0002] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theory.

[0003] Deep pre-trained language models have become a necessary means to improve tasks such as search, recommendation, and natural language processing. They are pre-trained on massive amounts of corpus, giving them extremely strong text representation capabilities.

[0004] In the search business, semantic search based on deep pre-trained language models is particularly prominent. Using semantic search technology, it can represent queries as semantic vectors and then match them based on similarity between vectors. Semantic vectors can represent implicit and ambiguous semantic information within text.

[0005] Current deep pre-trained language models still face some thorny challenges, such as when users enter complex queries that require multiple rounds of reasoning. For example, a question like "where did Algeria qualify for the first time into the round of 16?" correctly answers this question by searching for a single document through semantics. It must first find the document "Algeria national football team" and understand that the "Algeria" in the question refers to the "Algerian national football team" and that the team "reached the round of 16 in the 2014 World Cup." It then searches for the document "2014 World Cup" and understands from it that the World Cup was held in Brazil. Only after a series of multiple rounds of searches can the correct answer, "Brazil," be found. However, when searching with a search engine improved by current deep pre-trained language models, the keyword "Brazil" is not mentioned in the first three documents, and Brazil appears for the first time in the fourth document. This introduces significant noise and influences the generation of the answer, resulting in poor search results.

[0006] When the search is improved from a single-round search to a multi-round search, there is a greater chance of finding supporting documents with answers. Therefore, there are currently many multi-round search technologies based on deep pre-trained language models. The existing technology provides an information retrieval method. Figure 1 As shown in FIG, it performs multiple rounds of searches from the candidate document library for the search text Query input by the user, performs the next round of searches based on the documents retrieved in the previous round, and finally outputs the document set corresponding to the Query.

[0007] However, the document set corresponding to the query output by the prior art is not a retrieval result that meets the requirements of the query, and the retrieval effect is poor. Summary of the Invention

[0008] The present application discloses an information retrieval method and related system and storage medium, which can improve the accuracy and efficiency of information retrieval.

[0009] In a first aspect, an embodiment of the present application provides an information retrieval method, comprising: S1, obtaining Mi i+1-th hop candidate documents based on a search text Query and Ki i-th hop candidate documents; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi; S2, obtaining a score for each candidate document in the Mi i+1-th hop candidate documents, wherein the score is used to characterize the relevance of the candidate document to the Query; S3, for any candidate document Pjy(i+1) in the Mi i+1-th hop candidate documents, obtaining a probability of a path corresponding to the candidate document Pjy(i+1) based on the score of the candidate document Pjy(i+1) and the probability of a path L, wherein the path L is a path starting from the first hop candidate document and ending at the candidate document Pji in the Ki i-th hop candidate documents, The candidate document Pji is the jth one among the Ki candidate documents of the i-th hop, and the candidate document Pjy(i+1) is the yth one among the candidate documents of the i+1-th hop obtained based on the candidate document Pji of the i-th hop, where j and y are both positive integers; S4, according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1-th hop, obtain K(i+1) candidate documents of the i+1-th hop, wherein the K(i+1) candidate documents of the i+1-th hop are candidate documents among the Mi candidate documents of the i+1-th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1-th hop are higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi; S5, according to the K(i+1) candidate documents of the i+1-th hop, obtain the retrieval result corresponding to the Query.

[0010] It should be noted that steps S1-S4 are only used to indicate a part of the information retrieval method, and the execution order of S1-S4 is not limited to the order described in S1-S4. For example, the probability of calculating the path in step S3 can be performed in parallel with step S2.

[0011] Among them, in step S1, Mi candidate documents of the i+1th hop are obtained according to the search text Query and Ki candidate documents of the i-th hop. It can be understood that each of the search text Query and Ki candidate documents of the i-th hop is processed separately, and a total of Mi candidate documents of the i+1th hop are obtained.

[0012] The probability of a path in step S3 refers to the probability of that path being selected. A path begins with the candidate document of the first hop and ends at any candidate document of the i-th hop. It indicates that the candidate document for the current hop is derived based on the candidate document corresponding to the aforementioned path. The path can be used to interpret the candidate document for the current hop, making it easier to understand the process by which it was derived.

[0013] For example, let's take a path starting from the candidate document at the first hop and ending at any candidate document at the fourth hop as an example. This path includes four hops and, accordingly, four candidate documents. The candidate document at the first hop can be used to obtain the candidate document at the second hop corresponding to the path. The candidate document at the second hop can be used to obtain the candidate document at the third hop corresponding to the path. The candidate document at the third hop can be used to obtain the candidate document at the fourth hop corresponding to the path. Each path is unique.

[0014] Furthermore, any candidate document Pjy(i+1) in step S3 is used to indicate a random selection. The candidate document is obtained based on the candidate document of the previous jump. The j and y are only used for ease of description and do not have a specific limiting effect.

[0015] The above Ki and K(i+1) can be the same or different, and this solution does not make any specific limitation on this.

[0016] In this embodiment of the present application, the probability of the path corresponding to the candidate document in the current hop is obtained by taking the score of each candidate document from the Mi candidate documents obtained in the current hop and the probability of the path corresponding to the candidate document in the previous hop. The next hop candidate document is then obtained based on the multiple candidate documents with higher probabilities, and the search results are finally obtained. Compared to the prior art, which only relies on the score of each candidate document to sort and filter the multiple candidate documents obtained in each hop, this solution can sort from the perspective of the global optimality of the global search path, thereby improving the accuracy and efficiency of information retrieval.

[0017] As an optional implementation method, obtaining the retrieval result corresponding to the Query based on the K(i+1) i+1th hop candidate documents includes: when the K(i+1) i+1th hop candidate documents reach the first preset stop condition, obtaining the retrieval result corresponding to the Query based on the K(i+1) i+1th hop candidate documents.

[0018] Wherein, the retrieval result corresponding to the Query is obtained based on the K(i+1) i+1-th hop candidate documents, including: obtaining the path corresponding to each candidate document in the K(i+1) i+1-th hop candidate documents, the path corresponding to each candidate document being a path starting from the candidate document of the first hop and ending at the candidate document of the i+1-th hop; wherein, the path corresponding to each candidate document is used as the retrieval result corresponding to the Query; or, each candidate document is used as the retrieval result corresponding to the Query, and the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

[0019] Compared with the prior art which only outputs a document collection, this solution makes the search results explainable by outputting the entire path.

[0020] As an optional implementation, if the K(i+1) candidate documents at the i+1th hop do not meet the first preset stop condition, set i=i+1 and repeat steps S1-S4 until the first preset stop condition is met.

[0021] The first preset stop condition is that a stop symbol exists in at least one candidate document among the K(i+1) candidate documents at the (i+1)th hop.

[0022] In a second aspect, an embodiment of the present application provides a model training method, comprising: obtaining a training sample, wherein the training sample comprises a retrieval text Query and a positive sample; the positive sample comprises a path corresponding to the first document sample of the N+1th hop, wherein the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; the first document sample of the N+1th hop is a document sample corresponding to a path having a probability higher than that of the remaining K(N+1)-1 among the K(N+1) document samples of the N+1th hop; the probability of the path corresponding to any document sample Pjy(N+1) among the K(N+1) document samples of the N+1th hop is obtained based on the score of the document sample Pjy(N+1) and the probability of the path T, wherein the score is used to characterize the document The relevance of the sample to the Query, the path T is a path starting from the document sample of the first jump and ending at the document sample PjN in the K(N+1)th document samples of the N+1th jump, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers; the first neural network is trained according to the training samples to obtain the trained first neural network.

[0023] As an optional implementation, the first neural network is trained according to the training sample to obtain the trained first neural network, including: S1, obtaining Mi i-th hop candidate samples according to the query, the i-th hop document sample corresponding to the positive sample, Ki-1 i-th hop negative samples and i-th hop path negative samples; any path negative sample in the i-th hop path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer; S2, obtaining the score of each candidate sample in the Mi i-th hop candidate samples; S3, obtaining the score of each candidate sample in the Mi i-th hop candidate samples according to the score of each candidate sample The probabilities of the paths corresponding to the samples are obtained respectively; S4, according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, K(i+1) i+1-th hop document samples are obtained, wherein the probabilities of the paths of the K(i+1) i+1-th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, the K(i+1) is a positive integer not greater than Mi, and the K(i+1) i+1-th hop document samples include the i+1-th hop document samples corresponding to the positive samples and K(i+1)-1 i+1-th hop negative samples; S5, according to the paths corresponding to the K(i+1) i+1-th hop document samples and the positive samples, the parameters of the first neural network are adjusted to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

[0024] In an embodiment of the present application, when training a model, the top K candidate documents with a higher probability of the path are selected, and the negative sample of the next hop is determined based on the top K candidate documents. Compared with the prior art method of randomly determining the negative sample of each hop, this method dynamically determines the negative sample of the next hop based on the top K candidate documents with a higher probability of the path of each hop, thereby achieving dynamic adaptive selection of candidates in each round and enhancing robustness. At the same time, during the training process, the introduction of the path supervision signal in this solution allows the K candidates selected for each hop to change dynamically as the supervision signal adjusts the model, gradually increasing the difficulty of negative samples, enhancing the generalization ability of the model sorting, and improving the model accuracy.

[0025] The adjusting the parameters of the first neural network according to the paths and the positive samples respectively corresponding to the K(i+1)th i+1th hop document samples includes: when the K(i+1)th i+1th hop document samples reach a second preset stop condition, adjusting the parameters of the first neural network according to the paths and the positive samples respectively corresponding to the K(i+1)th i+1th hop document samples.

[0026] Optionally, if the K(i+1)th document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

[0027] Optionally, the second preset stop condition is that a stop symbol exists in at least one candidate sample among the K(i+1)th (i+1)th hop document samples.

[0028] In a third aspect, an embodiment of the present application provides an information retrieval method, comprising: receiving a query input by a user; inputting the query into a first neural network trained by the training method for processing, and obtaining a retrieval result corresponding to the query.

[0029] In a fourth aspect, an embodiment of the present application provides an information retrieval device, comprising: a retrieval module, for S1, obtaining Mi candidate documents of the i+1th hop based on a retrieval text Query and Ki candidate documents of the i-th hop; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi; S2, obtaining a score for each candidate document in the Mi candidate documents of the i+1th hop, wherein the score is used to characterize the relevance of the candidate document to the Query; S3, for any candidate document Pjy(i+1) in the Mi candidate documents of the i+1th hop, obtaining a probability of a path corresponding to the candidate document Pjy(i+1) based on the score of the candidate document Pjy(i+1) and the probability of a path L, wherein the path L is a path starting from the candidate document of the first hop and ending at the candidate document Pji in the Ki candidate documents of the i-th hop. The candidate document Pji is the jth one among the Ki candidate documents of the i-th hop, and the candidate document Pjy(i+1) is the yth one among the candidate documents of the i+1-th hop obtained based on the candidate document Pji of the i-th hop, where j and y are both positive integers; S4, obtain K(i+1) candidate documents of the i+1-th hop according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1-th hop, wherein the K(i+1) candidate documents of the i+1-th hop are candidate documents among the Mi candidate documents of the i+1-th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1-th hop are higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi; a determination module is used for S5, obtaining the retrieval result corresponding to the Query according to the K(i+1) candidate documents of the i+1-th hop.

[0030] Optionally, the determination module is used to: when the K(i+1) i+1th hop candidate documents reach the first preset stop condition, obtain the retrieval result corresponding to the Query based on the K(i+1) i+1th hop candidate documents.

[0031] Optionally, the determination module is further used to: obtain the path corresponding to each candidate document in the K(i+1) i+1-th hop candidate documents, the path corresponding to each candidate document being a path starting from the candidate document of the first hop and ending at the candidate document of the i+1-th hop; wherein, the path corresponding to each candidate document is used as the retrieval result corresponding to the Query; or, each candidate document is used as the retrieval result corresponding to the Query, and the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

[0032] Optionally, the determination module is further used to: if the K(i+1) candidate documents at the i+1th hop do not meet the first preset stop condition, set i=i+1 and repeat steps S1-S4 until the first preset stop condition is met.

[0033] The first preset stop condition is that a stop symbol exists in at least one candidate document among the K(i+1) candidate documents at the (i+1)th hop.

[0034] In a fifth aspect, an embodiment of the present application provides a model training device, comprising: an acquisition module for acquiring training samples, wherein the training samples include a retrieval text Query and a positive sample; the positive sample includes a path corresponding to the first document sample of the N+1th hop, and the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; the first document sample of the N+1th hop is a document sample corresponding to a path whose probability is higher than that of the remaining K(N+1)-1 document samples of the N+1th hop; the probability of the path corresponding to any document sample Pjy(N+1) among the K(N+1) document samples of the N+1th hop is obtained based on the score of the document sample Pjy(N+1) and the probability of the path T, wherein the score is used to characterize the document The relevance of the sample to the Query, the path T is a path starting from the document sample of the first jump and ending at the document sample PjN in the K(N+1)th document sample of the N+1th jump, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers; a training module is used to train the first neural network according to the training samples to obtain the trained first neural network.

[0035] As an optional implementation method, the training module is used to: S1, obtain Mi i-th hop candidate samples according to the query, the i-th hop document sample corresponding to the positive sample, Ki-1 i-th hop negative samples and i-th hop path negative samples; any path negative sample in the i-th hop path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer; S2, obtain the score of each candidate sample in the Mi i-th hop candidate samples; S3, obtain the probability of the path corresponding to each candidate sample according to the score of each candidate sample in the Mi i-th hop candidate samples; S4 , obtaining K(i+1) i+1th hop document samples according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, wherein the probabilities of the paths of the K(i+1) i+1th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, the K(i+1) is a positive integer not greater than Mi, and the K(i+1) i+1th hop document samples include the i+1th hop document sample corresponding to the positive sample and K(i+1)-1 i+1th hop negative samples; S5, adjusting the parameters of the first neural network according to the paths corresponding to the K(i+1) i+1th hop document samples and the positive samples, to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

[0036] Among them, the training module is also used to: when the K(i+1)th i+1-th hop document samples reach a second preset stop condition, adjust the parameters of the first neural network according to the paths and the positive samples corresponding to the K(i+1)th i+1-th hop document samples.

[0037] The training module is further configured to: if the K(i+1)th document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

[0038] The second preset stop condition is that a stop symbol exists in at least one candidate sample among the K(i+1)th document samples at the (i+1)th hop.

[0039] In a sixth aspect, an embodiment of the present application provides an information retrieval system, comprising: a receiving module for receiving a query input by a user; a processing module for inputting the query into a first neural network trained by the training method for processing to obtain a retrieval result corresponding to the query.

[0040] In the seventh aspect, the present application provides a computer storage medium comprising computer instructions, which, when the computer instructions are run on an electronic device, enable the electronic device to execute the method provided in any possible implementation of the first aspect and / or any possible implementation of the second aspect and / or any possible implementation of the third aspect.

[0041] In an eighth aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is run on a computer, enables the computer to execute the method provided by any possible implementation of the first aspect and / or any possible implementation of the second aspect and / or any possible implementation of the third aspect.

[0042] In the ninth aspect, an embodiment of the present application provides an information retrieval device, comprising a processor and a memory; wherein the memory is used to store program code, and the processor is used to call the program code to execute the method provided by any possible implementation of the first aspect and / or any possible implementation of the second aspect and / or the method provided by any possible implementation of the third aspect.

[0043] It is understandable that the apparatus described in the fourth aspect, the apparatus described in the fifth aspect, the system described in the sixth aspect, the computer storage medium described in the seventh aspect, the computer program product described in the eighth aspect, or the apparatus described in the ninth aspect are all used to execute any of the methods provided in the first aspect, any of the methods provided in the second aspect, and any of the methods provided in the third aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The following is an introduction to the drawings used in the embodiments of this application.

[0045] Figure 1 It is a schematic diagram of information retrieval in the prior art;

[0046] Figure 2a A schematic diagram of an artificial intelligence main framework provided in an embodiment of the present application;

[0047] Figure 2b A schematic diagram of an application environment provided in an embodiment of the present application;

[0048] Figure 2c A schematic diagram of the structure of a neural network processor provided in an embodiment of the present application;

[0049] Figure 3 This is a flow chart of an information retrieval method provided in an embodiment of the present application;

[0050] Figure 4 This is a flow chart of a model training method provided in an embodiment of the present application;

[0051] Figure 5 This is a flow chart of an information retrieval method provided in an embodiment of the present application;

[0052] Figure 6 This is a schematic diagram of a web page hyperlink provided by an embodiment of the present application;

[0053] Figure 7 is a schematic diagram of an information retrieval method provided in an embodiment of the present application;

[0054] Figure 8 This is a sample schematic diagram provided in an embodiment of the present application;

[0055] Figure 9 This is another sample schematic diagram provided in the embodiment of the present application;

[0056] Figure 10 This is a schematic diagram of the structure of an information retrieval device provided in an embodiment of the present application;

[0057] Figure 11 It is a structural diagram of an information retrieval device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0059] Figure 2a A schematic diagram of an artificial intelligence main framework is shown, which describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence field needs.

[0060] The following explains the above artificial intelligence theme framework from the two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0061] The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom."

[0062] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0063] (1) Infrastructure

[0064] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0065] (2) Data

[0066] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0067] (3) Data processing

[0068] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0069] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0070] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0071] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0072] (4) General ability

[0073] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0074] (5) Smart products and industry applications

[0075] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, safe cities, smart terminals, etc.

[0076] See attached Figure 2b The embodiment of the present invention provides a system architecture 200. A data collection device 260 is used to collect webpage / text paragraph data and store it in a database 230. A training device 220 generates a target model / rule 201 based on the webpage / text paragraph data maintained in the database 230. The following describes in more detail how the training device 220 generates the target model / rule 201 based on the webpage / text paragraph data. The target model / rule 201 can generate corresponding search results based on the search text input by the user.

[0077] The work of each layer in a deep neural network can be expressed mathematically as To describe: From a physical perspective, the work of each layer in a deep neural network can be understood as completing the transformation from input space to output space (i.e., from the row space to the column space of a matrix) through five operations on the input space (a set of input vectors). These five operations include: 1. Dimensionality increase / decrease; 2. Zoom in / out; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are represented by Completed, operation 4 is completed by +b, and operation 5 is implemented by a(). The word "space" is used here because the object being classified is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things. Among them, W is a weight vector, and each value in the vector represents the weight value of a neuron in this layer of the neural network. This vector W determines the spatial transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space. The purpose of training a deep neural network is to eventually obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by many layers of vectors W). Therefore, the training process of a neural network is essentially to learn how to control spatial transformation, and more specifically to learn the weight matrix.

[0078] Because we want the output of a deep neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the deep neural network). For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value, and this adjustment is continued until the neural network can predict the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value." This is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Then, the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0079] The target model / rule obtained by the training device 220 can be applied to different systems or devices. Figure 2b In the embodiment, the execution device 210 is configured with an I / O interface 212 for data interaction with external devices, and a “user” can input data into the I / O interface 212 through a client device 240 .

[0080] The execution device 210 can call data, code, etc. in the data storage system 250 , and can also store data, instructions, etc. in the data storage system 250 .

[0081] The calculation module 211 uses the target model / rule 201 to process the input data, including: S1, obtaining Mi candidate documents of the i+1th hop according to the search text Query input by the user and the K candidate documents of the i-th hop; wherein i, K, and Mi are all positive integers, and K is not greater than Mi; S2, obtaining the score of each candidate document in the Mi candidate documents of the i+1th hop, and the score is used to represent the relevance of the candidate document to the Query; S3, for any candidate document Pjy(i+1) in the Mi candidate documents of the i+1th hop, obtaining the probability of the path corresponding to the candidate document Pjy(i+1) according to the score of the candidate document Pjy(i+1) and the probability of the path L, wherein the path Path L is a path starting from the candidate document of the first hop and ending at the candidate document Pji among the K candidate documents of the i-th hop, the candidate document Pji is the j-th among the K candidate documents of the i-th hop, and the candidate document Pjy(i+1) is the y-th among the candidate documents of the i+1-th hop obtained based on the candidate document Pji of the i-th hop; S4, according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1-th hop, K candidate documents of the i+1-th hop are obtained, wherein the K candidate documents of the i+1-th hop are the K documents with higher probabilities of the paths among the Mi candidate documents; according to the K candidate documents of the i-th hop and the K candidate documents of the i+1-th hop, the retrieval result corresponding to the Query is obtained.

[0082] The correlation function module 213 can process the search results corresponding to the query to obtain a processed result. For example, the path with the highest probability can be used as the processing result.

[0083] The association function module 214 may also extract a target answer based on the path with the highest probability.

[0084] Finally, the I / O interface 212 returns the processing result to the client device 240 and provides it to the user.

[0085] More deeply, the training device 220 can generate corresponding target models / rules 201 based on different data for different goals to provide users with better results.

[0086] In the attached Figure 2bIn the case shown in , the user can manually specify the data to be input into the execution device 210, for example, by operating in the interface provided by the I / O interface 212. In another case, the client device 240 can automatically input data into the I / O interface 212 and obtain the results. If the automatic data input of the client device 240 requires user authorization, the user can set the corresponding permissions in the client device 240. The user can view the results output by the execution device 210 on the client device 240, and the specific presentation form can be specific methods such as display, sound, and action. The client device 240 can also act as a data collection terminal to store the collected web page / text paragraph data in the database 230.

[0087] It is worth noting that Figure 2b This is only a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in the attached Figure 2b In the embodiment, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0088] Reference Figure 2c FIG. 1 is a chip hardware structure diagram provided by an embodiment of the present invention.

[0089] The neural network processor NPU 50 is mounted on the host CPU as a coprocessor and is assigned tasks by the host CPU. The core part of the NPU is the arithmetic circuit 50. The controller 504 controls the arithmetic circuit 503 to extract data from the memory (weight memory or input memory) and perform calculations.

[0090] In some implementations, the arithmetic circuit 503 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0091] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 502 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 501 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 508.

[0092] The vector calculation unit 507 can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector calculation unit 507 can be used for network calculations of non-convolutional / non-FC layers in a neural network, such as pooling, batch normalization, local response normalization, etc.

[0093] In some implementations, the vector calculation unit 507 can store the processed output vector to the unified memory 506. For example, the vector calculation unit 507 can apply a nonlinear function to the output of the operation circuit 503, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 507 generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 503, for example, for use in a subsequent layer in a neural network.

[0094] The unified memory 506 is used to store input data and output data.

[0095] The memory unit access controller 505 (Direct Memory Access Controller, DMAC) moves the input data in the external memory to the input memory 501 and / or the unified memory 506, stores the weight data in the external memory into the weight memory 502, and stores the data in the unified memory 506 into the external memory.

[0096] The bus interface unit (BIU) 510 is used to implement interaction between the main CPU, DMAC and instruction fetch memory 509 through the bus.

[0097] An instruction fetch buffer 509 connected to the controller 504 and used to store instructions used by the controller 504;

[0098] The controller 504 is used to call the instructions cached in the memory 509 to control the working process of the computing accelerator.

[0099] Generally, the unified memory 506, the input memory 501, the weight memory 502 and the instruction fetch memory 509 are all on-chip memories, and the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM) or other readable and writable memory.

[0100] The information retrieval method provided in the embodiment of the present application is described in detail below. Figure 3 As shown in FIG, a flow chart of an information retrieval method provided in an embodiment of the present application is provided. Figure 3 As shown, the method includes steps 301-305, which are specifically as follows:

[0101] 301. Obtain Mi candidate documents at the (i+1)th hop based on the search text Query and Ki candidate documents at the i-th hop; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi;

[0102] The above search text Query can be the text content entered by the user in the search box.

[0103] As an optional implementation, before step 301, the method further includes:

[0104] Build a candidate document library.

[0105] The candidate document library includes documents with paragraph granularity. That is, any candidate document in the candidate document library is composed of a paragraph of content.

[0106] Optionally, the candidate document library is obtained by collecting multiple web pages or texts, breaking the content of each web page or text into paragraphs, using the title as the label of each paragraph, and indexing all the labeled paragraphs.

[0107] Optionally, Mi candidate documents are obtained by inputting the query and K candidate documents of the i-th hop into a retriever composed of a deep pre-trained language model.

[0108] As an optional implementation method, the query is directly concatenated with each candidate document in the K i-th hop candidate documents, and then the K concatenated texts are retrieved by the retriever to obtain Mi i+1-th hop candidate documents.

[0109] Specifically, a deep pre-trained language model is used to calculate the representation vector of each pre-selected document in a massive number of pre-selected documents, and each representation vector is stored and indexed; then the representation vector of each query is obtained for different queries; finally, multiple documents most relevant to the query are obtained through a fast search based on the vector index.

[0110] The above description only takes the example of directly concatenating the query with each candidate document in the K i-th hop candidate documents. It can also be in other forms, which is not specifically limited in this solution.

[0111] 302. Obtain a score for each candidate document in the Mi candidate documents at the (i+1)th hop, where the score is used to represent the relevance of the candidate document to the query.

[0112] Optionally, the retriever composed of the deep pre-trained language model can score each candidate document based on the relevance of the candidate document to the query, and then output the score of each candidate document in the Mi i+1-th hop candidate documents.

[0113] As an optional implementation method, we will use the deep pre-trained language model Bert, which is open sourced by Google, as an example. This model Bert is obtained by stacking some basic network units, such as the Transformer network, into a huge network model and using massive data for pre-training. Taking the i-th hop as an example, the input is Query and the r-th candidate document Pri obtained in the i-th hop. The output is the model's score for the candidate document Pri under the condition of input Q, which can be shown in the following formula (1):

[0114] Score(Pri|Q)=Model([Pri,Q]), where ri=1, 2,...Ni; (1)

[0115] 303. For any candidate document Pjy(i+1) among the Mi candidate documents at the i+1th hop, obtain the probability of the path corresponding to the candidate document Pjy(i+1) based on the score of the candidate document Pjy(i+1) and the probability of the path L, where the path L is a path starting from the candidate document of the first hop and ending at the candidate document Pji among the Ki candidate documents at the i-th hop, the candidate document Pji is the j-th candidate document among the Ki candidate documents at the i-th hop, and the candidate document Pjy(i+1) is the y-th candidate document obtained based on the candidate document Pji.

[0116] It should be noted that the aforementioned candidate document Pjy(i+1) is used to indicate a random selection. The candidate document is obtained based on the candidate document of the previous hop. The values ​​j and y are only used for ease of description and do not have any specific limiting effect.

[0117] Specifically, when i=1 (i.e., the first jump), the input is Query and the rth candidate document Pr1 obtained in the first jump, and the output is the model's score Score(Pr1|Q) for the candidate document Pr1 under the condition of input Q, which can be referred to as shown in the above formula (1).

[0118] Based on the M1 candidate documents obtained in the first hop, the score of each candidate document and the probability of the path corresponding to each candidate document are obtained respectively.

[0119] The probability of a path is the chance that it will be selected. A path begins with the candidate document of the first hop and ends at any candidate document of the i-th hop. It indicates that the candidate document for the current hop is derived based on the candidate document corresponding to the preceding path. The path can be used to explain the candidate document for the current hop, making it easier to understand the process by which it was derived.

[0120] For example, let's take a path starting from the candidate document at the first hop and ending at any candidate document at the fourth hop as an example. This path includes four hops and, accordingly, four candidate documents. The candidate document at the first hop can be used to obtain the candidate document at the second hop corresponding to the path. The candidate document at the second hop can be used to obtain the candidate document at the third hop corresponding to the path. The candidate document at the third hop can be used to obtain the candidate document at the fourth hop corresponding to the path. Each path is unique.

[0121] When i=1, obtain K candidate documents with higher scores from the M1 candidate documents, and normalize the scores of the K candidate documents into probabilities, so as to obtain K probabilities (conditional probabilities) of the first-hop search path under the condition of input Q.

[0122] Specifically, the scores of the K candidate documents may be normalized using a Softmax function to obtain K probabilities.

[0123] Accordingly, the probability of the path corresponding to any candidate document s1 among the K candidate documents in the first hop can be expressed as:

[0124] G(Ps1|Q)=Softmax(TopK(Score(Ps1|Q))), where s1=1, 2, ...K; (2)

[0125] The above description uses the probability of the path of the first hop as an example. Correspondingly, the probability of the path corresponding to the candidate document Pst2 of the second hop can be expressed as:

[0126] G(Pst2, Ps1|Q)=G(Pst2|Q, Ps1)*G(Ps1|Q), where st2=1, 2, ... N2; (3)

[0127] The candidate document Pst2 of the second hop is obtained based on the candidate document Ps1 of the first hop.

[0128] By analogy, the probability of the path corresponding to any candidate document Pjy(i+1) in the candidate documents of the i+1th hop can be obtained.

[0129] 304. Obtain K(i+1) candidate documents of the i+1th hop based on the probabilities of the paths corresponding to the Mi candidate documents of the i+1th hop, wherein the K(i+1) candidate documents of the i+1th hop are candidate documents among the Mi candidate documents of the i+1th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1th hop are all higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi.

[0130] Specifically, based on the K paths corresponding to the K candidate documents of the first hop, M2 candidate documents for the second hop are generated. Based on the score of each candidate document for the second hop and the probability of the path of the previous hop corresponding to each candidate document, the probability of the path corresponding to each candidate document for the second hop is obtained. The K paths with higher probabilities are used as the basis paths for the next hop, resulting in multiple candidate documents. This process is repeated to obtain K documents with higher probabilities for the path of the i+1th hop.

[0131] That is to say, the probabilities of the paths of the K i+1-th candidate documents are all higher than the probabilities of the paths of the remaining candidate documents in the Mi candidate documents.

[0132] It should be noted that the embodiment of the present application is described by taking K as an example of selecting each hop. It is also possible to select a different number for each hop, and this solution does not make specific limitations on this.

[0133] The method for generating candidate next-hop documents may use documents linked to hyperlinks in the previous-hop document as candidate next-hop documents. For example, documents linked to webpage hyperlinks in the previous-hop document may be used as candidate next-hop documents. A webpage hyperlink refers to a hyperlink in an internet webpage that can be clicked to jump to another webpage or webpage paragraph. Alternatively, candidate next-hop documents may be obtained through other methods, which are not specifically limited in this solution.

[0134] 305. Obtain the search result corresponding to the Query according to the K(i+1) candidate documents at the i+1th hop.

[0135] Optionally, step 305 may include:

[0136] 3051. Determine whether the K(i+1) candidate documents at the (i+1)th hop meet a first preset stopping condition;

[0137] 3052. If the K(i+1) candidate documents at the i+1th hop meet the first preset stop condition, obtain the search result corresponding to the Query based on the K(i+1) candidate documents at the i+1th hop.

[0138] The first preset stopping condition can be to determine whether at least one of the K candidate documents contains a stop symbol. If so, the search result corresponding to the query is returned. If not, i is set to i + 1, and steps 301-304 are repeated. For example, the query input by the user is merged with the K candidate documents obtained above, and the merged text is used as the new input for the search engine to perform a new search and obtain the candidate document for the next hop. The search engine used for each hop in this solution can also be different, and this solution does not specifically limit this.

[0139] It should be noted that the stop character can be any document with a special tag, so that the model can automatically stop iterative retrieval.

[0140] The above step 305 may include the following two implementation forms:

[0141] As an optional implementation method, by obtaining the path corresponding to each candidate document in the K i+1-th hop candidate documents, the path corresponding to each candidate document is a path starting from the candidate document of the first hop and ending at the candidate document of the i+1-th hop; the path corresponding to each candidate document is used as the retrieval result corresponding to the Query.

[0142] Compared with the prior art which only outputs a document collection, this solution makes the search results explainable by outputting the entire path.

[0143] That is, based on the query input by the user, K documents are output. Each document includes the candidate document at the i+1th hop and the candidate documents at the previous i hops corresponding to the candidate document. Specifically, each document corresponds to the i+1th candidate document at the i+1th hop, and the i+1th candidate document starts from the candidate document at the first hop and ends at the candidate document at the i+1th hop.

[0144] Furthermore, the K candidate documents at the (i+1)th hop may be sorted based on the probabilities of the paths corresponding to them, and then the sorted K documents may be output.

[0145] As another optional implementation, each candidate document in the K i+1-th hop candidate documents is used as the retrieval result corresponding to the Query, wherein the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

[0146] That is, the search result includes each candidate document in the K candidate documents at the i+1th hop, and the path corresponding to each candidate document is used to explain the search result corresponding to the candidate document, so that users can understand the process of obtaining the search result.

[0147] This embodiment is described by taking the retrieval results including candidate documents of two hops and candidate documents of i+1 hops as an example. It can also be any other hops, and this solution does not make any specific limitation on this.

[0148] In this embodiment of the present application, the probability of the path corresponding to the candidate document in the current hop is obtained by taking the score of each candidate document from the Mi candidate documents obtained in the current hop and the probability of the path corresponding to the candidate document in the previous hop. The next hop candidate document is then obtained based on the multiple candidate documents with higher probabilities, and the search results are finally obtained. Compared to the prior art, which only relies on the score of each candidate document to sort and filter the multiple candidate documents obtained in each hop, this solution can sort from the perspective of the global optimality of the global search path, thereby improving the accuracy and efficiency of information retrieval.

[0149] As an optional implementation, the above Figure 3 The embodiment shown may be to input the query into the first neural network for processing, thereby obtaining the search result corresponding to the query. Figure 4 The model training method shown is obtained by training. The model training method provided in the embodiment of this application is described in detail below. Figure 4 , is a flow chart of a model training method provided in an embodiment of the present application. The method includes steps 401-402, which are as follows:

[0150] 401. Obtain training samples, wherein the training samples include a search text Query and a positive sample; the positive sample includes a path corresponding to the first document sample of the N+1th hop, wherein the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; the first document sample of the N+1th hop is a document sample corresponding to a path whose probability is higher than that of the remaining K(N+1)-1 document samples of the N+1th hop; the probability of a path corresponding to any document sample Pjy(N+1) among the K(N+1) document samples of the N+1th hop is obtained based on the score of the document sample Pjy(N+1) and the probability of the path T, wherein the score Used to characterize the relevance of the document sample to the query, the path T is a path starting from the document sample of the first jump and ending at the document sample PjN in the K(N+1)th document sample of the N+1th jump, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers;

[0151] It should be noted that the number of document samples in each of the above hops may be the same or different, and this solution does not impose any specific limitation on this.

[0152] The following description is made by taking the example that the number of document samples in each hop is K.

[0153] Specifically, the training sample includes the query, the path corresponding to the first document sample at the N+1th hop, and K-1 negative samples from the first hop. Accordingly, it can be understood that the positive sample mentioned above may include positive document samples corresponding to each hop in the N+1 hops. The positive document samples at each hop are derived based on the positive document samples at the previous hop.

[0154] The N+1 is the number of hops or rounds required to obtain the search result corresponding to the query. In other words, after N+1 rounds of searching, the final search result is obtained.

[0155] 402. Train the first neural network according to the training samples to obtain the trained first neural network.

[0156] Step 402 may specifically include:

[0157] S1. Obtain Mi candidate samples of the i-th hop according to the query, the document sample of the i-th hop corresponding to the positive sample, Ki-1 negative samples of the i-th hop, and the i-th path negative samples; any path negative sample in the i-th path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer;

[0158] S2. Obtain a score for each candidate sample among the Mi candidate samples of the i-th hop;

[0159] S3. Obtain the probability of the path corresponding to each candidate sample according to the score of each candidate sample in the Mi candidate samples of the i-th hop;

[0160] S4. Obtain K(i+1) document samples of the i+1th hop according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, wherein the probabilities of the paths of the K(i+1) i+1th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, K(i+1) is a positive integer not greater than Mi, and the K(i+1) i+1th hop document samples include the i+1th hop document sample corresponding to the positive sample and K(i+1)-1 i+1th hop negative samples;

[0161] S5. Adjust the parameters of the first neural network according to the paths and the positive samples corresponding to the K(i+1)th document samples at the i+1th hop, to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

[0162] Step S5 may include: when the K(i+1)th i+1th hop document samples reach a second preset stop condition, adjusting the parameters of the first neural network according to the paths and the positive samples corresponding to the Kth i+1th hop document samples, to obtain the trained first neural network.

[0163] If the K(i+1)th document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

[0164] The second preset condition may be that at least one candidate sample among the K(i+1)th document samples at the i+1th hop contains a stop character. For the description of the stop character, please refer to the above embodiment and will not be repeated here.

[0165] When i=1, the path negative sample of the first hop does not exist, that is, the first hop does not input the path negative sample during input.

[0166] For determining the scores of candidate samples, the probabilities of paths corresponding to candidate samples, etc. during the above training process, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0167] It is understandable that each hop may include K samples, one of which is a positive sample for the hop, and the remaining K-1 samples are negative samples for the hop. The above K may be an arbitrarily set value.

[0168] That is, during the training process, the correct candidate documents for the current hop and a fixed number of incorrect candidate documents are applied to each retrieval step. Specifically, the deep pre-trained language model corresponding to each hop is trained based on the positive and negative sample pairs generated for each hop. The fixed number of incorrect candidate documents is the K-1 negative samples for each hop. These negative samples are dynamically generated based on the samples from the previous hop. Furthermore, probabilistic models are used to globally model and score multi-round search paths. This allows the supervisory signal from each hop to supervise all paths prior to the current hop during training, enhancing the model's retrieval accuracy.

[0169] Any path negative sample in the above-mentioned i-th hop path negative samples includes a path starting from any document sample in the K document samples in the first hop and ending at any negative sample in the K-1 negative samples in the i-th hop.

[0170] That is, during the training process, the correct path at the current jump and the wrong path at the current jump are applied to the K paths expressed by the probability of the path in each jump. For example, the positive sample corresponding to the first jump is Pagei + , the negative sample is Pagei 1- , Pagei 2- ...to Pagei K , the positive sample corresponding to the second jump is Pagej + , the negative sample is Pagej 1- , Pagej 2- ...to Pagej K The positive sample of the second hop retrieval path is (Pagei + ,Pagej + ), then the negative sample of the second hop path is (Pagei + ,Pagej 1- ),(Pagei + ,Pagej 2- )…(Pagei 1- ,Pagej - ), (Pagei2- ,Pagej - )…Specifically, supervision can be formed on the deep pre-trained language model corresponding to each hop based on the positive and negative paths formed by each hop. Accordingly, the path negative samples of the third hop include samples starting from the first hop and ending at all samples in the third hop except the document samples of the third hop corresponding to the positive samples. In other words, the negative samples of the third hop are the ending point. By applying a path supervision signal when the model dynamically selects the first K candidates during training, the path arriving in the current round is supervised at all times. This, combined with the local supervision signal, can significantly improve the model accuracy. At the same time, the K candidates can be continuously changed with the supervision signal, thereby enhancing the robustness of the model.

[0171] The negative samples in the training samples input in the embodiment of the present application are obtained based on K samples with higher probabilities. Compared with the prior art method of training based on randomly selected negative samples, this solution can improve the accuracy of the model.

[0172] This solution determines the negative sample for the next hop based on the top K candidate documents with higher probability along the path. Compared to the existing method of randomly determining negative samples for each hop, this method dynamically determines the negative sample for the next hop based on the top K candidate documents with higher probability along the path of each hop, achieving dynamic and adaptive selection of candidates in each round and enhancing robustness. Furthermore, during the training process, this solution introduces a path supervision signal, allowing the K candidates selected for each hop to change dynamically as the supervision signal adjusts the model, gradually increasing the difficulty of negative samples, enhancing the generalization ability of the model sorting, and improving model accuracy.

[0173] Reference Figure 5 , which is a flow chart of an information retrieval method provided in an embodiment of the present application. The method is applied to a question-and-answer scenario. It includes steps 501-504, as follows:

[0174] 501. Build a candidate document library;

[0175] Using wiki pages as data sources, breaking them down into paragraphs, and using titles as labels for each paragraph, we construct a candidate document library by indexing. Optionally, the paragraphs contain hyperlinks to other web pages. Figure 6 shown.

[0176] 502. Receive the query input by the user;

[0177] Optionally, the question-answering system receives a query input by a user, such as “where did Algeria qualify for the first time into the round of 16?”, that is, where did Algeria qualify for the first time into the round of 16?

[0178] 503. Obtain search results corresponding to the query;

[0179] Specifically, a first-hop search is performed on the query to obtain M1 candidate documents, and a score is obtained for each candidate document. In this embodiment, the deep pre-trained language model is selected as Bert, and Bert1 represents the model used in the first-hop search, where the correct document is "Wiki: Algeria at the FIFA World Cup".

[0180] Sort the scores of each candidate document in the first hop from high to low, and select the K candidate documents with higher scores. Use the Softmax function to normalize the K scores into probabilities. This embodiment of the application is described with K being 4. Figure 7 As shown in , the hyperlinks of the top 4 first-hop search results (Z1, Z2, Z3, and Z4) are used to form candidate documents for the second-hop search. For example, wiki:2014FIFA World Cup is a document linked to by a hyperlink in the paragraph wiki:Algeria at the FIFA World Cup. Figure 7 The document content corresponding to Mz.

[0181] Confirm whether there is a stop character in the candidate document of the first jump. If there is no stop character, use the second jump deep pre-trained language model to calculate the score from the Z-Mz path. For example, concatenate Query with the first jump retrieval document Zi and the second jump retrieval document Mzij and input them into Bert for scoring. The result is bij.

[0182] The scores of the candidate documents calculated in the second hop are normalized to bzpij using the Softmax function, and the probability of the first and second hop retrieval paths constituting api×bzpij is calculated, as follows: Figure 7 shown.

[0183] The top 4 paths are selected based on the probability api×bzpij of the first and second hop search paths. The next hop probability is then calculated and the paths are screened until a stop character appears in the candidate documents. Each search path and its corresponding probability are then returned.

[0184] 504. Extract answers based on the search results corresponding to the Query and send them to the user.

[0185] The documents in the retrieved path are concatenated and fed into the answer extraction model for answer extraction. For example, if the retrieved path is wiki:Algeria at the FIFA World Cup -> wiki:2014FIFA World Cup, and the latter wiki document contains "Brazil," machine reading comprehension can be used to extract the answer and return it to the user.

[0186] Optionally, the above implementation method is obtained by inputting the Query into a first neural network for processing.

[0187] The first neural network is trained based on the following training method:

[0188] During training, the correct document supervision signal is applied to the corresponding deep pre-trained language model in each round, such as Figure 8 As shown in the figure, when training the first round of deep pre-training language model, the model will be told that wiki:Algeria at the FIFAWorld Cup is a positive sample, and a fixed negative sample will be selected for it. The same is true for the second round of retrieval.

[0189] Moreover, during the training process, for each round of K retrieval path scores expressed by probability, the correct candidate path at the current round and the wrong candidate path corresponding to the candidate document of the previous round of retrieval results are applied, such as Figure 9 After retrieving the Mz series in the second round, we apply both correct and incorrect path supervision signals to the first and second rounds of deep pre-trained language models. The correct signal is wiki:Algeria at the FIFAWorld Cup->wiki:2014FIFAWorldCup, and the incorrect signal in the second round is generated by the top 4 candidates in the first round.

[0190] Reference Figure 10 , is a structural diagram of an information retrieval device provided in an embodiment of the present application. Figure 10 As shown, the device includes a retrieval module 1001 and a determination module 1002, which are specifically as follows:

[0191] The retrieval module 1001 is used for S1, obtaining Mi candidate documents of the i+1th hop according to the search text Query and Ki candidate documents of the i-th hop; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi; S2, obtaining the score of each candidate document in the Mi candidate documents of the i+1th hop, wherein the score is used to represent the relevance of the candidate document to the Query; S3, for any candidate document Pjy(i+1) in the Mi candidate documents of the i+1th hop, obtaining the probability of the path corresponding to the candidate document Pjy(i+1) according to the score of the candidate document Pjy(i+1) and the probability of the path L, wherein the path L is a path starting from the candidate document of the first hop and ending at the candidate document in the Ki candidate documents of the i-th hop. The file Pji is the path to the end point, the candidate document Pji is the jth one among the Ki candidate documents of the i-th hop, and the candidate document Pjy(i+1) is the yth one among the candidate documents of the i+1-th hop obtained according to the candidate document Pji of the i-th hop, where j and y are both positive integers; S4, obtaining K(i+1) candidate documents of the i+1-th hop according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1-th hop, wherein the K(i+1) candidate documents of the i+1-th hop are candidate documents among the Mi candidate documents of the i+1-th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1-th hop are all higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi;

[0192] The determination module 1002 is used for S5, obtaining the search result corresponding to the query according to the K(i+1) candidate documents at the i+1th hop.

[0193] The determining module 1002 is configured to:

[0194] When the K(i+1) candidate documents at the i+1th hop reach the first preset stop condition, a retrieval result corresponding to the Query is obtained according to the K(i+1) candidate documents at the i+1th hop.

[0195] The determining module 1002 is further configured to:

[0196] Obtain a path corresponding to each candidate document among the K(i+1) candidate documents at the i+1th hop, where the path corresponding to each candidate document is a path starting from the candidate document of the first hop and ending at the candidate document at the i+1th hop; wherein the path corresponding to each candidate document is used as a retrieval result corresponding to the Query; or, each candidate document is used as a retrieval result corresponding to the Query, and the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

[0197] Optionally, the determining module 1002 is further configured to:

[0198] If the K(i+1) candidate documents at the i+1th hop do not meet the first preset stopping condition, set i=i+1 and repeat steps S1-S4 until the first preset stopping condition is met.

[0199] Optionally, the first preset stop condition is that a stop symbol exists in at least one candidate document among the K(i+1) candidate documents at the (i+1)th hop.

[0200] The embodiment of the present application provides a model training device, including: an acquisition module, for acquiring training samples, wherein the training samples include a search text Query and a positive sample; the positive sample includes a path corresponding to the first document sample of the N+1th hop, and the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; the first document sample of the N+1th hop is a document sample corresponding to a path with a probability higher than that of the remaining K(N+1)-1 among the K(N+1) document samples of the N+1th hop; the probability of a path corresponding to any document sample Pjy(N+1) among the K(N+1) document samples of the N+1th hop is calculated based on the score of the document sample Pjy(N+1) and the probability of the path T. The score is obtained by the method of the present invention, wherein the score is used to characterize the relevance of the document sample to the query, the path T is a path starting from the document sample of the first jump and ending at the document sample PjN in the K(N+1)th document samples of the N+1th jump, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers;

[0201] A training module is used to train the first neural network according to the training samples to obtain the trained first neural network.

[0202] As an optional implementation method, the training module is used to: S1, obtain Mi i-th hop candidate samples according to the query, the i-th hop document sample corresponding to the positive sample, Ki-1 i-th hop negative samples and i-th hop path negative samples; any path negative sample in the i-th hop path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer; S2, obtain the score of each candidate sample in the Mi i-th hop candidate samples; S3, obtain the probability of the path corresponding to each candidate sample according to the score of each candidate sample in the Mi i-th hop candidate samples; S4 , obtaining K(i+1) i+1th hop document samples according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, wherein the probabilities of the paths of the K(i+1) i+1th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, the K(i+1) is a positive integer not greater than Mi, and the K(i+1) i+1th hop document samples include the i+1th hop document sample corresponding to the positive sample and K(i+1)-1 i+1th hop negative samples; S5, adjusting the parameters of the first neural network according to the paths corresponding to the K(i+1) i+1th hop document samples and the positive samples, to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

[0203] Among them, the training module is also used to: when the K(i+1)th i+1-th hop document samples reach a second preset stop condition, adjust the parameters of the first neural network according to the paths and the positive samples corresponding to the K(i+1)th i+1-th hop document samples.

[0204] The training module is further configured to: if the K(i+1)th document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

[0205] The second preset stop condition is that a stop symbol exists in at least one candidate sample among the K(i+1)th document samples at the (i+1)th hop.

[0206] Reference Figure 11 As shown in FIG, an information retrieval device is provided in an embodiment of the present application. Figure 11As shown, the apparatus 1100 includes at least one processor 1101, at least one memory 1102, and at least one communication interface 1103. The processor 1101, the memory 1102, and the communication interface 1103 are connected via the communication bus and communicate with each other.

[0207] The processor 1101 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.

[0208] The communication interface 1103 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0209] The memory 1102 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0210] The memory 1102 is used to store application code for executing the above solution, and the execution is controlled by the processor 1101. The processor 1101 is used to execute the application code stored in the memory 1102.

[0211] The codes stored in the memory 1102 may execute any of the information retrieval methods provided above.

[0212] An embodiment of the present application provides an information retrieval system, including: a receiving module for receiving a query input by a user; a processing module for inputting the query into a first neural network trained by the training method for processing to obtain a retrieval result corresponding to the query.

[0213] An embodiment of the present application also provides a chip system, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a line; the interface circuit is used to receive a signal from the memory of the electronic device and send the signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method.

[0214] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or processor executes one or more steps in any of the above methods.

[0215] The present application also provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

[0216] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0217] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An information retrieval method, characterized in that: include: S1. Obtain Mi candidate documents of the i+1th hop according to the search text Query and Ki candidate documents of the i-th hop; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi; S2. Obtain a score for each candidate document in the Mi candidate documents at the (i+1)th hop, where the score is used to represent the relevance of the candidate document to the query; S3. For any candidate document Pjy(i+1) among the Mi candidate documents of the i+1th hop, obtain the probability of the path corresponding to the candidate document Pjy(i+1) based on the score of the candidate document Pjy(i+1) and the probability of the path L, wherein the path L is a path starting from the candidate document of the first hop and ending at the candidate document Pji among the Ki candidate documents of the i-th hop, the candidate document Pji is the j-th candidate document among the Ki candidate documents of the i-th hop, and the candidate document Pjy(i+1) is the y-th candidate document among the i+1-th hop obtained based on the candidate document Pji of the i-th hop, where j and y are both positive integers; S4. Obtain K(i+1) candidate documents of the i+1th hop according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1th hop, wherein the K(i+1) candidate documents of the i+1th hop are candidate documents among the Mi candidate documents of the i+1th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1th hop are all higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi; S5. Obtain the search result corresponding to the Query according to the K (i+1) candidate documents at the i+1th hop.

2. The method according to claim 1, characterized in that Obtaining the search result corresponding to the Query based on the K(i+1) candidate documents at the i+1th hop includes: When the K(i+1) candidate documents at the i+1th hop reach a first preset stop condition, a retrieval result corresponding to the Query is obtained according to the K(i+1) candidate documents at the i+1th hop.

3. The method according to claim 2, characterized in that Obtaining the search result corresponding to the Query based on the K(i+1) candidate documents at the i+1th hop includes: Obtain a path corresponding to each candidate document in the K(i+1) candidate documents at the (i+1)th hop, where the path corresponding to each candidate document is a path starting from the candidate document at the first hop and ending at the candidate document at the (i+1)th hop; The path corresponding to each candidate document is used as the search result corresponding to the query; Alternatively, each candidate document is used as a retrieval result corresponding to the Query, and the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

4. The method according to claim 2 or 3, characterized in that The method further comprises: If the K(i+1) candidate documents at the i+1th hop do not meet the first preset stopping condition, set i=i+1 and repeat steps S1-S4 until the first preset stopping condition is met.

5. The method according to claim 2, characterized in that The first preset stop condition is that a stop symbol exists in at least one candidate document among the K(i+1) candidate documents at the (i+1)th hop.

6. A model training method, characterized in that: include: Obtaining a training sample, the training sample including a search text Query and a positive sample; the positive sample including a path corresponding to the first document sample of the N+1th hop, wherein the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; The first document sample of the N+1th hop is the document sample corresponding to the path with a probability higher than that of the remaining K(N+1)-1 document samples in the N+1th hop; the probability of the path corresponding to any document sample Pjy(N+1) in the K(N+1) document samples of the N+1th hop is obtained based on the score of the document sample Pjy(N+1) and the probability of the path T, wherein the score is used to characterize the relevance of the document sample to the query, and the path T is a path starting from the document sample of the first hop and ending at the K(N+1)th ... The document sample PjN in the document sample of the jump is the path with the end point, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers; The first neural network is trained according to the training samples to obtain the trained first neural network.

7. The method according to claim 6, characterized in that The step of training the first neural network according to the training sample to obtain the trained first neural network includes: S1. Obtain Mi candidate samples of the i-th hop according to the query, the document sample of the i-th hop corresponding to the positive sample, Ki-1 negative samples of the i-th hop, and the i-th path negative samples; any path negative sample in the i-th path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer; S2. Obtain a score for each candidate sample among the Mi candidate samples of the i-th hop; S3. Obtain the probability of the path corresponding to each candidate sample according to the score of each candidate sample in the Mi candidate samples of the i-th hop; S4. Obtain K(i+1) document samples of the i+1th hop according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, wherein the probabilities of the paths of the K(i+1) i+1th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, and K(i+1) is a positive integer not greater than Mi. The K(i+1) document samples of the i+1th hop include the document sample of the i+1th hop corresponding to the positive sample and K(i+1)-1 i+1th hop negative samples; S5. Adjust the parameters of the first neural network according to the paths and the positive samples corresponding to the K(i+1)th document samples at the (i+1)th hop, to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

8. The method according to claim 7, characterized in that The adjusting the parameters of the first neural network according to the paths and the positive samples respectively corresponding to the K(i+1)th document samples at the (i+1)th hop includes: When the K(i+1)th i+1-th hop document samples reach a second preset stop condition, the parameters of the first neural network are adjusted according to the paths and the positive samples corresponding to the K(i+1)th i+1-th hop document samples respectively.

9. The method according to claim 8, characterized in that The method further comprises: If the K(i+1) document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

10. The method according to claim 8 or 9, characterized in that The second preset stop condition is that a stop symbol exists in at least one candidate sample among the K(i+1) document samples at the (i+1)th hop.

11. An information retrieval method, characterized in that: include: Receive the query input by the user; The query is input into a first neural network trained by the training method according to any one of claims 6 to 10 for processing to obtain a retrieval result corresponding to the query.

12. An information retrieval device, characterized in that: include: A retrieval module is used for S1, obtaining Mi candidate documents of the i+1th hop according to the retrieval text Query and Ki candidate documents of the i-th hop; wherein i, Ki, and Mi are all positive integers, and Ki is not greater than Mi; S2, obtaining the score of each candidate document in the Mi candidate documents of the i+1th hop, wherein the score is used to characterize the relevance of the candidate document to the Query; S3, for any candidate document Pjy(i+1) in the Mi candidate documents of the i+1th hop, obtaining the probability of the path corresponding to the candidate document Pjy(i+1) according to the score of the candidate document Pjy(i+1) and the probability of the path L, wherein the path L is a path starting from the candidate document of the first hop and ending at the candidate document in the Ki candidate documents of the i-th hop. Pji is the path to the end point, the candidate document Pji is the jth one among the Ki candidate documents of the i-th hop, the candidate document Pjy(i+1) is the yth one among the candidate documents of the i+1-th hop obtained based on the candidate document Pji of the i-th hop, and both j and y are positive integers; S4, according to the probabilities of the paths corresponding to the Mi candidate documents of the i+1-th hop, obtain K(i+1) candidate documents of the i+1-th hop, wherein the K(i+1) candidate documents of the i+1-th hop are candidate documents among the Mi candidate documents of the i+1-th hop, and the probabilities of the paths of the K(i+1) candidate documents of the i+1-th hop are all higher than the probabilities of the paths of the remaining candidate documents among the Mi candidate documents, and K(i+1) is a positive integer not greater than Mi; The determination module is used for S5 to obtain the search result corresponding to the query according to the K (i+1) candidate documents at the i+1th hop.

13. The device according to claim 12, characterized in that The determining module is configured to: When the K(i+1) candidate documents at the i+1th hop reach a first preset stop condition, a retrieval result corresponding to the Query is obtained according to the K(i+1) candidate documents at the i+1th hop.

14. The device according to claim 13, characterized in that The determining module is further configured to: Obtain a path corresponding to each candidate document in the K(i+1) candidate documents at the (i+1)th hop, where the path corresponding to each candidate document is a path starting from the candidate document at the first hop and ending at the candidate document at the (i+1)th hop; The path corresponding to each candidate document is used as the search result corresponding to the query; Alternatively, each candidate document is used as a retrieval result corresponding to the Query, and the path corresponding to each candidate document is used to interpret the retrieval result corresponding to the candidate document.

15. The device according to claim 13 or 14, characterized in that The determining module is further configured to: If the K(i+1) candidate documents at the i+1th hop do not meet the first preset stopping condition, set i=i+1 and repeat steps S1-S4 until the first preset stopping condition is met.

16. The device according to claim 13, characterized in that The first preset stop condition is that a stop symbol exists in at least one candidate document among the K(i+1) candidate documents at the (i+1)th hop.

17. A model training device, characterized in that: include: An acquisition module is configured to acquire a training sample, wherein the training sample includes a search text Query and a positive sample; the positive sample includes a path corresponding to the first document sample of the N+1th hop, wherein the path corresponding to the first document sample of the N+1th hop is a path starting from the document sample of the first hop and ending at the first document sample of the N+1th hop; The first document sample of the N+1th hop is the document sample corresponding to the path with a probability higher than that of the remaining K(N+1)-1 document samples in the N+1th hop; the probability of the path corresponding to any document sample Pjy(N+1) in the K(N+1) document samples of the N+1th hop is obtained based on the score of the document sample Pjy(N+1) and the probability of the path T, wherein the score is used to characterize the relevance of the document sample to the query, and the path T is a path starting from the document sample of the first hop and ending at the K(N+1)th ... The document sample PjN in the document sample of the jump is the path with the end point, the document sample PjN is the jth document sample in the KN document samples of the Nth jump, and the document sample Pjy(N+1) is the yth document sample obtained according to the document sample PjN; the training sample also includes K1-1 negative samples of the first jump, and the K1-1 negative samples of the first jump are the document samples in the K1 document samples of the jump except the document sample of the current jump corresponding to the positive sample; N is a positive integer, K1 and K(N+1) are both integers not less than 2, and j and y are both positive integers; A training module is used to train the first neural network according to the training samples to obtain the trained first neural network.

18. The device according to claim 17, characterized in that The training module is used to: S1. Obtain Mi candidate samples of the i-th hop according to the query, the document sample of the i-th hop corresponding to the positive sample, Ki-1 negative samples of the i-th hop, and the negative sample of the path of the i-th hop; Any path negative sample in the i-th hop path negative samples includes a path starting from any document sample in the K1 document samples of the first hop and ending at any negative sample in the Ki-1 negative samples of the i-th hop; i is an integer not less than 1, Ki is an integer not less than 2, and Mi is a positive integer; S2. Obtain a score for each candidate sample among the Mi candidate samples of the i-th hop; S3. Obtain the probability of the path corresponding to each candidate sample according to the score of each candidate sample in the Mi candidate samples of the i-th hop; S4. Obtain K(i+1) document samples of the i+1th hop according to the probabilities of the paths corresponding to the Mi i-th hop candidate samples, wherein the probabilities of the paths of the K(i+1) i+1th hop document samples are all higher than the probabilities of the paths of the remaining candidate samples in the Mi candidate samples, and K(i+1) is a positive integer not greater than Mi. The K(i+1) document samples of the i+1th hop include the document sample of the i+1th hop corresponding to the positive sample and K(i+1)-1 i+1th hop negative samples; S5. Adjust the parameters of the first neural network according to the paths and the positive samples corresponding to the K(i+1)th document samples at the (i+1)th hop, to obtain the trained first neural network; wherein, when i=1, the negative sample of the path of the first hop does not exist.

19. The device according to claim 18, characterized in that The training module is further used to: When the K(i+1)th i+1-th hop document samples reach a second preset stop condition, the parameters of the first neural network are adjusted according to the paths and the positive samples corresponding to the K(i+1)th i+1-th hop document samples respectively.

20. The device according to claim 19, characterized in that The training module is further used to: If the K(i+1) document samples at the i+1th hop do not meet the second preset stopping condition, set i=i+1 and repeat steps S1-S4 until the second preset stopping condition is met.

21. The device according to claim 19 or 20, characterized in that The second preset stop condition is that a stop symbol exists in at least one candidate sample among the K(i+1) document samples at the (i+1)th hop.

22. An information retrieval system, characterized in that: include: Receiving module, used to receive the query input by the user; A processing module is used to input the query into a first neural network trained by the training method according to any one of claims 6 to 10 for processing, so as to obtain a search result corresponding to the query.

23. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 11.

24. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

25. An information retrieval device, characterized in that: The method comprises a processor and a memory; wherein the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1 to 11.

Images