A sentence retrieval method and related device

Abstract

The embodiment of the present application discloses a sentence retrieval method and related devices, which extract a first searchable word sequence from a query statement, wherein the first searchable word sequence includes core words and non-core words, and the semantic weight of the core words is greater than the semantic weight of the non-core words. After using the core words to retrieve the original document set, a target document set is obtained. The relevance between the first searchable word sequence and each document in the target document set is calculated, and the retrieval result of the query statement is determined. Since the core words are words with higher semantic weights in the query statement, by using the core words to retrieve documents, the relevance of the target document and the query statement in the retrieval results can be guaranteed, while reducing the occupancy rate of read and write resources and improving the sentence retrieval speed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sentence retrieval method and related devices. Background Art

[0002] An information retrieval device (such as a search engine) performs a query based on a query statement (query) input by a user, and generates relevant query results and returns them to the user. After the information retrieval device obtains the query statement input by the user, it can segment the query statement to obtain an ordered sequence of queryable terms (term), and the ordered sequence includes one or more queryable terms. For another example, the information retrieval device converts text information in data such as web pages or entries into a document (document), and the document can also be regarded as an ordered sequence of queryable terms. The index of each queryable term in the document is stored in the form of an inverted index list (inverted list). The inverted index list is used to indicate in which documents a queryable term appears, and the position of a queryable term in the document.

[0003] In the prior art solution, first, the information retrieval device generates an ordered sequence of searchable words according to the query statement. Then, the information retrieval device takes the intersection of each searchable word generated by the query statement in the inverted index list of the document collection. When the inverted index list of a certain document is retrieved, which includes each searchable word of the query statement, the information retrieval device calculates the correlation between the document and the query statement, and quantitatively scores the correlation between the two. The higher the correlation score of the document, the more the document matches the query phrase.

[0004] Taking the query statement "QWER" as an example, through word segmentation processing, four searchable words "Q", "W", "E" and "R" can be obtained. When the information retrieval device searches the query statement, each searchable word in the ordered list of searchable words needs to traverse the inverted index list corresponding to the document collection. For the searchable words "Q" and "W", after semantic analysis, it can be obtained that the semantic weight (semantic weight) of the searchable words is low, and the searchable words contribute less when calculating the relevance score between the document and the query statement, while "E" and "R" have higher semantic weights. When performing statement retrieval, for example: the following two statements are queried, A "QWTY"; B "ERUI". After word segmentation of A / B, it can be obtained that the word segmentation of A is consistent with "Q" and "W" in the query statement, and the word segmentation of B is consistent with "E" and "R" in the query statement.

[0005] In the retrieval results obtained by the prior art, the relevance of A / B to the query statement is consistent, that is, there are two consistent searchable words. However, it is obvious that the relevance of B to the query statement is greater than the relevance of A to the query statement, and the user needs to obtain the search result of B. In summary, for these searchable words with lower semantic weights, the current information retrieval device still needs to search in the inverted index list, resulting in a waste of read and write resources and reducing the sentence retrieval speed. Summary of the invention

[0006] The embodiment of the present application provides a sentence retrieval method and related devices, which extract a first searchable word sequence from a query statement, wherein the first searchable word sequence includes core words and non-core words, and through hierarchical retrieval, the step of using non-core words to search the original document set is omitted. After the core words are used to search the original document set, a target document set is obtained. The correlation between the first searchable word sequence and each document in the target document set is calculated, and the retrieval result of the query statement is determined. Under the premise of ensuring the accuracy of the retrieval, the occupancy rate of the read and write resources is reduced, and the sentence retrieval speed is improved.

[0007] In the first aspect, the embodiment of the present application provides a sentence retrieval method, which may include: first, extracting a first queryable word sequence from a query statement, wherein each queryable word in the first queryable word sequence is arranged in the order in the query statement, and the first queryable word sequence includes at least one core word and at least one non-core word, and the core word is a queryable word in the first queryable word sequence whose semantic weight is greater than the non-core word. Taking the query statement "about the development of Sino-US trade" as an example, the first queryable word sequence can be: "about", "China and the United States", "trade" and "development", and the word order of the sequence is "about" - "China and the United States" - "trade" - "development", that is, consistent with the order in which these words are arranged in the query statement. Secondly, searching the original document set according to the core word to obtain a target document set, wherein the target document set includes at least one target document, and the target document includes the core word; thirdly, calculating the relevance of the first queryable word sequence and each document in the target document set, and determining the retrieval result of the query statement.

[0008] In the embodiment of the present application, through hierarchical retrieval, it is not necessary to use non-core words to retrieve the original document set. After using the core words to retrieve the original document set, the target document set is obtained. The relevance between the first searchable word sequence and each document in the target document set is calculated, and the retrieval result of the query statement is obtained. Since the core words are words with higher semantic weights in the query statement, by using the core words to retrieve documents, the relevance of the target document and the query statement in the retrieval results can be guaranteed, while reducing the occupancy rate of read and write resources and improving the statement retrieval speed.

[0009] In combination with the first aspect, in a possible implementation of the first aspect, the original document set is retrieved according to at least one core word to obtain a target document set, including: the original document set is retrieved according to a core word sequence composed of all the core words in the first searchable word sequence to obtain a target document set, the target document includes a core word sequence, and the core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement. A core word sequence is composed according to all the core words in the first searchable word sequence, and the core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement. By using the core word sequence to retrieve documents, the relevance of the target document and the query statement in the search results can be guaranteed, while reducing the occupancy rate of read and write resources and improving the statement retrieval speed.

[0010] In combination with the first aspect, in a possible implementation of the first aspect, searching the original document set according to the core word to obtain the target document set may include: searching the original document set according to the core word to obtain a first document set, the first document set including at least one first document, the first document including the core word; searching the first document set according to the core word sequence in the first queryable word sequence to obtain the target document set, the target document including the core word sequence in the first queryable word sequence. In an embodiment of the present application, after using the core word to search the original document set, a first document set is obtained. Then, the core word sequence in the first queryable word sequence is used to search the first document set to obtain the target document set. Through secondary retrieval, under the premise of ensuring a low read and write resource occupancy rate, since all the core words in the first queryable word sequence are not omitted for retrieval, the relevance of the target document and the query statement in the retrieval result is further improved.

[0011] In combination with the first aspect, in a possible implementation of the first aspect, the first queryable word sequence includes multiple non-core words, and the original document set is retrieved according to at least one core word to obtain a target document set, including: first, the original document set is retrieved according to the core word sequence composed of all the core words in the first queryable word sequence to obtain a second document set, the second document set includes at least one second document, and the second document includes the core word sequence in the first queryable word sequence; second, the second document set is retrieved according to multiple non-core words to obtain a target document set, wherein each target document in the target document set includes the core word sequence in the first queryable word sequence and at least one of the multiple non-core words, and in the target document set, the non-core words included in at least two target documents are not completely the same. Through the secondary search, first, all the core words in the first queryable word sequence are not omitted for retrieval, and secondly, one or more non-core words are used to search in the second document set obtained in the previous search, and the relevance of the target document and the query statement in the retrieval result is further improved under the premise of ensuring a low read and write resource occupancy rate.

[0012] In combination with the first aspect, in a possible implementation of the first aspect, searching the original document set according to the core word to obtain the target document set may include: searching the original document set according to the core word to obtain a first document set, the first document set includes at least one first document, and the first document includes the core word; searching the first document set according to the core word sequence in the first queryable word sequence to obtain a second document set, the second document set includes at least one second document, and the second document includes the core word sequence in the first queryable word sequence; searching the second document set according to the non-core word to obtain the target document set, the target document set includes one or more target documents. The target document includes the core word sequence in the first queryable word sequence, and one or more non-core words. In the embodiment of the present application, first, after using the core word to search the original document set, the first document set is obtained. Secondly, the first document set is retrieved using the core word sequence in the first queryable word sequence to obtain the second document set. Again, the second document set is retrieved according to the non-core word to obtain the target document set, wherein the target document includes the core word sequence and at least one non-core word in the first searchable word sequence. The target document retrieved according to the query statement includes the core word sequence and at least one non-core word in the first searchable word sequence. Through three searches, first, one or more core words are used to perform a preliminary screening of the original document set; second, all the core words in the first searchable word sequence are searched without missing out; and third, one or more non-core words are used to search in the second document set obtained in the previous search, thereby further improving the relevance of the target document and the query statement in the search results while ensuring a low read and write resource occupancy rate.

[0013] In combination with the first aspect, in a possible implementation of the first aspect, calculating the relevance between the first searchable word sequence and each document in the target document set may include: determining an approximate phrase based on a first vector and a second vector, wherein the first vector indicates the position of the core word sequence in the first searchable word sequence in the target document, for example, the position of the core word sequence "China-US-Trade" in the target document is respectively <13> and <15> , then the first vector is <13, 15>; the second vector indicates the position of the non-core word in the target document, for example, the position of the non-core word "development" in the target document is <16> , the location of "About" in the target document is <12> , then the second vector is <12, 16>. The approximate phrase includes the core word sequence and at least one non-core word in the first queryable word sequence, and the third vector indicates the position of the approximate phrase in the target document. For example, the first vector is <13, 15> and the second vector is <12, 16>, then the third vector is <12, 13, 15, 16>. The distance between the core word and the non-core word in the approximate phrase is less than the second threshold, and the second threshold can be 2, 5, 8 or 10; according to the third vector and the fourth vector, the first distance is determined, wherein the fourth vector indicates the position of the first queryable word sequence in the target document, and the position vector of the first approximate phrase is the distance between the third vector and the fourth vector from the first distance; according to the first distance, the relevance of the first queryable word sequence to each document in the target document set is determined. Use a "sliding window" to match whether the current document has a core word sequence in the first queryable word sequence.

[0014] It should be noted that if any non-core word in the approximate phrase does not appear near the sliding window, "dm" is taken as the component of the non-core word in the third vector, and "dm" is a fixed value, such as 3, 5 or 10. For example: the first query word sequence is "about", "China-US", "trade" and "development". When there is no non-core word "about" near the approximate phrase in the target document, the third vector of the approximate phrase is<dm,13,15,16> .

[0015] The approximate phrase is determined by the first vector and the second vector, and the first distance is determined according to the third vector and the fourth vector, and the relevance between the first searchable word sequence and each document in the target document set is determined according to the first distance. In terms of how to determine the relevance between the query statement and the target document, the implementation flexibility is improved.

[0016] In combination with the first aspect, in a possible implementation of the first aspect, determining the first distance according to the third vector and the fourth vector may include: determining a fifth vector using the third vector and an offset, wherein the offset indicates the distance between each queryable term in the first queryable term sequence and the first queryable term in the first queryable term sequence, and the fifth vector indicates the distance between the approximate phrase and the offset; determining the first distance according to the fourth vector and the fifth vector. By the above method, the influence of the offset is taken into account when calculating the first distance, thereby improving the accuracy and reliability of the retrieval results.

[0017] In combination with the first aspect, in a possible implementation of the first aspect, after extracting the first queryable word sequence from the query statement, it may also include: determining the phrase center of gravity according to the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase center of gravity is the queryable word whose cumulative sum is greater than the first threshold when the weights are accumulated in sequence according to the first queryable word sequence. Determining the first distance according to the fourth vector and the fifth vector may include: determining the first distance according to the fourth vector, the fifth vector and the weight, wherein the weight is used to perform a weighted correction on the distance between the fourth vector and the fifth vector. Through the above method, when calculating the first distance, the influence of the phrase center of gravity is taken into account, thereby improving the accuracy and reliability of the retrieval results.

[0018] In combination with the first aspect, in a possible implementation manner of the first aspect, the value of any component in the fourth vector is consistent with the size of the component corresponding to the centroid of the phrase in the fifth vector.

[0019] In combination with the first aspect, in a possible implementation of the first aspect, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the norm distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Minkowski distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight.

[0020] In combination with the first aspect, in a possible implementation manner of the first aspect, a value of any component in the fourth vector is consistent with a value of the smallest component in the fifth vector.

[0021] In combination with the first aspect, in a possible implementation of the first aspect, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, or the first distance is the norm distance between the fourth vector and the fifth vector, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or the first distance is the Minkowski distance between the fourth vector and the fifth vector.

[0022] In combination with the first aspect, in a possible implementation manner of the first aspect, taking the Chebyshev distance as an example, the first distance is calculated by the following method:

[0023] dis i =max j∈N (|p j -q j |);

[0024] Among them, dis i is the first distance corresponding to the i-th fifth vector, p j is the Jth component of the fifth vector, q j is the J-th component of the fourth vector, i is a positive integer, J is a positive integer, and N is a positive integer.

[0025] In combination with the first aspect, in a possible implementation manner of the first aspect, taking Manhattan distance as an example, the first distance is calculated by the following method:

[0026]

[0027] Among them, dis i is the first distance corresponding to the i-th fifth vector, p t is the tth component of the fourth vector, q t is the tth component in the fifth vector, i is a positive integer, w t is the weight of the tth query word in the first query word sequence, and Q is the total number of query words in the first query word sequence.

[0028] In combination with the first aspect, in a possible implementation manner of the first aspect, the following method calculates the score of the approximate phrase:

[0029]

[0030] Among them, pf i is the score of the ith approximate phrase, dis i is the first distance corresponding to the i-th approximate phrase, pos iis the i-th component in the third vector, i is a positive integer, k1 is the first parameter, α is the second parameter, and decay is the decay function. Specifically, k1 adjusts the influence of the scores of each approximate phrase in the current document on the calculation of the relevance between the first queryable word sequence and the target document; the slope α adjusts the pf when the first distance increases. i Decay (pos i ) is the decay function with the matching position as input, and the matching position is the approximate phrase P i The position in the target document.

[0031] In a second aspect, an embodiment of the present application provides an information retrieval device, which may include: an extraction module, configured to extract a first queryable word sequence from a query statement, wherein the first queryable word sequence indicates a plurality of words arranged in a word order of the query statement, the first queryable word sequence includes at least one core word, and one or more non-core words, the at least one core word being a queryable word in the first queryable word sequence having a semantic weight greater than that of the non-core word;

[0032] A retrieval module, configured to retrieve an original document set according to the at least one core word to obtain a target document set, wherein the target document set includes at least one target document, and the target document includes the at least one core word;

[0033] The calculation module is used to calculate the correlation between the first searchable word sequence and each document in the target document set, so as to obtain the retrieval result of the query statement from the target document set based on the correlation.

[0034] In conjunction with the second aspect, in a possible implementation of the second aspect, the information retrieval device may include: the retrieval module, specifically configured to retrieve the original document set according to the core word to obtain a first document set, the first document set including at least one first document, the first document including the core word;

[0035] The first document set is retrieved according to the core word sequence in the first searchable word sequence to obtain the target document set, wherein the target document includes the core word sequence in the first searchable word sequence.

[0036] In conjunction with the second aspect, in a possible implementation of the second aspect, the information retrieval device may include: the retrieval module, specifically configured to retrieve the original document set according to the core word to obtain a first document set, the first document set including at least one first document, the first document including the core word;

[0037] Retrieving the first document set according to the core word sequence in the first searchable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first searchable word sequence;

[0038] The retrieval module is specifically used to retrieve the second document set according to the non-core word to obtain the target document set, wherein the target document includes the core word sequence and at least one of the non-core words in the first searchable word sequence.

[0039] In combination with the second aspect, in a possible implementation of the second aspect, the information retrieval device may include: the retrieval module, which is specifically used to retrieve the original document set based on the core word sequence composed of all the core words in the first queryable word sequence to obtain the target document set, the target document includes the core word sequence, and the core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement.

[0040] In conjunction with the second aspect, in a possible implementation of the second aspect, the retrieval module is specifically configured to retrieve the original document set according to a core word sequence composed of all core words in the first searchable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first searchable word sequence;

[0041] The retrieval module is specifically used to retrieve the second document set based on the multiple non-core words to obtain the target document set, wherein each target document in the target document set includes the core word sequence in the first queryable word sequence and at least one of the multiple non-core words, and in the target document set, the non-core words included in at least two target documents are not completely the same.

[0042] In combination with the second aspect, in a possible implementation of the second aspect, the information retrieval device may include: the calculation module is specifically used to determine the approximate phrase according to the first vector and the second vector, wherein the first vector indicates the position of the core word sequence in the first searchable word sequence in the target document, the second vector indicates the position of the non-core word in the target document, the approximate phrase includes the core word sequence and at least one non-core word in the first searchable word sequence, and the third vector indicates the position of the approximate phrase in the target document;

[0043] Determine a first distance according to the third vector and the fourth vector, wherein the fourth vector indicates the position of the first searchable word sequence in the target document, and the first distance is the distance between the third vector and the fourth vector;

[0044] The relevance of the first queryable word sequence to each document in the target document set is determined according to the first distance and the scores of one or more first approximate phrases in the target document.

[0045] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0046] The calculation module is specifically configured to determine a fifth vector based on the third vector and an offset, wherein the offset indicates a distance between each searchable term in the first searchable term sequence and a first searchable term in the first searchable term sequence, and the fifth vector indicates a distance between the approximate phrase and the offset;

[0047] The calculation module is specifically used to determine the first distance according to the fourth vector and the fifth vector.

[0048] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0049] The retrieval module is also used to determine the phrase center of gravity based on the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase center of gravity is the queryable word whose cumulative sum is greater than a first threshold when the weights are accumulated in sequence according to the first queryable word sequence.

[0050] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0051] The calculation module is specifically used to determine the first distance according to the fourth vector, the fifth vector and the weight, wherein the weight is used to perform weighted correction on the distance between the fourth vector and the fifth vector.

[0052] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0053] The value of any component in the fourth vector is consistent with the magnitude of the component corresponding to the centroid of the phrase in the fifth vector.

[0054] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0055] The first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the norm distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight, or the first distance is the Minkowski distance between the fourth vector and the fifth vector, which is weighted and corrected by the weight.

[0056] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0057] The value of any component in the fourth vector is consistent with the value of the smallest component in the fifth vector.

[0058] In conjunction with the second aspect, in a possible implementation manner of the second aspect, the information retrieval device may include:

[0059] The first distance is the Chebyshev distance between the fourth vector and the fifth vector, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, or the first distance is the norm distance between the fourth vector and the fifth vector, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or the first distance is the Minkowski distance between the fourth vector and the fifth vector.

[0060] In a third aspect, an embodiment of the present application provides a computer device, which terminal device includes at least one processor, a memory, a communication port, a display, and computer execution instructions stored in the memory and executable on the processor. When the computer execution instructions are executed by the processor, the processor executes the first aspect or any possible implementation of the first aspect described above.

[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the above-mentioned first aspect or any possible implementation method of the first aspect.

[0062] In the fifth aspect, an embodiment of the present application provides a computer program product (or computer program) storing one or more computer execution instructions. When the computer execution instructions are executed by the processor, the processor executes the above-mentioned first aspect or any possible implementation method of the first aspect.

[0063] In a sixth aspect, the present application provides a chip system, which includes a processor for supporting a computer device to implement the functions involved in the above aspects. In a possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the computer device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0064] In the seventh aspect, an embodiment of the present application provides a terminal device, which includes at least one processor, a memory, a communication port, a display, and computer execution instructions stored in the memory and executable on the processor. When the computer execution instructions are executed by the processor, the processor executes the above-mentioned first aspect or any possible implementation method of the first aspect.

[0065] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0066] Extract the first searchable word sequence from the query statement, the first searchable word sequence includes core words and non-core words, and the core words are searchable words with a semantic weight greater than the non-core words. Through hierarchical retrieval, the step of using non-core words to retrieve the original document set is omitted. After using the core words to retrieve the original document set, the target document set is obtained. Calculate the correlation between the first searchable word sequence and each document in the target document set, and obtain the retrieval result of the query statement. Under the premise of ensuring the accuracy of the retrieval, the occupancy rate of the read and write resources is reduced, and the statement retrieval speed is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A schematic diagram of the framework of an information retrieval device proposed in an embodiment of the present application;

[0068] Figure 2 A schematic diagram of an embodiment of a sentence retrieval method proposed in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of another embodiment of a sentence retrieval method proposed in an embodiment of the present application;

[0070] Figure 4 A schematic diagram of a process flow of determining a first distance according to an embodiment of the present application;

[0071] Figure 5 A schematic diagram of another embodiment flow of determining a first distance proposed in an embodiment of the present application;

[0072] Figure 6a A schematic diagram of a method for matching searchable terms provided in an embodiment of the present application;

[0073] Figure 6bA schematic diagram of a method for matching searchable terms provided in an embodiment of the present application;

[0074] Figure 6c This is a schematic diagram of an application scenario in an embodiment of the present application;

[0075] Figure 7 A schematic diagram of a functional relationship between an approximate phrase score and a first distance proposed in an embodiment of the present application;

[0076] Figure 8 A schematic diagram of another functional relationship between an approximate phrase score and a first distance proposed in an embodiment of the present application;

[0077] Fig. 9 A schematic diagram of another functional relationship between an approximate phrase score and a first distance proposed in an embodiment of the present application;

[0078] Fig.10 A schematic diagram of an embodiment of an information retrieval device in an embodiment of the present application;

[0079] Fig.11 A schematic diagram of the hardware structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The embodiment of the present application provides a sentence retrieval method and related devices, extracting a first queryable word sequence (terms) from a query statement, the first queryable word sequence includes core words and non-core words, the core words are queryable words in the first queryable word sequence with a semantic weight greater than the non-core words, and through hierarchical retrieval, the step of using non-core words to retrieve the original document set is omitted. After the core words are used to retrieve the original document set, a target document set is obtained. The relevance of the first queryable word sequence and each document in the target document set is calculated, and the retrieval result of the query statement is determined. Since the core words are words with higher semantic weights in the query statement, by using the core words to retrieve documents, the relevance of the target document and the query statement in the retrieval result can be guaranteed, while reducing the occupancy rate of read and write resources and improving the sentence retrieval speed. The document can be a text document, such as a "txt" format document, a "doc" format document, the document can also be a link such as a web page, and the document can also be a data entry stored in the form of a database, which is not limited here.

[0081] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0082] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances. This is merely a way of distinguishing objects with the same properties when describing the embodiments of the present application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, so that a process, method, system, product, or device that includes a series of units is not necessarily limited to those units, but may include other units that are not explicitly listed or inherent to these processes, methods, products, or devices.

[0083] See also Figure 1 , Figure 1 A schematic diagram of the framework of an information retrieval device proposed in an embodiment of the present application. The information retrieval device is described using a search engine based on an elastic search (ES) cluster architecture as an example. It should be noted that the sentence retrieval method proposed in an embodiment of the present application can also be deployed in other information retrieval engines based on a distributed architecture or a traditional architecture, which is not limited here. The client and the search engine in the information retrieval device can be deployed in the cloud at the same time, or in a terminal device at the same time, or partly in the cloud and partly in the terminal device, which is not limited here.

[0084] Figure 1 In the invention, the information retrieval engine includes a client and a search engine deployed in the cloud (or server). The search engine includes one or more coordinating nodes (coordinator), as shown in the figure, coordinating node 1 to coordinating node N, N is a positive integer greater than 1; and one or more data nodes (datanode), as shown in the figure, data node 1 to data node M, M is a positive integer greater than 1; neural network. The above-mentioned coordinating nodes and data nodes are connected through a cluster network. A cluster network is a group of independent computers interconnected by a high-speed network. These computers form a group and are managed in the mode of a single system. When a client interacts with a cluster network, the cluster network can be regarded as an independent server.

[0085] For ease of understanding, the following is an explanation using coordination node 1 and data node 1 as examples. It should be noted that other nodes in the search engine are similar to the coordination node 1 or data node 1, and will not be repeated here. The client is used to receive the query statement input by the user. Optionally, the client establishes a connection with each node in the search engine through a representational state transfer (REST) ​​interface. In the search engine, the coordination node 1 can be divided into a query processing unit and a cluster management unit according to the functional division. Specifically, the query processing unit in the coordination node 1 processes the query statement from the client in a domain specific language (DSL) and generates a first queryable word sequence after parsing. The cluster management unit in the coordination node 1 is used to manage each coordination node and data node in the current search engine, specifically, to manage the metadata in the coordination node. The coordination node 1 sends the first queryable word sequence to the data node and neural network under its management through the cluster management unit. The data node can be data node 1, other data nodes, or multiple data nodes working together. Take the coordination node 1 managing data nodes 1 to data nodes M as an example for explanation. First, the neural network performs semantic analysis on the first searchable word sequence and determines at least one core word and at least one non-core word. The core word is a searchable word in the first searchable word sequence whose semantic weight is greater than that of the non-core word. It should be noted that when the first searchable word sequence includes only one searchable word, the searchable word is the core word; when all the searchable words in the first searchable word sequence are subjected to semantic analysis and the semantic weights of all the searchable words are greater than a certain threshold, all the searchable words are core words. The neural network can be deployed inside the search engine and connected to other nodes through a cluster network (such as Figure 1As shown); the neural network can also be independently deployed outside the search engine and establish a connection with the search engine through a REST interface or the like. Secondly, the neural network marks the core words in the first searchable word sequence and returns the result to the coordination node 1, and the coordination node 1 sends the first searchable word sequence (marked core words) to the data node 1 to the data node M. These data nodes perform retrieval according to the first searchable word sequence. Specifically, data node 1 is taken as an example for explanation. Data nodes can be divided into an index management unit and a query management unit according to their functions. The data node can be implemented using the library (Lucene) of the full-text retrieval engine. In the information retrieval device, data such as web pages or entries are abstracted as documents (document, doc). Each document can be regarded as an ordered sequence of searchable words. The documents proposed in the embodiments of the present application include but are not limited to text documents or web pages. The index relationship between the searchable words and the document is stored in the index management unit of the data node. The index relationship between the searchable words and the document is called document retrieval data. Optionally, the document retrieval data is implemented in the form of an inverted list. The inverted list is explained below:

[0086] An inverted list is a document indexing method that is used to store the mapping relationship between which documents a certain queryable word appears in during a full-text search. It is an ordered list of document numbers (doc id) arranged from small to large. For example, if the queryable word "A" appears in documents with numbers (id) 0, 2, and 5, then the inverted list of A can be represented as {0, 2, 5}. In order to find a phrase composed of multiple queryable words in an article, we also need to record the position of the queryable words in the document. In the embodiment of the present application, the position refers to the position information of a certain queryable word in the document. The positions of all queryable words in the document can form a vector, and each component in the vector indicates the position of the corresponding queryable word in the document. Assuming that A appears at positions 11, 25, and 46 in document 0, at position 54 in document 2, and at positions 14 and 39 in document 5, we have an inverted list {(0, <11, 25, 46>), (2, <54> ),(5,<14,39>)}. When an information retrieval device uses an inverted list, it usually uses an iterator to traverse the data on the inverted list. Specifically, the following abstract interface should be provided for an inverted list of a searchable word:

[0087] a) "nextDoc": Get the next doc id containing the searchable word. For example, if there is a reverse index of the searchable word A in the example above, calling nextDoc three times in sequence can get doc ids 0, 2, and 5;

[0088] b) "advance": try to fast-forward the inverted list to doc x, where x is the target document number of the input expected fast-forward. If there is no doc x in the inverted list, the state after advance is the first doc id beyond x. For example, for the inverted list {0,5,7}, the advance(5) operation will return 5, while the advance(6) operation will return 7, indicating that there is no 6 in the inverted list.

[0089] c) "nextPosition": Get the position of the next doc of the queryable word in the current doc. For example, in the above inverted index, if the inverted index iterator is at doc id = 5, calling nextPosition twice in sequence can get the position 14, 39.

[0090] After receiving the first queryable word sequence from the coordination node 1, the data node 1 searches for core words in the document index data managed by the index management unit in the query management unit. The data node 1 scores the relevance between the first queryable word sequence and the document according to a predefined relevance matching formula. Then the retrieval result is returned to the coordination node 1, and the retrieval result consists of the top K documents with the highest relevance scores, where K is a positive integer. The retrieval of the document by the data node 1 using the first queryable sequence can be a Boolean query, a phrase match (match_phrase) query, or a more complex tree-like logic query such as a query tree, which is not limited here. Optionally, the index management unit and the query management unit of the data node 1 can be implemented through the library of the full-text search engine (Lucene), and "Lucene" is an open source full-text search engine toolkit that provides the architecture of the full-text search engine.

[0091] The working mode of the coordination node N and the data node M is similar to the working mode of the above-mentioned data node 1 and the coordination node 1, which will not be repeated here.

[0092] The following is a detailed description of the steps of the embodiment of the present application in conjunction with the accompanying drawings. Figure 2 . Figure 2 A schematic diagram of an embodiment of a sentence retrieval method proposed in an embodiment of the present application. A sentence retrieval method proposed in an embodiment of the present application includes:

[0093] 201. Get the query statement.

[0094] In this embodiment, the information retrieval device obtains a query sentence input by a user. In this embodiment, for the sake of convenience, the query sentence "about the development of Sino-US trade" is taken as an example to introduce this solution.

[0095] 202. Extract a first searchable word sequence from the query statement.

[0096] In this embodiment, after the information retrieval device obtains the query statement, it performs word segmentation processing on the query statement to extract the first searchable word sequence. Taking English query statements as an example, English words are naturally separated by spaces, and it is easy to segment words according to spaces, but sometimes it is necessary to treat multiple words as one searchable word, such as some nouns such as "New York", which need to be treated as a searchable word. For Chinese query statements, since there are no spaces, a dictionary-based word segmentation algorithm can usually be used for word segmentation. It is also possible to use neural networks, using tools such as hidden Markov models (HMM), conditional random fields (CRF) algorithms, or deep learning algorithms for word segmentation, which are not limited here.

[0097] Taking the query sentence "about the development of Sino-US trade" as an example, the first searchable word sequence that can be extracted is: "about", "Sino-US", "trade" and "development".

[0098] 203. Determine core words and non-core words in the first searchable word sequence.

[0099] In this embodiment, after the information retrieval device determines the first searchable word sequence, it performs semantic analysis on the first searchable word sequence. The main purpose of semantic analysis is to study the semantics of individual words (i.e., word meaning); how the meaning of individual words is combined to form the meaning of a sentence (or a larger unit). From the perspective of analysis granularity, it can be divided into: word-level semantic analysis, sentence-level semantic analysis, and paragraph-level semantic analysis. Word-level semantic analysis includes: word disambiguation, word representation, mining of synonyms and hyponyms, etc. Sentence-level semantic analysis includes: semantic role labeling (SRL), a shallow semantic analysis technology that labels certain phrases in a sentence as arguments (semantic roles) of a given predicate, such as agent, patient, time and place, etc.; textual entailment refers to the fact that two text fragments have a directional relationship. When one text segment is considered to be authentic, the authenticity of another text segment can be inferred; sentence representation: Similar to word representation, sentence representation studies the use of a multi-dimensional vector to represent the meaning of a sentence; semantic dependency parsing (SDP) analyzes the semantic associations between the various language units of a sentence and presents the semantic associations in a dependency structure.

[0100] Specifically, after semantic analysis of the first searchable word sequence is performed by a neural network, it is determined which words in the first searchable word sequence are core words and which words are non-core words. The neural network can be selected from: a "state-of-the-art language model" based on a recurrent neural network (RNN), a long short term memory (LSTM) model, an "N-Gram language model" or a bag-of-words model. The following uses a term weighting algorithm as an example to illustrate how to determine the core words and non-core words in the first searchable word sequence. In the term weighting algorithm, a semantic weight needs to be calculated for each searchable word, and important searchable words are given a higher weight. For example, the term weighting calculation result of the query sentence "What product is most helpful for weight loss" may be: what 0.1, product 0.5, right 0.1, weight loss 0.8, help 0.3, maximum 0.2". Common term weighting algorithms include: "TF-IDF", "Okapi" or "MI" algorithms, which are not limited here.

[0101] For example, taking the query sentence "about the development of Sino-US trade" as an example, the semantic analysis results are shown in Table 1.

[0102] Searchable words "about" "China and the U.S." "trading" "develop" Semantic weight 0.1 0.3 0.4 0.2 Core words - · · - Non-core words · - - ·

[0103] Table 1

[0104] 204. Retrieve the original document set according to the core word to obtain a first document set, where the first document includes the core word.

[0105] In this embodiment, an original document set is retrieved according to the core words obtained in step 203, and the original document set includes one or more original documents. A first document set is retrieved from the original document set, and the first document set includes one or more first documents, and the first document includes one or more core words.

[0106] Specifically, the original document set includes document 1 (doc1) to document 11 (doc11) as an example for description. The document sequence number of each document in the original document set and the index relationship of the searchable words in each document are shown in Table 2:

[0107]

[0108]

[0109] Table 2

[0110] In Table 2, taking "About:12" as an example, "About:12" means that the position of the searchable word "About" in document 1 is 12.

[0111] In step 203, it is determined that the query statement includes the core words "China-US" and "trade", and the non-core words "about" and "development". Then, "China-US" and "trade" are used to retrieve the original document set shown in Table 2 above. The first document set {2, 4, 5, 6, 7, 8, 9, 10} including the core words "China-US" and / or "trade" is obtained. The first document includes the core words, for example, document 2 includes "China-US" and "trade"; document 7 includes "trade". In an optional implementation, the abstract interface "nextDoc" can be used to retrieve the original document set.

[0112] 205. Retrieve the first document set according to the core word sequence in the first searchable word sequence to obtain a second document set.

[0113] In this embodiment, after obtaining the first document set, the information retrieval device retrieves the first document set according to the core word sequence in the first optional word sequence to obtain a second document set, and the second document set includes one or more second documents. The second document includes the core word sequence in the first searchable word sequence. The core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement.

[0114] Specifically, taking the original document set shown in Table 2 as an example, according to the core word sequence "China-US" and "trade" in the first searchable word sequence, the first document set {2, 4, 5, 6, 7, 8, 9, 10} is retrieved. A second document set including the core words "China-US" and "trade" is obtained, and the second document set is {2, 5, 8, 9, 10}. The second document set is shown in Table 3.

[0115]

[0116]

[0117] Table 3

[0118] 206. When the query statement further includes non-core words, a second document set is retrieved according to the non-core words to obtain a target document set.

[0119] In this embodiment, when the query statement also includes non-core words, the second document set is retrieved according to the non-core words to obtain a target document set, which includes one or more target documents. The target document includes the core word sequence in the first searchable word sequence and one or more non-core words.

[0120] Specifically, taking the second document set {2, 5, 8, 9, 10} as an example, non-core words are used to retrieve the second document set to obtain the target document set {2, 5, 8, 9, 10}. For example: Document 2 includes the core word sequence "China-US" and "trade" in the first queryable word sequence, and a non-core word "about". Document 5 includes the core word sequence "China-US" and "trade" in the first queryable word sequence, and two non-core words "about" and "development". Document 10 includes the core word sequence "China-US" and "trade" in the first queryable word sequence, and a non-core word "development".

[0121] It should be noted that when the first searchable word sequence does not include non-core words, the second document set obtained in step 205 is set as the target document set.

[0122] 207. Determine an approximate phrase according to the first vector and the second vector.

[0123] In this embodiment, after obtaining the target document set, the information retrieval device determines a proximity phrase based on the first vector and the second vector. In this embodiment of the present application, a vector is used to represent the position of a queryable word in a document. Each component in the vector corresponds to the position of a queryable word in the document. In step 207, the first vector indicates the position of the core word sequence in the first queryable word sequence in the target document. The second vector indicates the position of the non-core word in the first queryable word sequence in the target document. The proximity phrase includes the core word sequence and at least one non-core word in the first queryable word sequence. The distance between the core word and the non-core word in the proximity phrase is less than a second threshold value, and the second threshold value is determined according to the actual needs of the user or the length of the "sliding window". For example, the optional second threshold value is 5.

[0124] In an optional implementation, a "sliding window" is used to match whether the current document contains the core word sequence in the first queryable word sequence. Figure 6a , Figure 6a A schematic diagram of a method for matching searchable words provided in an embodiment of the present application. Use a sliding window to match whether there are other core words near each core word in the document, and these core words together constitute the core word sequence in the first searchable word sequence. Figure 6aFor example, the size of the sliding window is 5, that is, the sliding window can retrieve 5 consecutive searchable words at the same time. The sliding window traverses document 8 starting from the searchable word at position 0 in document 8. When traversing to "China and the United States: 13", according to the vector "13" and the vector "15", it is determined that "China and the United States" and "trade" exist in the sliding window "Pc1" at the same time, that is, the first vector is <13, 15>; similarly, the first vector <38, 39> is determined in the sliding window "Pc2"; the first vector <50, 53> is determined in the sliding window "Pc3".

[0125] Then, according to the position of the non-core words "about" and "development" in document 8, that is, the second vector, the approximate phrase is determined. For example, "China-US: 13" and "trade: 15" determined in the sliding window "Pc1" have multiple non-core words nearby (about: 11, about: 12, development: 16). At this time, the non-core words closest to the core words are filtered, and the approximate phrases are determined by the core word sequence and the non-core words, and the third vector is obtained. The third vector indicates the position of the approximate phrase in the target document. The third vector corresponding to "China-US: 13" and "trade: 15" is <12,13,15,16>, where 12 corresponds to "about", 13 corresponds to "China-US", 15 corresponds to "trade", and 16 corresponds to "development".

[0126] It should be noted that if any non-core word does not appear near the sliding window, "dm" is taken as the component of the non-core word in the third vector, and the "dm" is a fixed value, such as 5 or 10.

[0127] By analogy, we can obtain the third vectors corresponding to the following three approximate phrases in document 8: <12,13,15,16>,<dm,38,39,41> and <52,50,53,dm>.

[0128] 208. Determine a first distance.

[0129] In this embodiment, there are two solutions for determining the first distance:

[0130] (1) Determine a fifth vector based on the third vector and the offset. Determine a first distance based on the fourth vector, the fifth vector and the weight. The third vector is the position of the approximate phrase in the target document, and the fourth vector indicates the position of the first queryable word sequence in the target document. The fifth vector indicates the distance between the approximate phrase and the offset.

[0131] Optionally, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, or the first distance is the norm distance between the fourth vector and the fifth vector, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or the first distance is the Minkowski distance between the fourth vector and the fifth vector.

[0132] For example, taking the Chebyshev distance as an example, the first distance is calculated by the following method:

[0133] dis i =max j∈N (|p j -q j |);

[0134] Among them, dis i is the first distance corresponding to the i-th fifth vector, p j is the Jth component of the fifth vector, q j is the J-th component of the fourth vector, i is a positive integer, J is a positive integer, and N is a positive integer.

[0135] (2) Determine a fifth vector based on the third vector and the offset. Determine a first distance based on the fourth vector, the fifth vector and the weight, wherein the weight is used to perform weighted correction on the distance between the fourth vector and the fifth vector. The third vector is the position of the approximate phrase in the target document, and the fourth vector indicates the position of the first queryable word sequence in the target document.

[0136] Optionally, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the norm distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Minkowski distance between the fourth vector and the fifth vector, which is corrected by weighting.

[0137] For example, taking Manhattan distance as an example, the first distance is calculated by the following method:

[0138]

[0139] Among them, dis i is the first distance corresponding to the i-th fifth vector, p t is the tth component of the fourth vector, q t is the tth component in the fifth vector, i is a positive integer, w t is the weight of the tth query word in the first query word sequence, and Q is the total number of query words in the first query word sequence.

[0140] Since the fourth vector indicates the position of the first queryable word sequence in the target document, and the first queryable word sequence does not actually appear in the target document, the value of each component in the fourth vector is determined by the information retrieval device. Optionally, the value of any component in the fourth vector is consistent with the size of the component corresponding to the centroid of the phrase in the fifth vector. Optionally, the value of any component in the fourth vector is consistent with the value of the smallest component in the third vector.

[0141] The offset indicates the distance between each queryable word in the first queryable word sequence and the first queryable word in the first queryable word sequence. It should be noted that in the embodiment of the present application, the distance between queryable words refers to the absolute value of the difference between the components corresponding to the queryable words. Taking document 8 in Table 2 as an example, the core words are "China and the United States" and "trade". According to the first queryable word sequence "about", "China and the United States", "trade" and "development", the offset of each queryable word can be determined, as shown in Table 4:

[0142]

[0143] Table 4

[0144] by Figure 6a For example, document 8 includes the third vectors corresponding to the following three approximate phrases: <12,13,15,16>,<dm,38,39,41> and <52,50,53,dm>. Calculate the first distance corresponding to each approximate phrase respectively.

[0145] For a detailed solution of step 208, please refer to Figure 4 and Figure 5 The corresponding embodiments.

[0146] 209. Determine a score of the approximate phrase according to the first distance.

[0147] In this embodiment, the information retrieval device determines the score of the approximate phrase corresponding to the first distance according to the first distance.

[0148] In an optional implementation, the first distance itself corresponding to each approximate phrase in the target document is used as the score of the corresponding approximate phrase.

[0149] In another optional implementation, the score of the approximate phrase is calculated by the following method:

[0150]

[0151] Among them, pf i is the score of the ith approximate phrase, dis i is the first distance corresponding to the i-th approximate phrase, pos iis the i-th component in the third vector, i is a positive integer, k1 is the first parameter, α is the second parameter, and decay is the decay function. Specifically, k1 adjusts the influence of the scores of each approximate phrase in the current document on the calculation of the relevance between the first queryable word sequence and the target document; the slope α adjusts the pf when the first distance increases. i Decay (pos i ) is the decay function with the matching position as input, and the matching position is the approximate phrase P i The position in the target document. Generally speaking, the earlier the matching position is, the more relevant the target document is to the first query word sequence. i The later the position is, the greater the penalty for correlation is. The decay function can be selected from a variety of decay functions such as no decay, Gaussian decay, exponential decay, linear decay, etc. according to different application scenarios, and is not limited here.

[0152] For easier understanding, see Figure 7 , Figure 8 and Fig. 9 , Figure 7-9 A schematic diagram of a functional relationship between an approximate phrase score and a first distance proposed in an embodiment of the present application. Figure 7 In the equation, the first parameter k1=1, and the second parameter α=2. Figure 8 , the first parameter k1=4, and the second parameter α=1. Fig. 9 In the above, the first parameter k1=4, the second parameter α=0.05. Figure 7-Figure 9 It can be seen that when α is large, the score pf of the approximate phrase i It tends to fully match the searchable words, that is, when the first distance is small, pf i The decrease is very fast, while when the first distance is large, the scores are close and the decrease is slow. The information retrieval device determines the value of the first parameter, the value of the second parameter and the type of the attenuation function according to different actual needs.

[0153] by Figure 6a For example, document 8 includes the third vectors corresponding to the following three approximate phrases: <12,13,15,16>,<dm,38,39,41> and <52,50,53,dm>. According to the first distance corresponding to each approximate phrase, the score of the approximate phrase is determined respectively.

[0154] 210. Determine the relevance of the first searchable term to each document in the target document set based on the scores of one or more similar phrases in the target document, and determine the retrieval result.

[0155] In this embodiment, the information retrieval device determines the relevance between the first searchable word and each document in the target document set according to the scores of one or more approximate phrases in the target document, and determines the search result. For example, when there is only one approximate phrase in the target document, the relevance between the current target document and the first searchable word sequence is determined according to the score of the approximate phrase. Then, the search result is determined according to the relevance between each target document in the current target document set and the first searchable word sequence. Optionally, the search result can be the first target document with the strongest relevance, and the search result can also be the ordering of each document in the target document set according to the relevance, which is not limited here.

[0156] In an optional implementation, the information retrieval device may process the scores of one or more approximate phrases in the target document. Then, based on the processing result, the relevance of the target document to the first queryable word sequence is determined. Optionally, the scores of the approximate phrases are processed using the BM25 algorithm, and the processing result of the BM25 algorithm is the relevance of the current target document to the first queryable word sequence. BM25 is an algorithm used to evaluate the relevance between a query statement and a document, and it is an algorithm proposed based on a probabilistic retrieval model. Optionally, the scores of the approximate phrases are processed using a term frequency-inverse document frequency (TF-IDF) algorithm.

[0157] In another optional implementation, the information retrieval device can directly determine the relevance of the target document to the first searchable word sequence based on the scores of the approximate phrases in the current target document. For example, the scores of one or more approximate phrases are summed. The retrieval result of the query statement is determined by comparing the sum of the scores of different target documents in the target document set (the sum of the scores of the approximate phrases in the current target document).

[0158] by Figure 6a For example, document 8 includes the third vectors corresponding to the following three approximate phrases: <12,13,15,16>,<dm,38,39,41> Based on the scores of each approximate phrase, the relevance of document 8 to the first queryable word sequence is calculated.

[0159] In the embodiment of the present application, after the information retrieval device obtains the query statement, first, the first queryable word sequence is extracted from the query statement. Secondly, the core words and non-core words in the first queryable word sequence are determined. Thirdly, the original document set is retrieved according to the core words to obtain the first document set, and the first document includes the core words. When the query statement also includes non-core words, the second document set is retrieved according to the non-core words to obtain the target document set. In the process of retrieving the second document set according to the non-core words, the approximate phrase is determined according to the first vector and the second vector. The first vector indicates the position of the core word sequence in the first queryable word sequence in the target document. The second vector indicates the position of the non-core word in the first queryable word sequence in the target document. After determining the approximate phrase, the first distance is determined according to the third vector, the fourth vector and the fifth vector, the third vector is the position of the approximate phrase in the target document, the fourth vector indicates the position of the first queryable word sequence in the target document, and the fifth vector indicates the distance between the approximate phrase and the offset. The offset indicates the distance between each queryable word in the first queryable word sequence and the first queryable word in the first queryable word sequence. Thirdly, the retrieval result of the query statement is determined according to the first distance. Through hierarchical retrieval, a large number of read and write operations involving non-core words are saved, the occupancy rate of read and write resources is reduced, and the sentence retrieval speed is improved. At the same time, compared with the existing technical solutions, the retrieved target documents can include approximate phrases that lack some non-core words.

[0160] exist Figure 2 Based on the examples shown, please refer to Figure 3 , Figure 3 This is a schematic diagram of another embodiment of a sentence retrieval method proposed in an embodiment of the present application. When the query sentence only includes core words, the sentence retrieval method proposed in an embodiment of the present application includes:

[0161] 301. Get the query statement.

[0162] In this embodiment, the information retrieval device obtains a query sentence input by a user. In this embodiment, for the sake of convenience, the query sentence "Sino-US trade" is used as an example to introduce this solution.

[0163] 302. Extract a first searchable word sequence from the query statement.

[0164] In this embodiment, the specific implementation method is similar to the aforementioned step 202, and will not be repeated here. The first searchable word sequence extracted by the query statement is "China and the United States" and "trade".

[0165] 303. Determine a core word in the first searchable word sequence.

[0166] In this embodiment, the specific implementation method is similar to the aforementioned step 203, which will not be repeated here. The first searchable word sequence extracted by the query statement is "China-US" and "trade", and the determined words are "China-US" and "trade".

[0167] 304. Retrieve the original document set according to the core word to obtain a first document set, where the first document includes the core word.

[0168] In this embodiment, the specific implementation method is similar to the aforementioned step 204 and will not be repeated here.

[0169] 305. When the query statement does not contain non-core words, retrieve the first document set according to the core word sequence in the first searchable word sequence to obtain a second document set, and make the second document set the target document set.

[0170] In this embodiment, the core words in the first searchable word sequence are "China-US" and "trade", and the first document set is retrieved according to the core word sequence in the first searchable word sequence to obtain the second document set. Since there are no non-core words in the first searchable word sequence, the second document set is made the target document set. The specific implementation method is similar to the aforementioned step 205, and will not be repeated here.

[0171] 306. Determine a first distance based on the first vector and the fourth vector, where the first vector indicates a position of the core phrase in the target document.

[0172] In this embodiment, since there is no non-core word in the first queryable word sequence, the core phrase indicated by the first vector is located in the target document, and the core phrase is composed of the core word sequence in the first queryable word sequence. The specific implementation method is similar to the aforementioned steps 206-208, and will not be repeated here.

[0173] 307. Determine a score of the core phrase according to the first distance.

[0174] In this embodiment, the specific implementation method is similar to the aforementioned step 209 and will not be repeated here.

[0175] 308. Determine the relevance of the first searchable term to each document in the target document set based on the scores of one or more core phrases in the target document, and determine the retrieval result.

[0176] In this embodiment, the specific implementation method is similar to the aforementioned step 210 and will not be repeated here.

[0177] In the embodiment of the present application, when there are no non-core words in the first searchable word sequence extracted from the query sentence, the sentence retrieval method proposed in the present application can also be used. The original document set is initially searched by the core words, and then the core word sequence (core phrase) is used to perform a secondary search on the documents containing the core words. Through hierarchical retrieval, a large amount of computing resources occupied by calculating the distance is saved, the sentence retrieval speed is improved, and the performance of the information retrieval device is improved.

[0178] exist Figure 2-Figure 3 Based on the embodiments shown, the following describes how to determine the first distance in combination with the accompanying drawings. Figure 4 , Figure 4 A schematic diagram of a flow chart of an embodiment of determining a first distance proposed in an embodiment of the present application.

[0179] 401. Determine a fifth vector according to the third vector and the offset.

[0180] In this embodiment, the information retrieval device determines the fifth vector according to the third vector and the offset.

[0181] In an optional implementation, the fifth vector is calculated by the following method:

[0182] phrase_position(J)=position(J)–offset(J);

[0183] Among them, phrase_position(J) is the Jth component in the fifth vector, position(J) is the Jth component in the third vector, offset(J) is the offset of the Jth searchable word in the first searchable word sequence, and J is a positive integer.

[0184] For example, according to the third vector <12, 13, 15, 16> and the offset (0, 1, 2, 3), the fifth vector is determined to be <12, 12, 13, 13>.

[0185] 402. Determine a first distance according to the fourth vector and the fifth vector.

[0186] In this embodiment, the first distance is determined according to the fourth vector and the fifth vector. The value of any component in the fourth vector is consistent with the value of the smallest component in the fifth vector. For example, according to the third vector <12,13,15,dm>, the fourth vector is determined to be <12,12,12,12>. The first distance is determined according to the fourth vector and the fifth vector.

[0187] Specifically, the distance is determined according to the fourth vector and the fifth vector, and a variety of vector distance calculation methods can be used, including but not limited to: Manhattan distance, Chebyshev distance, norm distance, Mahalanobis distance, Minkowski distance or Euclidean distance.

[0188] Taking Chebyshev distance as an example, the first distance is calculated by the following method:

[0189] dis i =max j∈N (|p j -q j |);

[0190] Among them, dis i is the first distance corresponding to the i-th fifth vector, p j is the Jth component of the fifth vector, q j is the J-th component of the fourth vector, i is a positive integer, J is a positive integer, and N is a positive integer.

[0191] In an embodiment of the present application, a method for calculating a first distance is provided, which improves the implementation flexibility of the present solution.

[0192] For details, please refer to Figure 5 , Figure 5 A schematic diagram of a flow chart of another embodiment of determining a first distance proposed in an embodiment of the present application.

[0193] 501. Determine a phrase center of gravity according to a first searchable word sequence and a weight of each searchable word in the first searchable word sequence.

[0194] In this embodiment, the phrase center of gravity is determined based on the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase center of gravity is the queryable word whose weight is accumulated in sequence according to the first queryable word sequence, and the cumulative sum is greater than the first threshold.

[0195] Specifically, taking the query sentence "about the development of Sino-US trade" as an example, the corresponding relationship between the query words and the weights in the first query word sequence is shown in Table 5.

[0196] Searchable words "about" "China and the U.S." "trading" "develop" Weight 0.1 0.3 0.4 0.2

[0197] Table 5

[0198] According to the order in which the searchable words appear in the first searchable word sequence, "about", "China-US", "trade" and "development" are accumulated in sequence. When the accumulation reaches "trade", the cumulative sum reaches 0.8. Let the first threshold be 0.7, then "trade" is determined as the phrase center. It should be noted that the first threshold is determined according to the weight of the searchable words in the query statement, and can also be determined according to the actual needs of the user, which is not limited here.

[0199] 502. Determine a fifth vector according to the third vector and the offset.

[0200] In this embodiment, the specific implementation method is similar to step 401 and will not be repeated here.

[0201] 503. Determine a first phrase distance according to the fourth vector, the fifth vector, and the weight.

[0202] In this embodiment, the first phrase distance is calculated based on the fourth vector, the fifth vector and the phrase centroid. The value of any component in the fourth vector is consistent with the value of the component corresponding to the phrase centroid in the fifth vector.

[0203] Optionally, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the norm distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is corrected by weighting, or the first distance is the Minkowski distance between the fourth vector and the fifth vector, which is corrected by weighting.

[0204] For example, taking Manhattan distance as an example, the first distance is calculated by the following method:

[0205]

[0206] Among them, dis i is the first distance corresponding to the i-th fifth vector, p t is the tth component of the fourth vector, q t is the tth component in the fifth vector, i is a positive integer, w t is the weight of the tth query word in the first query word sequence, and Q is the total number of query words in the first query word sequence.

[0207] For details, please refer to Figure 6b Taking the sliding window "Pc1" as an example, according to the third vector <12,13,15,16> and the offset (0,1,2,3), the fifth vector is determined to be <12,12,13,13>.

[0208] When the third vector is <12,13,15,16>, the offset is (0,1,2,3), the fifth vector is determined to be <12,12,13,13>. The fourth vector is <13,15,15,15>. Substituting into the above formula, the first distance dis1 is calculated to be 0.4:

[0209] dis1=0.1·|12-13|+0.3·|12-13|+0.4·|13-13|+0.2·|13-13|=0.4

[0210] In an embodiment of the present application, a method for calculating a first distance is provided, which improves the implementation flexibility of the present solution.

[0211] Next, an application scenario of the sentence retrieval method proposed in the embodiment of the present application is introduced in conjunction with the accompanying drawings. Figure 6c , Figure 6c This is a schematic diagram of an application scenario in an embodiment of the present application. When the sentence retrieval method proposed in an embodiment of the present application is applied to a terminal device, it includes:

[0212] 601. Obtain a query statement, and extract a first searchable word sequence from the query statement.

[0213] In step 601, the user may input a query statement to the terminal device through an input device such as a keyboard, a touch screen or a mouse. The user may also input a voice command to the terminal device through a microphone, and the terminal device recognizes the voice command and generates a query statement. The user may also input a picture to be queried to the terminal device through an input device such as a camera, and the terminal device recognizes the picture and extracts the query statement. The user may also input a query statement to the terminal device through other methods, which are not limited here. The terminal device may also generate a query statement by recognizing an internal media stream, such as an audio file or a video file.

[0214] After the terminal device obtains the query statement, it extracts the first searchable word sequence from the query statement. The specific method of extracting the first searchable word sequence is similar to the aforementioned step 202 and will not be repeated here.

[0215] 602. Determine core words and non-core words in the first searchable word sequence.

[0216] In step 602, the terminal device performs word segmentation on the query statement to obtain a first searchable word sequence, which includes one or more searchable words. The terminal device then uses a neural network to perform semantic analysis on the first searchable word sequence to determine core words and non-core words in the first searchable word sequence.

[0217] The neural network can be run in the processor inside the terminal device, and the processor includes: application processor (AP), modem processor, graphics processor (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. When the neural network runs inside the terminal device, the terminal device performs semantic analysis on the first searchable word sequence through the neural network running on the internal processor. Then determine the core words and non-core words in the first searchable word sequence.

[0218] The neural network can also be deployed in a cloud server, and the terminal device sends the first searchable word sequence to the neural network deployed in the cloud through a communication link, and the neural network performs semantic analysis on the first searchable word sequence, and then determines the core words and non-core words in the first searchable word sequence.

[0219] The specific implementation method is similar to the aforementioned step 203 and will not be repeated here.

[0220] 603. Retrieve the original document set according to the core word sequence and at least one non-core word in the first searchable word sequence to obtain a second document set.

[0221] In step 603, the terminal device retrieves the original document set according to the core word sequence, and the original document set may be a set of various web pages in a server that establishes a communication connection with the terminal device, or a set of text documents stored in the terminal device, which is not limited here. The specific method of using the core word sequence and at least one non-core word for retrieval is similar to the aforementioned steps 204-206, and will not be repeated here.

[0222] 604. Determine an approximate phrase according to the first vector and the second vector.

[0223] Step 604 is similar to the aforementioned step 207 and will not be described again here.

[0224] 605. Determine a first distance.

[0225] Step 605 is similar to the aforementioned step 208 and will not be described again here.

[0226] 606. Determine a score of the approximate phrase according to the first distance.

[0227] Step 606 is similar to the aforementioned step 209 and will not be described again here.

[0228] 607. Determine the relevance of the first searchable term to each document in the target document set according to the scores of one or more similar phrases in the target document, and determine the retrieval result.

[0229] In step 607, after the terminal device determines the relevance between the first searchable term and each document in the target document set, the search results are displayed to the user through the screen. For example, if the search statement is "about the development of Sino-US trade", the search results are sorted according to the relevance, and document 1 "···about the development of Sino-US trade··", document 2 "···about Sino-US trade···development", and document 3 "···Sino-US···trade···".

[0230] In the embodiment of the present application, through hierarchical retrieval, first, all the core words in the first queryable word sequence are not omitted for retrieval, and secondly, one or more non-core words are used to search in the second document set obtained in the previous retrieval, and the relevance of the target document and the query statement in the retrieval result is further improved under the premise of ensuring a low read and write resource occupancy rate. It should be noted that the terminal device can include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem. The terminal device can be a mobile station (mobile station, MS), a subscriber module (subscriber unit), a cellular phone (cellular phone), a smart phone (smart phone), a wireless data card, a personal digital assistant (personal digital assistant, referred to as: PDA) computer, a tablet computer, a wireless modem (modem), a handheld device (handset), a laptop computer (laptopcomputer), a machine type communication (machine type communication, MTC) terminal device, etc.

[0231] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the above information retrieval device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0232] The embodiment of the present application can divide the functional modules of the information retrieval device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one retrieval module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0233] The information retrieval device in this application is described in detail below. Fig.10 , Fig.10 Schematic diagram of an embodiment of an information retrieval device in an embodiment of the present application. The information retrieval device 1000 includes:

[0234] Extraction module 1001, used to extract a first queryable word sequence from a query statement, wherein the first queryable word sequence indicates a plurality of words arranged according to the word order of the query statement, the first queryable word sequence includes at least one core word and one or more non-core words, the at least one core word is a queryable word in the first queryable word sequence whose semantic weight is greater than that of the non-core word;

[0235] A retrieval module 1002 is configured to retrieve an original document set according to the at least one core word to obtain a target document set, wherein the target document set includes at least one target document, and the target document includes the at least one core word;

[0236] The calculation module 1003 is used to calculate the correlation between the first searchable word sequence and each document in the target document set, so as to obtain the retrieval result of the query statement from the target document set based on the correlation.

[0237] In some embodiments of the present application,

[0238] The retrieval module 1002 is specifically used to retrieve the original document set according to the core word to obtain a first document set, the first document set includes at least one first document, and the first document includes the core word;

[0239] The retrieval module 1002 is specifically configured to retrieve a first document set according to a core word sequence in a first searchable word sequence to obtain a target document set, wherein the target document includes the core word sequence in the first searchable word sequence.

[0240] In some embodiments of the present application,

[0241] The retrieval module 1002 is specifically used to retrieve the original document set according to the core word sequence composed of all the core words in the first searchable word sequence to obtain the target document set, and the target document includes the core word sequence, and the core words in the core word sequence are arranged according to the order in which the core words are arranged in the query statement.

[0242] In some embodiments of the present application,

[0243] The retrieval module 1002 is specifically configured to retrieve the original document set according to a core word sequence composed of all core words in the first searchable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first searchable word sequence;

[0244] The retrieval module 1002 is specifically used to retrieve the second document set based on the multiple non-core words to obtain the target document set, wherein each target document in the target document set includes the core word sequence in the first queryable word sequence and at least one of the multiple non-core words, and in the target document set, the non-core words included in at least two target documents are not completely the same.

[0245] In some embodiments of the present application,

[0246] The retrieval module 1002 is specifically used to retrieve the original document set according to the core word to obtain a first document set, the first document set includes at least one first document, and the first document includes the core word;

[0247] The retrieval module 1002 is specifically configured to retrieve a first document set according to a core word sequence in a first searchable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first searchable word sequence;

[0248] The retrieval module 1002 is specifically configured to retrieve the second document set according to the non-core words to obtain a target document set, wherein the target document includes the core word sequence and at least one non-core word in the first searchable word sequence.

[0249] In some embodiments of the present application,

[0250] The calculation module 1003 is specifically used to determine an approximate phrase according to the first vector and the second vector, wherein the first vector indicates the position of the core word sequence in the first searchable word sequence in the target document, the second vector indicates the position of the non-core word in the target document, the approximate phrase includes the core word sequence and at least one non-core word in the first searchable word sequence, and the third vector indicates the position of the approximate phrase in the target document;

[0251] The calculation module 1003 is specifically configured to determine a first distance according to the third vector and the fourth vector, wherein the fourth vector indicates a position of the first searchable word sequence in the target document, and the first distance is a distance between the third vector and the fourth vector;

[0252] The calculation module 1003 is specifically configured to determine the relevance between the first searchable word sequence and each document in the target document set according to the first distance.

[0253] In some embodiments of the present application,

[0254] A calculation module 1003 is specifically configured to determine a fifth vector according to the third vector and an offset, wherein the offset indicates a distance between each queryable term in the first queryable term sequence and a first queryable term in the first queryable term sequence, and the fifth vector indicates a distance between the approximate phrase and the offset;

[0255] The calculation module 1003 is specifically configured to determine the first distance according to the fourth vector and the fifth vector.

[0256] In some embodiments of the present application,

[0257] The retrieval module 1002 is further used to determine the phrase centroid according to the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase centroid is a queryable word whose weight is accumulated in sequence according to the first queryable word sequence and the cumulative sum is greater than the first threshold.

[0258] In some embodiments of the present application,

[0259] The calculation module 1003 is specifically used to determine the first distance according to the fourth vector, the fifth vector and the weight, wherein the weight is used to perform weighted correction on the distance between the fourth vector and the fifth vector.

[0260] In some embodiments of the present application, the value of any component in the fourth vector is consistent with the size of the component corresponding to the centroid of the phrase in the fifth vector.

[0261] In some embodiments of the present application, the first distance is the Chebyshev distance between the fourth vector and the fifth vector after weighted correction, or the first distance is the Manhattan distance between the fourth vector and the fifth vector after weighted correction, or the first distance is the norm distance between the fourth vector and the fifth vector after weighted correction, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector after weighted correction, or the first distance is the Minkowski distance between the fourth vector and the fifth vector after weighted correction.

[0262] In some embodiments of the present application, the value of any component in the fourth vector is consistent with the value of the smallest component in the fifth vector.

[0263] In some embodiments of the present application, the first distance is the Chebyshev distance between the fourth vector and the fifth vector, or the first distance is the Manhattan distance between the fourth vector and the fifth vector, or the first distance is the norm distance between the fourth vector and the fifth vector, or the first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or the first distance is the Minkowski distance between the fourth vector and the fifth vector.

[0264] It should be noted that the information interaction, execution process, etc. between the modules of the above-mentioned information retrieval device 1000 are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown in the above of the present application, and will not be repeated here.

[0265] The above describes the information retrieval device in the embodiment of the present application from the perspective of modular functional entities, and the following describes the information retrieval device in the embodiment of the present application from the perspective of hardware processing. Fig.11 FIG. 1 is a schematic diagram of the hardware structure of the computer device in the embodiment of the present application. Fig.11 As shown, the computer device may include:

[0266] The computer device includes at least one processor 1101 , a communication line 1107 , a memory 1103 and at least one communication interface 1104 .

[0267] Processor 1101 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (server IC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0268] The communication link 1107 may include a pathway to transmit information between the above-mentioned components.

[0269] The communication interface 1104 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet.

[0270] The memory 1103 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and the memory may exist independently and be connected to the processor via a communication line 1107. The memory may also be integrated with the processor.

[0271] The memory 1103 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 1101. The processor 1101 is used to execute the computer-executable instructions stored in the memory 1103, thereby realizing the sentence retrieval method provided in the above embodiment of the present application.

[0272] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0273] In a specific implementation, as an embodiment, a computer device may include multiple processors, such as Fig.11 1 and 1102. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0274] In a specific implementation, as an embodiment, the computer device may further include an output device 1105 and an input device 1106. The output device 1105 communicates with the processor 1101 and may display information in a variety of ways. The input device 1106 communicates with the processor 1101 and may receive user input in a variety of ways. For example, the input device 1106 may be a mouse, a touch screen device, or a sensor device.

[0275] When the computer device is a terminal device, in the computer device, the processor 1102 may include one or more processing units, for example, the processor 1102 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units may be independent devices, or may be integrated in one or more processors.

[0276] The controller may be the nerve center and command center of the computer device 1000. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0277] The processor 1102 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 1102 is a cache memory. The memory may store instructions or data that the processor 1102 has just used or cyclically used. If the processor 1102 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 1102, and thus improves the efficiency of the system.

[0278] In some embodiments, the processor 1102 may include one or more interfaces. The interface may include an inter-integrated circuit (IC) interface, an inter-integrated circuit sound (IS) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0279] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the computer device 1000. In other embodiments of the present application, the computer device 1000 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0280] The charging management module is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module may receive charging input from the wired charger through a USB interface.

[0281] The power management module is used to connect the battery, the charging management module and the processor 1102. The power management module receives input from the battery and / or the charging management module to power the processor 1102, the internal memory, the external memory, the display screen, the camera, and the wireless communication module.

[0282] The wireless communication function of the computer device 1000 can be implemented through antenna 1, antenna 2, a mobile communication module, a wireless communication module, a modem processor, and a baseband processor.

[0283] In some possible implementations, the computer device 1000 may communicate with other devices using a wireless communication function.

[0284] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in computer device 1000 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0285] The mobile communication module can provide solutions for wireless communications including 1G / 3G / 4G / 5G applied to the computer device 1000. The mobile communication module may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves from antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through antenna 2. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the processor 1102. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the same device as at least some of the modules of the processor 1102.

[0286] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, a receiver, etc.), or displays an image or video through a display screen. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 1102 and be set in the same device as the mobile communication module or other functional modules.

[0287] The wireless communication module can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the computer device 1000. The wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via antenna 1, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 1102. The wireless communication module can also receive the signal to be sent from the processor 1102, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through antenna 2.

[0288] In some embodiments, the antenna 1 of the computer device 1000 is coupled to the mobile communication module, and the antenna 2 is coupled to the wireless communication module, so that the computer device 1000 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0289] The computer device 1000 implements display functions through a GPU, a display screen, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1102 may include one or more GPUs, which execute program instructions to generate or change display information.

[0290] The display screen is used to display images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the computer device 1000 may include 1 or N display screens, where N is a positive integer greater than 1.

[0291] In some feasible implementations, the display screen may be used to display various interfaces output by the system of the computer device 1000. For various interfaces output by the computer device 1000, reference may be made to the relevant descriptions of the subsequent embodiments.

[0292] The computer device 1000 can realize the shooting function through an ISP, a camera, a video codec, a GPU, a display screen, and an application processor.

[0293] ISP is used to process the data fed back by the camera. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in the camera.

[0294] The camera is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the computer device 1000 may include 1 or N cameras, where N is a positive integer greater than 1.

[0295] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0296] Video codecs are used to compress or decompress digital videos. The computer device 1000 may support one or more video codecs. Thus, the computer device 1000 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG1, MPEG3, MPEG4, etc.

[0297] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, the computer device 1000 can realize intelligent cognition and other applications, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0298] The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the computer device 1000. The external memory card communicates with the processor 1102 through the external memory interface to implement a data storage function, such as storing music, video and other files in the external memory card.

[0299] The internal memory can be used to store computer executable program codes, which include instructions. The processor 1102 executes various functional applications and data processing of the computer device 1000 by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the computer device 1000 (such as audio data, a phone book, etc.), etc. In addition, the internal memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0300] The computer device 1000 can implement audio functions through an audio module, a speaker, a receiver, a microphone, a headphone jack, and an application processor, etc. For example, music playing, recording, etc. In some feasible implementations, the audio module can be used to play the sound corresponding to the video. For example, when the display screen displays the video playing screen, the audio module outputs the sound of the video playing.

[0301] The audio module is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0302] A speaker, also called a "horn", is used to convert audio electrical signals into sound signals.

[0303] A receiver, also called a "handset", is used to convert audio electrical signals into sound signals.

[0304] A microphone, also known as a "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0305] The headphone jack is used to connect wired headphones. The headphone jack can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0306] The computer device may be a general-purpose device or a dedicated device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a wireless terminal device, an embedded device or a computer with a Fig.11 The embodiment of the present application does not limit the type of computer device. The computer device can be a cloud server or a terminal device, which is not limited here.

[0307] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the computer device 1000. In other embodiments of the present application, the computer device 1000 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0308] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0309] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0310] Through the description of the above implementation methods, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation method in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc., including a number of instructions to enable a computer device to execute the methods described in each embodiment of the present application.

[0311] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0312] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, an XX device, a computing device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, XX device, computing device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as an XX device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0313] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0314] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0315] It should be understood that in the embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0316] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0317] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0318] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0319] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0320] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0321] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0322] In short, the above is only a preferred embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. A sentence retrieval method, characterized in that: include: Extracting a first queryable word sequence from a query statement, wherein the first queryable word sequence indicates a plurality of words arranged in a word order according to the query statement, the first queryable word sequence includes at least one core word and one or more non-core words, and the at least one core word is a queryable word in the first queryable word sequence whose semantic weight is greater than that of the non-core word; Retrieving an original document set according to the at least one core word to obtain a target document set, wherein the target document set includes at least one target document, and the target document includes the at least one core word; Calculating the correlation between the first queryable word sequence and each document in the target document set, so as to obtain the retrieval result of the query statement from the target document set based on the correlation, wherein calculating the correlation between the first queryable word sequence and each document in the target document set includes: Determine an approximate phrase according to a first vector and a second vector, wherein the first vector indicates a position of a core word sequence in the first searchable word sequence in the target document, the second vector indicates a position of the non-core word in the target document, the approximate phrase includes the core word sequence and at least one of the plurality of non-core words, and the core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement; Determine a fifth vector according to the third vector minus the offset, wherein the third vector indicates the position of the approximate phrase in the target document, the offset indicates the distance between each queryable term in the first queryable term sequence and the first queryable term in the first queryable term sequence, and the fifth vector indicates the distance between the approximate phrase and the offset; Determine a first distance according to a fourth vector and the fifth vector, wherein the fourth vector indicates a position of the first queryable word sequence in the target document, and the first distance is a distance between the third vector and the fourth vector; The relevance of the first queryable word sequence to each document in the target document set is determined according to the first distance.

2. The method according to claim 1, characterized in that After extracting the first searchable word sequence from the query statement, the method further includes: The phrase center of gravity is determined based on the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase center of gravity is the queryable word that is last added when the weights of the queryable words are added in sequence according to the first queryable word sequence and the accumulated sum is greater than a first threshold.

3. The method according to claim 1, characterized in that Retrieving the original document set according to the at least one core word to obtain the target document set includes: The original document set is retrieved according to a core word sequence composed of all core words in the first queryable word sequence to obtain the target document set, wherein the target document includes the core word sequence.

4. The method according to claim 1, characterized in that The first searchable word sequence includes a plurality of the non-core words, and searching the original document set according to the at least one core word to obtain a target document set includes: Retrieving the original document set according to a core word sequence composed of all core words in the first queryable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first queryable word sequence; The second document set is retrieved according to the multiple non-core words to obtain the target document set, wherein each target document in the target document set includes the core word sequence in the first queryable word sequence and at least one of the multiple non-core words, and the non-core words included in at least two target documents in the target document set are not completely the same.

5. The method according to any one of claims 1 to 4, characterized in that Determining the first distance according to the fourth vector and the fifth vector includes: The first distance is determined according to the fourth vector, the fifth vector and the weight, wherein the weight is used to perform weighted correction on the distance between the fourth vector and the fifth vector.

6. The method according to claim 5, characterized in that The first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Manhattan distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the norm distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Minkowski distance between the fourth vector and the fifth vector, which is corrected by the weight.

7. The method according to claim 5, characterized in that The first distance is the Chebyshev distance between the fourth vector and the fifth vector, or, The first distance is the Manhattan distance between the fourth vector and the fifth vector, or, The first distance is the norm distance between the fourth vector and the fifth vector, or, The first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or, The first distance is the Minkowski distance between the fourth vector and the fifth vector.

8. An information retrieval device, characterized in that: include: An extraction module is used to extract a first queryable word sequence from a query statement, wherein the first queryable word sequence indicates a plurality of words arranged according to the word order of the query statement, the first queryable word sequence includes at least one core word and one or more non-core words, and the at least one core word is a queryable word in the first queryable word sequence whose semantic weight is greater than that of the non-core word; A retrieval module, configured to retrieve an original document set according to the at least one core word to obtain a target document set, wherein the target document set includes at least one target document, and the target document includes the at least one core word; A calculation module is used to calculate the correlation between the first queryable word sequence and each document in the target document set, so as to obtain the retrieval result of the query statement from the target document set based on the correlation, wherein calculating the correlation between the first queryable word sequence and each document in the target document set includes: Determine an approximate phrase according to a first vector and a second vector, wherein the first vector indicates a position of a core word sequence in the first searchable word sequence in the target document, the second vector indicates a position of the non-core word in the target document, the approximate phrase includes the core word sequence and at least one of the plurality of non-core words, and the core words in the core word sequence are arranged in the order in which the core words are arranged in the query statement; Determine a fifth vector according to the third vector minus the offset, wherein the third vector indicates the position of the approximate phrase in the target document, the offset indicates the distance between each queryable term in the first queryable term sequence and the first queryable term in the first queryable term sequence, and the fifth vector indicates the distance between the approximate phrase and the offset; Determine a first distance according to a fourth vector and the fifth vector, wherein the fourth vector indicates a position of the first queryable word sequence in the target document, and the first distance is a distance between the third vector and the fourth vector; The relevance of the first queryable word sequence to each document in the target document set is determined according to the first distance.

9. The information retrieval device according to claim 8, characterized in that: The calculation module is also used to determine the phrase center of gravity based on the first queryable word sequence and the weights of each queryable word in the first queryable word sequence, wherein the phrase center of gravity is the queryable word that is last added when the weights of the queryable words are added in sequence according to the first queryable word sequence and the accumulated sum is greater than a first threshold.

10. The information retrieval device according to claim 8, characterized in that: The retrieval module is specifically used to retrieve the original document set according to the core word sequence composed of all the core words in the first searchable word sequence to obtain the target document set, and the target document includes the core word sequence.

11. The information retrieval device according to claim 8, characterized in that: The retrieval module is specifically configured to retrieve the original document set according to a core word sequence composed of all core words in the first searchable word sequence to obtain a second document set, wherein the second document set includes at least one second document, and the second document includes the core word sequence in the first searchable word sequence; The retrieval module is specifically used to retrieve the second document set based on the multiple non-core words to obtain the target document set, wherein each target document in the target document set includes the core word sequence in the first queryable word sequence and at least one of the multiple non-core words, and in the target document set, the non-core words included in at least two target documents are not completely the same.

12. The information retrieval device according to any one of claims 8 to 11, characterized in that: The calculation module is specifically used to determine the first distance according to the fourth vector, the fifth vector and the weight, wherein the weight is used to perform weighted correction on the distance between the fourth vector and the fifth vector.

13. The information retrieval device according to claim 12, characterized in that: The first distance is the Chebyshev distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Manhattan distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the norm distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Mahalanobis distance between the fourth vector and the fifth vector, which is corrected by the weight. or, The first distance is the Minkowski distance between the fourth vector and the fifth vector, which is corrected by the weight.

14. The information retrieval device according to claim 12, characterized in that: The first distance is the Chebyshev distance between the fourth vector and the fifth vector, or, The first distance is the Manhattan distance between the fourth vector and the fifth vector, or, The first distance is the norm distance between the fourth vector and the fifth vector, or, The first distance is the Mahalanobis distance between the fourth vector and the fifth vector, or, The first distance is the Minkowski distance between the fourth vector and the fifth vector.

15. A computer device, characterized in that: The computer device includes: an input / output (I / O) interface, a processor and a memory, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to perform the method according to any one of claims 1 to 7.

16. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a computer device, the computer device is caused to perform the method according to any one of claims 1 to 7.

17. A computer program product comprising instructions, characterized in that When the instructions are executed on a computer device or a processor, the computer device or the processor is caused to execute the method according to any one of claims 1 to 7.

18. A terminal device, characterized in that: The terminal device includes: an input / output (I / O) interface, a processor and a memory, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to perform the method according to any one of claims 1 to 7.

Images