A method and apparatus for data retrieval

Abstract

This application discloses a data retrieval method and apparatus. The method may include: receiving search conditions; generating a first structured object, the first structured object including a query statement contained in the search conditions; creating a scorer based on the first structured object, the scorer including the first structured object, M document identifiers matching the query statement, and a scoring component, where M is a positive integer greater than 0; obtaining M first documents matching the M document identifiers, and generating a score description information corresponding to each of the M first documents through the scoring component; generating M second structured objects based on the score description information corresponding to each first document and the first structured object, wherein one second structured object corresponds to one first document. This application embodiment can solve the latency problem caused by obtaining detailed information of search results during data retrieval, and is applicable to more business scenarios, while also improving retrieval efficiency.

Description

Technical Field

[0001] This application relates to the field of data retrieval technology, and in particular to a data retrieval method and apparatus. Background Technology

[0002] Data retrieval is an important branch of computer science and information retrieval that involves finding information related to search criteria from large datasets. Data retrieval can be applied to various types of data, including text, images, audio, and video.

[0003] In the existing field of data retrieval, data can be retrieved using any conditions. Conditions can be combined in various ways and with multiple fields to obtain the corresponding data. For the retrieved search results, detailed information, such as explanatory information, can be obtained by performing interpretive query operations.

[0004] However, in existing technologies, obtaining detailed information, such as explanatory information, requires performing an explanatory query operation for each hit document in the search results, which may result in a long latency. Furthermore, explanatory queries can only obtain explanatory information for search results in simple business scenarios and may not be applicable to some complex business scenarios. Therefore, how to solve the problem of long latency caused by obtaining explanatory information from search results during data retrieval, and increase applicability to various business scenarios, is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a data retrieval method and apparatus to solve the problem of long delays caused by obtaining detailed information of search results during data retrieval, thereby improving retrieval efficiency.

[0006] In a first aspect, embodiments of this application provide a data retrieval method, which may include:

[0007] The system receives search criteria and generates a first structured object, which includes the query statement contained in the search criteria. Based on the first structured object, a scorer is created. The scorer includes the first structured object, M document identifiers matching the query statement, and a scoring component, where M is a positive integer greater than 0. M first documents matching the M document identifiers are obtained, and a scoring description is generated for each of the M first documents using the scoring component. Based on the scoring description for each first document and the first structured object, M second structured objects are generated, where one second structured object corresponds to one first document.

[0008] In existing technologies, when a user performs a content search, the search engine typically queries and returns preliminary search results (e.g., preliminary search results including one or more hit documents) based on the search criteria. When the user needs to further view one or more hit documents in the search results, the search engine needs to query and extract detailed information (e.g., hit reasons, hit details, detailed content, explanatory information, etc.) from those hit documents again. This results in query latency, leading to a poor user experience and wasting system resources. To address this technical problem, this application embodiment configures the search engine, upon first receiving the user's search criteria, to not only obtain preliminary search results but also to further obtain more in-depth detailed information from those search results and pre-generate corresponding structured information. This allows the search engine to immediately present the pre-generated structured information to the user when they need to conduct in-depth queries on a specific hit document in the search results. This significantly reduces the latency caused by the multiple queries required by existing technologies, saves system resources, and improves system performance. Specifically, in this embodiment, during the search process, the search engine creates a scorer based on the search criteria. This scorer may include preliminary structured information (i.e., a first structured object containing the query statement), identifiers of the hit documents, and a scoring component. Further, the search engine obtains corresponding candidate documents based on the identifier of each hit document and generates score description information for each hit document through the scoring component. It then further generates in-depth structured information (i.e., a second structured object) containing the aforementioned first structured object and the score description information for each hit document, so that relevant information can be accurately and efficiently obtained when the user needs to query later. In summary, this embodiment can obtain and save the required detailed information at the same time as the first hit document is found. When the user performs further queries, there is no need to perform repeated interpretation and query operations, thus greatly reducing the latency of obtaining detailed information during the search process and improving search performance. It can be applied to businesses with larger data volumes and increases its applicability to more business scenarios.

[0009] In one possible implementation, the search engine includes an inverted index, which includes a mapping relationship between document identifiers and corresponding documents. Obtaining the M first documents matching the M document identifiers includes searching the inverted index for the M first documents corresponding to the M document identifiers. In this embodiment, by applying the inverted index structure, the search engine can retrieve the M candidate documents matching the M document identifiers more quickly, thereby improving retrieval efficiency. Specifically, the inverted index, through the mapping relationship between document identifiers (i.e., IDs) and candidate documents (i.e., first documents), can quickly locate and retrieve the M candidate documents matching the search criteria. When processing large amounts of data, this method significantly reduces search time, allowing users to obtain relevant search results in a short time. In this embodiment, finding the corresponding document based on the inverted index structure is particularly important in business scenarios involving complex queries and large-scale document databases, ensuring that the search engine maintains high accuracy and response speed even under high load.

[0010] In one possible implementation, the rating description information corresponding to each first document includes hit information and rating information; the step of generating the rating description information corresponding to each of the M first documents through the rating component includes: for the scorer, filtering out Q first documents with nested structures and J first documents with non-nested structures from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q; generating the hit information and rating information corresponding to each of the M first documents through the rating component, wherein the rating information is the score of the matching degree between the corresponding first document and the search conditions. In this embodiment, by providing detailed descriptions of the rating description information in the search engine and classifying the structure of each candidate document in the scorer, different processing can be performed on documents with different structures when obtaining detailed information later, thereby improving the relevance and accuracy of the search results. Specifically, in this embodiment, the scorer can identify nested and non-nested document structures (i.e., when processing documents containing multi-level data, such as nested JSON or XML documents, the scorer can distinguish and identify whether the document has a nested structure), which allows the search engine to process complex queries more accurately; whereas in the prior art, different processing is not applied to documents with different structure types. Furthermore, the scorer can generate score description information (i.e., hit information and score information) for each candidate document through the scoring component, and the score information is a score based on the matching degree between the corresponding candidate document and the search conditions, thereby improving the accuracy of search results.

[0011] In one possible implementation, the hit information in the J first documents includes one or more of the hit fields, hit words, and hit positions that match the search conditions in the corresponding first documents. In this embodiment, after determining that the J candidate documents are non-nested, the scoring component can determine the information that may be included in the hit information based on the matching situation between the search conditions and the non-nested candidate documents, such as one or more of the hit fields, hit words, and hit positions. The hit information can indicate the specific location where the query word appears in the document, and can also be used together with other information in the matching situation (e.g., document frequency (DF), term frequency (TF), document length, query word weight, etc.) to calculate the scoring information.

[0012] In one possible implementation, the first document with a nested structure includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, a set of document identifier sequences of the one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document. In the prior art, when a user needs to further view the hit reasons of one or more hit documents in the search results, for hit documents with complex structures, only shallow-level hit reasons (e.g., the hit reason of the main document in a nested document) can be obtained, and deeper-level hit reasons (e.g., the hit reason of the sub-document nested in the main document) cannot be obtained. Therefore, obtaining detailed information such as explanatory information can only be applied to simple business scenarios and has no practical business significance. To address this technical problem, in the embodiments of this application, the scoring component in the scorer can obtain the explanatory information of the main document and sub-documents in the candidate documents with the nested structure, rather than only obtaining the explanatory information of the main document, thus improving the applicability to more business scenarios. Specifically, if the Q candidate documents are nested documents, based on this nested structure, the scorer (e.g., the scorer in the Lucene search engine) can be modified so that after detecting a candidate document with a nested structure, it can obtain the hit information of the internal documents in each nested candidate document (e.g., it can obtain one or more of the following: the ID of the main document, a set of ID sequences of one or more sub-documents, the ID of the hit sub-document in the main document, and the hit position of the hit sub-document). This makes the embodiments of this application applicable to business scenarios with complex structures (e.g., documents with nested structures can include videos, note-taking apps, rich text, etc.). In summary, in the embodiments of this application, when detecting complex business scenarios with nested document structures, by enabling the scorer to obtain more and more detailed information (e.g., hit information of deep sub-documents), the embodiments of this application can be applied to more complex business scenarios, increasing the applicability to various business scenarios.

[0013] In one possible implementation, generating the first structured object includes: obtaining K sub-queries included in the query statement, where K is a positive integer greater than 0; and generating K third structured objects based on the K sub-queries, wherein each third structured object includes a corresponding sub-query, and the first structured object includes the K third structured objects. In this embodiment, by obtaining multiple sub-queries contained in the query statement and generating a corresponding structured object for each sub-query, the search engine can understand and process complex queries in a more granular way. The independent structuring of each subquery allows the search engine to process each subquery individually. (Furthermore, multiple conditions can be combined within a query to filter data; for example, logical operators such as "OR" and "AND" can be used to combine multiple subqueries into a single query.) When querying sub-documents under nested objects using subqueries, parent documents that are matched by the sub-document search conditions can be returned, improving the relevance and accuracy of search results. Simultaneously, breaking down complex queries into combinations of subqueries allows the search engine to employ different strategies and algorithms for different parts of the query conditions, enhancing its adaptability to complex information needs and its ability to handle complex queries. This contributes to improved information retrieval efficiency, especially when dealing with queries containing logical operators (such as AND / OR / NOT) or nested queries, resulting in more accurate search results.

[0014] In one possible implementation, K corresponding sub-scorers are created based on the K third structured objects. Each sub-scorer includes a corresponding third structured object, L document identifiers matching the corresponding third structured object, and the scoring component. The scorer is created based on the K corresponding sub-scorers, where L is a positive integer greater than 0. In this embodiment, a corresponding sub-scorer is created based on each sub-query statement in the query statement, and different sub-query statements are processed separately based on different sub-scorers, improving the flexibility for handling complex queries. Specifically, in this embodiment, a corresponding sub-scorer is created for each sub-query statement. These sub-scorers exist within a corresponding main scorer (i.e., the scorer corresponding to the first structured object generated based on the query statement). Each sub-scorer operates on a specific query segment (e.g., it can score and sort each individually), enabling the search engine to better interpret and optimize query results, while improving the ability to handle nested queries and multi-condition queries.

[0015] In one possible implementation, for each scorer, the corresponding first documents are sorted based on the rating description information; based on the result of the first sort, N first documents whose rating description information satisfies preset conditions are selected from the first documents corresponding to each scorer, where N is a positive integer less than or equal to M. In this embodiment, after obtaining detailed information such as explanatory information, the candidate documents are sorted according to the rating information of each scorer and its sub-scorers, and then the matching documents that meet the preset conditions are selected based on the sorting results. This can effectively distinguish and prioritize the display of data with the highest relevance to the query conditions from a large number of candidate documents. Specifically, during the sorting process, the candidate documents (i.e., the first documents) in each scorer and sub-scorer are sorted based on their scores (e.g., by sorting the candidate documents in order of relevance). Then, documents that meet specific criteria are selected based on preset conditions (e.g., if there are 100,000 candidate documents in the search results, but the business only needs the top ten most relevant documents, then the top ten documents are selected as the hit documents based on the sorting results). This ensures the relevance and accuracy of the search results, making the final results the information that best matches the search criteria, thereby improving the user's search experience.

[0016] In one possible implementation, the search engine includes a preset storage component; for each first document in the scorer, a corresponding second structured object is saved to the storage component; the method further includes: retrieving N corresponding second structured objects from the storage component based on the document identifiers of the N first documents; and generating N corresponding structured information based on the N second structured objects. In this embodiment, during the initial retrieval process, by pre-storing detailed information (such as explanatory information) of all candidate documents, only the required explanatory information is retrieved when outputting search results. Compared to the prior art, which requires multiple queries and retrieval operations on the returned search results, this avoids the latency caused by explanatory queries; furthermore, the prior art stores explanatory information by concatenating it into a string, which cannot be directly accessed, while in this embodiment, the explanatory information is constructed as structured information, making it easier for downstream businesses to access directly. Specifically, based on the explanation information of candidate documents in each scorer, corresponding explanation information objects can be generated and saved in an explanation information object collector. During the search result output stage, the corresponding explanation information object is retrieved from the explanation information object collector based on the ID of the matching document in the search results, generating the corresponding structured explanation information. This avoids retrieving all the explanation information from all candidate documents, thus preventing wasted memory resources. In existing technologies, explanation information typically requires multiple explanation queries, and this information is concatenated into a string format. Furthermore, the included hit information is insufficient, making it difficult to extract structured information for application in subsequent business logic. Such explanations are primarily used for development and debugging. Additionally, concatenating all explanation reasons into strings leads to high memory usage and lacks practical business application value. In summary, this embodiment obtains all explanation information at once during the search process and retrieves it when needed, avoiding the latency caused by explanation queries. The explanation information is constructed as structured information, allowing downstream businesses to directly access and use it, reducing memory usage and making it more convenient to use, thus increasing its practical business value.

[0017] In one possible implementation, for each scorer, the corresponding first document is sorted based on the score description information; based on the result of the first sort, N first documents whose score description information satisfies preset conditions are selected from the first documents corresponding to each scorer, where N is a positive integer less than or equal to M. In this embodiment, when the detailed information (such as explanatory information) is more comprehensive structured information, the explanatory information can be better utilized during the secondary sorting to perform in-depth filtering, so as to select documents that match the query conditions, thereby improving the accuracy of search results. Specifically, through the explanatory information in the structured information, the search engine can understand in detail how the score of each matching document is calculated, including the degree of matching between each word in the query and the document, word frequency, position information, etc.; based on the above information, the search engine can more accurately understand the relevance between the document and the query, thereby achieving a more refined sorting.

[0018] In one possible implementation, the N corresponding first documents are ranked in a second order based on the similarity score between each of the N structured information pieces and its corresponding first document; or, the N corresponding first documents are ranked in a second order based on the proportion score of the hit words in each of the N structured information pieces in the corresponding hit field. In this embodiment, the structured information based on detailed information (such as explanatory information) can be directly accessed by downstream businesses, making it easier to obtain more explanatory information, perform a more detailed secondary ranking of the hit documents, and thus conduct a more in-depth analysis and optimization of the search results. Specifically, compared to the string concatenation explanation information in the prior art, the structured explanation information in this application embodiment can be more conveniently and directly accessed, providing transparency about how records match search conditions and revealing how the scores of each condition affect the final ranking. Scores can be calculated based on the similarity between the explanation information and the corresponding candidate documents (for example, in the field of semantic retrieval, the similarity between query terms and documents can be calculated using a vector space model). The search results can then be re-ranked based on this similarity score. Alternatively, the weight ratio of the hit words in the hit field can be used to find hit documents with high weight ratios, further refining the ranking logic to filter out more relevant search results and improve the adaptability of the search engine in various business scenarios.

[0019] Secondly, embodiments of this application provide a data retrieval apparatus, which may include:

[0020] The first parsing unit is used to receive search conditions and generate a first structured object, wherein the first structured object includes the query statement contained in the search conditions;

[0021] The first creation unit creates a scorer based on the first structured object. The scorer includes the first structured object and M document identifiers that match the first structured object, where M is a positive integer greater than 0.

[0022] The information acquisition unit acquires M first documents that match the M document identifiers, and generates rating description information corresponding to each of the M first documents through the rating component;

[0023] The first generation unit generates M second structured objects based on the rating description information corresponding to each first document and the first structured object, wherein one second structured object corresponds to one first document.

[0024] In existing technologies, when a user performs a content search, the search engine typically queries and returns preliminary search results (e.g., preliminary search results including one or more hit documents) based on the search criteria. When the user needs to further view one or more hit documents in the search results, the search engine needs to query and extract detailed information (e.g., hit reason, hit details, detailed content, explanatory information, etc.) from those hit documents again. This results in query latency, leading to a poor user experience and wasting system resources. To address this technical problem, this application embodiment configures the search engine, upon first receiving the user's search criteria, to not only obtain preliminary search results but also to further obtain more detailed and in-depth information (such as explanatory information) from those search results and pre-generate corresponding structured information. This allows the search engine to immediately present the pre-generated structured information to the user when they need to conduct in-depth queries on a specific hit document in the search results. This significantly reduces the latency caused by the multiple queries required by existing technologies, saves system resources, and improves system performance. Specifically, in this embodiment, during the search process, the search engine creates a scorer based on the search criteria. This scorer may include preliminary structured information (i.e., a first structured object containing the query statement), identifiers of the hit documents, and a scoring component. Further, the search engine obtains corresponding candidate documents based on the identifier of each hit document and generates score description information for each hit document through the scoring component. It then further generates in-depth structured information (i.e., a second structured object) containing the aforementioned first structured object and the score description information for each hit document, so that relevant information can be accurately and efficiently obtained when the user needs to query later. In summary, this embodiment can obtain and save the required explanatory information at the same time as the first hit document is searched. When the user performs further queries, there is no need to perform repeated explanatory query operations, thus greatly reducing the latency of obtaining explanatory information during the search process and improving search performance. It can be applied to businesses with larger data volumes and increases applicability to more business scenarios.

[0025] In one possible implementation, the search engine includes an inverted index that includes a mapping between document identifiers and corresponding documents;

[0026] The information acquisition unit is specifically used for:

[0027] Find the M first documents corresponding to the M document identifiers from the inverted index.

[0028] In one possible implementation, the rating description information corresponding to each of the first documents includes hit information and rating information;

[0029] The information acquisition unit is specifically used for:

[0030] For each scorer, Q first documents with nested structures and J first documents with non-nested structures are selected from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q.

[0031] The scoring component generates hit information and score information for each of the M first documents, where the score information is a score indicating the degree of matching between the corresponding first document and the search criteria.

[0032] In one possible implementation, the hit information in the J first documents includes one or more of the hit fields, hit words, and hit locations that match the search criteria in the corresponding first documents.

[0033] In one possible implementation, the first document, which is a nested structure, includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, a set of document identifier sequences of the one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document.

[0034] In one possible implementation, the first generating unit is specifically used for:

[0035] Obtain the K subqueries included in the query statement, where K is a positive integer greater than 0;

[0036] Based on the K subqueries, K third structured objects are generated, wherein each third structured object includes a corresponding subquery, and the first structured object includes the K third structured objects.

[0037] In one possible implementation, the first creation unit is specifically used for:

[0038] K corresponding sub-scoring units are created based on the K third structured objects. Each sub-scoring unit includes a corresponding third structured object, L document identifiers that match the corresponding third structured object, and the scoring component.

[0039] The scorer is created based on the K corresponding sub-scorers, where L is a positive integer greater than 0.

[0040] In one possible implementation, the device further includes:

[0041] The first sorting unit sorts the corresponding first document based on the scoring description information for each scorer.

[0042] The filtering unit, based on the result of the first sorting, filters out N first documents from the first documents corresponding to each scorer whose scoring description information meets preset conditions, where N is a positive integer less than or equal to M.

[0043] In one possible implementation, the search engine includes a preset storage component; for each of the first documents in the scorer, the corresponding second structured object is saved to the storage component; the apparatus further includes:

[0044] The extraction unit extracts N corresponding second structured objects from the storage component based on the document identifiers of the N first documents.

[0045] The information generation unit generates N corresponding structured information items based on N second structured objects.

[0046] In one possible implementation, the device further includes:

[0047] The second sorting unit performs a second sorting on the N corresponding structured information on the N first documents, and outputs the sorting results to the search results.

[0048] In one possible implementation, the second sorting unit is specifically used for:

[0049] Based on the similarity score between each of the N structured information pieces and the corresponding first document, the N corresponding first documents are sorted in a second manner.

[0050] Alternatively, based on the percentage score of the hit words in each of the N structured information pieces in the corresponding hit field, the N corresponding first documents are sorted in a second manner.

[0051] Thirdly, embodiments of this application provide a computer storage medium, characterized in that the computer storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-11.

[0052] Fourthly, embodiments of this application provide a computer program, characterized in that the computer program includes instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-11. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0054] Figure 1A This is a schematic diagram of a system architecture provided in an embodiment of this application.

[0055] Figure 1B This is a schematic diagram of another system architecture provided in an embodiment of this application.

[0056] Figure 2 This is a schematic diagram of a search engine module provided in an embodiment of this application.

[0057] Figure 3A This is a flowchart illustrating a search method provided in an embodiment of this application.

[0058] Figure 3B This is a flowchart illustrating another search method provided in an embodiment of this application.

[0059] Figure 4A This is a schematic diagram of a search step provided in an embodiment of this application.

[0060] Figure 4B This is a schematic diagram of a score calculator structure provided in an embodiment of this application.

[0061] Figure 4C This is a schematic diagram of another scorer structure provided in an embodiment of this application.

[0062] Figure 4D This is a schematic diagram of a nested document structure provided in an embodiment of this application.

[0063] Figure 4E This is a schematic diagram of the structural process of a search engine provided in an embodiment of this application.

[0064] Figure 4F This is a schematic diagram of an application scenario provided in an embodiment of this application.

[0065] Figure 4G This is a schematic diagram of another application scenario provided by the embodiments of this application.

[0066] Figure 4H This is a schematic diagram of another application scenario provided in the embodiments of this application.

[0067] Figure 5 This is a schematic diagram of another search device provided in an embodiment of this application.

[0068] Figure 6A This is a hardware structure block diagram of the electronic device provided in the embodiments of this application.

[0069] Figure 6B This is the software architecture of the electronic device provided in the embodiments of this application. Detailed Implementation

[0070] The embodiments of this application will now be described with reference to the accompanying drawings.

[0071] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0072] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0073] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. For example, a component may communicate via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0074] First, some of the terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0075] (1) Data retrieval refers to the process of retrieving relevant or specific data from a large dataset using specific query criteria. It is an important component of computer science and information technology, applied in search engines, database management, information retrieval systems, and more. The data retrieval process typically includes steps such as query parsing, index lookup, result ranking, and result return, aiming to quickly and accurately find the data needed by users and meet their information needs.

[0076] (2) Lucene (Apache Lucene) is an open-source full-text search engine library provided as a Java library, enabling full-text search functionality for applications. It boasts powerful indexing and search capabilities, can handle large amounts of data, and offers various scoring models and extension mechanisms. Lucene's core functions include index building, query parsing, search execution, and result ranking, and it is widely used in search engines, databases, and information retrieval systems. Through Lucene, developers can build high-performance, highly scalable search solutions to meet various complex search needs.

[0077] (3) Candidate Documents: In the field of information retrieval, candidate documents refer to a set of documents that are initially selected from a large number of documents based on specific query conditions, and are considered to be potentially relevant to the user's query. Candidate documents are those documents that are considered likely to satisfy the user's query after the index lookup has been performed but before the final scoring and ranking. The size and quality of the candidate document set directly affect the efficiency and accuracy of the retrieval system. Typically, the retrieval system uses techniques such as inverted indexes to quickly find a set of candidate documents, and then uses more refined scoring and ranking algorithms to find the final retrieval results from the candidate documents.

[0078] (4) Hit Documents refer to documents that match the user's query criteria during the data retrieval process. Hit documents are typically sorted according to a scoring model and then returned to the user. They reflect the relevance between the document and the user's query and are a direct output of the core task of the retrieval system. In retrieval systems such as Lucene or Elasticsearch, hit documents usually contain the document's identifier, score, and other possible metadata information to help users understand and evaluate the quality and relevance of the search results.

[0079] (5) Indexed Documents refer to documents that have undergone indexing to facilitate rapid retrieval in information retrieval systems. During the indexing process, the system parses the document content, extracts keywords, and establishes a mapping relationship between keywords and documents, typically stored in an inverted index. Thus, when a user submits a query, the system can quickly find documents matching the query criteria by searching the index, improving retrieval efficiency and accuracy.

[0080] First, the specific technical problem to be solved in this application is analyzed and proposed. In the prior art, data retrieval techniques include the following solutions:

[0081] Existing data retrieval technologies, such as Lucene and Elasticsearch, provide an "Explain" function to help users understand the source of search result scores. Specifically, Lucene provides the `IndexSearcher.explain()` method, which generates explanatory information for a given query and document ID; while in Elasticsearch, setting the `explain` parameter to `true` in the query request generates explanatory information for each returned document. This explanatory information includes details about the document's score, the source of the score, and how the score was calculated. With this explanatory information, developers can understand why certain documents were matched and can debug and optimize their queries.

[0082] Solution: Based on the search explanation method in data retrieval, it can specifically include the following steps 1 and 2.

[0083] Step 1: Perform a separate interpretation query for each returned record.

[0084] Search engines retrieve multiple records based on search criteria. If explanations are required, a separate explanation query is executed for each returned document. For example, if the search results return 100 records, 101 queries are needed (1 result query + 100 explanation queries). In Lucene, explanation queries can be implemented using the `IndexSearcher.explain(Query query, int doc)` method. This method is called once for each record, where the input parameters are `query` (the original query object) and `doc` (the document ID). Executing this method returns an `Explanation` object containing a detailed explanation of the document's score, including how the score was calculated, the scoring model used, and the contributions of various scorers.

[0085] Step 2: The returned explanation is a complete character description.

[0086] In existing technologies, it is possible to perform interpretation queries on hit documents with simple structures (for example, the hit document is not a nested document containing multiple nested objects), obtain interpretation information from them, and concatenate the interpretation information into a complete string to return the results to the display interface.

[0087] The above solution is currently mainly applicable to scenarios where, after a search is completed, an explanatory query is performed based on the search results to obtain explanatory information. However, it also has the following drawbacks:

[0088] Disadvantage 1:

[0089] The aforementioned interpretive query method is typically suitable for data retrieval applications with small datasets. However, for applications with large datasets, since an interpretive query is required for each returned record, a large number of additional queries can occur when the retrieved result set is large. This increases the system load and response time, causing latency issues that cannot meet the demands of real-time applications. Furthermore, if the retrieval application operates in a high-concurrency environment, the need for an additional interpretive query for each query can significantly impact system throughput. Finally, while this interpretive query implementation provides detailed explanations to help understand the query results, it suffers from significant shortcomings in efficiency and scalability, making it unsuitable for supporting actual business needs. It is generally only used during the debugging and testing phases of the search process.

[0090] Disadvantage 2:

[0091] First, the information returned by the interpreter query is concatenated into a single string, and the reasons for the hit are not detailed enough (e.g., it does not include hit information for sub-documents within nested documents). This interpreter information may not be rich enough, for example, it cannot provide sufficient debugging information for debuggers. Second, because the string is presented as unstructured text, it is extremely difficult to extract structured information from it. Specifically, for complex nested objects, when searching for nested sub-objects, it is very difficult to know which sub-object within the nested object is being searched, and which attribute in the interpreter information is causing the hit. The unstructured interpreter information returned by the interpreter query makes it difficult to extract structured information, making it almost impossible to apply to complex business logic, lacking flexibility and applicability.

[0092] In summary, while explanation functions are quite useful, their latency and unstructured output format make them difficult to use in real-time applications, and they are currently typically used in the development and debugging phases. Furthermore, the format and richness of the explanation information may not meet users' needs for in-depth understanding and optimized queries.

[0093] To address the issue that current interpretive query techniques fail to meet actual business needs, and to achieve the goal of rapidly obtaining detailed information and being applicable to complex real-time business tasks, thereby improving the speed and applicability of interpretive queries in data retrieval, this application, considering the shortcomings of existing technologies, aims to solve one or more of the following three technical problems:

[0094] 1. Reduce the latency of obtaining detailed information (such as explanatory information) (Disadvantage 1). Explanatory query schemes are widely used in data retrieval, partially solving the problem of obtaining explanatory information. However, since only multiple records are returned after a result query, and obtaining explanatory information requires a separate explanatory query for each returned record, multiple explanatory queries are often required when the data volume is large, resulting in a long latency in obtaining explanatory information and impacting data retrieval operations. Therefore, an efficient method for obtaining explanatory information is needed, which can reduce the number of explanatory queries and lower the latency of obtaining explanatory information.

[0095] 2. Improve the usability of detailed information (such as explanatory information) (Disadvantage 2). Currently, returning explanatory information based on concatenated strings is mainly suitable for scenarios where professionals are conducting testing and debugging. In existing technologies, the explanatory information obtained from the returned record is usually concatenated into a string. This string is unstructured information, making it difficult to extract the required information for use in subsequent business logic. Downstream businesses cannot directly access and use the values, and it consumes too much memory. It lacks practical significance for business scenarios where ordinary users want to obtain detailed information (such as the reason for the hit, hit details, detailed content, explanatory information, etc.).

[0096] 3. Improve the applicability of detailed information (such as explanatory information) (Disadvantage 3). Explanatory information obtained based on multiple explanatory queries is currently mainly applicable to simple business scenarios. In existing technologies, explanatory information obtained based on returned records is usually concatenated into a string. This string is unstructured information and can only be used to store explanatory information for simple business scenarios. It cannot be applied to complex business needs (e.g., nested documents among candidate documents, where the candidate documents do not include the nested documents). For example, when a user searches for "cat," they need to retrieve nested documents containing the word "cat," and they need to know that the first sub-object in that nested document is the match, and that the match is due to the tag_name of that first sub-object containing "cat." They also need to know the score of this match condition. Therefore, a structure applicable to complex business scenarios is needed, which can be used to display explanatory information for complex business scenarios, thereby improving the applicability of explanatory information in data retrieval.

[0097] In summary, existing data retrieval solutions cannot meet the actual business needs for obtaining explanatory information. Therefore, the data retrieval method provided in this application is used to solve the above-mentioned technical problems.

[0098] Based on the technical problems mentioned above, and to facilitate understanding of the embodiments of this application, one system architecture on which the embodiments of this application are based is described below. Please refer to... Figure 1A , Figure 1A This is a schematic diagram of a system architecture provided in an embodiment of this application. The structure in this application may include... Figure 1A Search engine 01, search library 02 ( Figure 1A The search library 02 may include a web search database 22) and a terminal device 03.

[0099] Search Engine 01 retrieves data matching search criteria from a corresponding search library, which can be one or more search engines such as Lucene and Elasticsearch. Lucene is a high-performance, scalable search library providing accurate full-text search capabilities, while Elasticsearch, built on Lucene, offers real-time distributed search and analysis capabilities. Search Engine 01 can be applied to various scenarios, such as enterprise search, log analysis, video retrieval, rich text retrieval, and local application retrieval. With Lucene, it can handle complex queries and return high-quality results (e.g., a scorer can be used to score and rank retrieved documents); with Elasticsearch, it can handle large-scale data and provide a fast search experience. Its highly customizable features allow Search Engine 01 to meet different business needs, providing users with efficient and accurate search services. The search engine may include a scorer, which is responsible for calculating the match between documents and queries based on a specific scoring model, assigning a score to each document. This score is typically used to rank documents before returning results to the user. For example, in Lucene and Elasticsearch, the Scorer calculates a score for each document based on a scoring model such as TF-IDF or BM25 to assess the document's relevance to the user's query. Through the Scorer's calculations, the system can ensure that more relevant documents are ranked higher in search results, thereby improving user search satisfaction.

[0100] Search library 02 is used to store the retrieved data. This search library 02 can be a web search database 22, referring to a database resource accessed via the network. It can be one or more of a distributed database, cloud database, or any other type of database, primarily providing data storage and retrieval functions via the network. Such a database typically consists of multiple database servers, supporting remote data access and management. This embodiment does not impose specific limitations. After receiving the Query object obtained by the parser from the search conditions, search library 02 will quickly locate and retrieve matching documents in the inverted index based on the query conditions, providing users with accurate search results. Simultaneously, search library 02 also supports updating, deleting, and adding index data, ensuring the real-time nature and accuracy of the index data.

[0101] Terminal device 03 can be a smartphone, tablet, personal computer (PC), laptop, network terminal, multifunction printer (MFP), smart TV, game console, in-vehicle infotainment system (IVI), smartwatch, smart bracelet, and other smart devices with search capabilities. The search engines in these devices can be Lucene or Elasticsearch, providing users with convenient, fast, and accurate search services to meet the information retrieval needs of different scenarios and requirements.

[0102] Search Engine 01 varies depending on the business scenario. If the business scenario for Search Engine 01 is to retrieve local data, please refer to [link / reference]. Figure 1B ,in, Figure 1B This is a schematic diagram of another system architecture provided in an embodiment of this application. The structure in this application may include... Figure 1B Terminal device 03-A (see also) Figure 1A The embodiment of terminal device 03 is described in detail here, and the terminal device 03-A includes local data 04, which may include a search engine 01-A (a search engine that supports the search function of the local terminal device 03-A; please refer to [reference]). Figure 1AThe embodiment of the search engine 01 is described in detail here, and the local search database 21 is used. In this embodiment, the user operates the local data 04 in the terminal device 03-A (for example, it can be an APP that supports local search services), inputs search conditions, and the search engine 01-A parses the search conditions, obtains the search results from the local search database 21, and then the search engine 01-A performs preset operations (for example, it can be to rate the candidate documents, filter the hit documents based on the rating results, store the detailed information of the hit documents (such as explanation information), and construct the explanation information of the hit documents into structured information in the result retrieval stage). The search engine 01-A returns the search results to the result display interface in the local data 04.

[0103] Local data 04 can be various types of software or applications, such as document editors, spreadsheet software, image editing software, video players, music players, email clients, contact lists, calendar apps, map and navigation software, database management systems (DBMS), file managers, and other software that can generate, store, or manage data. Through search engines like Lucene or Elasticsearch, users can easily and quickly search and retrieve the local data they need within these software or applications, satisfying diverse information retrieval needs.

[0104] Understandable Figures 1A-1B The system architecture described above is only one or more exemplary implementations in the embodiments of this application, and the structures in the embodiments of this application include, but are not limited to, the above system architecture.

[0105] Based on the above system architecture, this application provides a search engine 01 applied to the above system architecture. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of a search engine module provided in an embodiment of this application. The search engine 01-A may include a parsing module 1001, an index reading module 1002, a scoring module 1003, a storage module 1004, and a result display module 1005. Wherein:

[0106] Parsing Module 1001: Responsible for parsing search conditions into Query objects that Lucene can understand. These Query objects can include multiple nested subquery objects. Specifically, during parsing, Parsing Module 1001 may perform lexical analysis, syntactic analysis, and semantic analysis to ensure it understands the query intent of the search conditions. By parsing the search conditions, Parsing Module 1001 can identify important keywords, phrases, and other elements, and then convert these elements into a structured query object (e.g., nested queries or combined queries, where each subquery in a combined query can be related by relational words such as OR or AND). This query object is the foundation for Lucene to execute the search, defining the search parameters and conditions. Parsing Module 1001 can handle various complex queries, including Boolean queries, phrase queries, and wildcard queries, providing accurate guidance for subsequent retrieval processes.

[0107] Index reading module 1002: Responsible for reading document data matching the query from the inverted index. This includes the inverted index itself. Upon receiving a query, the index reading module 1002 searches the inverted index for documents matching the query conditions (e.g., documents containing characters from the query conditions) and marks them as candidate documents. An inverted index is a data structure designed for search engines to store and retrieve keywords from a large number of documents, enabling the rapid discovery of documents containing specific keywords. In the inverted index, each unique keyword is associated with a list of documents containing the identifiers of all documents containing that keyword. Through the inverted index, the search engine can find documents matching the query conditions in a short time. The index reading module 1002 quickly locates relevant documents in the inverted index based on the conditions in the query and reads their data. It can process large amounts of data and return results quickly. Furthermore, the index reading module 1002 works in conjunction with the scoring module 1003 to ensure that the returned documents are highly relevant to the query conditions.

[0108] Scoring module 1003 is responsible for calculating the matching degree between each retrieved document and the query, and assigning a score to each document. The scoring module may include a score calculator, which in turn may include a query statement object, a scoring component, and document identifiers (e.g., IDs of candidate documents stored in the inverted index obtained from the index reading module 1002). The scoring component employs a specific scoring model, such as the TF-IDF model or the BM25 model, to calculate the score based on the document's content and the query conditions. Through these scores, scoring module 1003 can measure the relevance of the document to the query. The higher the score, the better the match between the document and the query. In this embodiment, the Scorer has been modified. The Scorer will store the hit fields, hit words, source query, score, and the location of the hit object of each candidate document as detailed explanations in a corresponding explainquery object. In addition, when a candidate document is detected as a nested document, the Scorer will also store the location of the hit object and the location of the nested sub-object of the candidate document in the above-mentioned query object. Then, the query object corresponding to each candidate document is stored in a preset collector for subsequent retrieval and for constructing structured information such as explanation information.

[0109] Storage Module 1004: This module is a core component of the search engine, including a pre-defined storage component responsible for managing and storing all second-structured objects. These second-structured objects are structured objects generated for the detailed information (such as explanatory information) of each candidate document. The aforementioned storage component is responsible for maintaining the storage logic of the second-structured objects within this module. Before the scoring module 1003 selects documents that meet the pre-defined criteria, it stores each second-structured object in the pre-defined storage component of storage module 1004, waiting to retrieve it after the collection of matched documents is completed.

[0110] Result Display Module 1005: Responsible for retrieving paginated results during the fetch stage and constructing structured information from the retrieved details (such as explanatory information). After the candidate documents are rated and filtered, the result display module can quickly respond by retrieving and providing the corresponding second structured object from the preset storage component in storage module 1004 based on the ID of the selected hit document. Specifically, based on the ID of the hit document, the query object corresponding to the ID of the hit document is found in the aforementioned preset storage component, and the query object is constructed into structured information. Compared with the existing technology of concatenating explanatory information into a string, this structured information can be displayed more intuitively, making it easier for users to view. Moreover, the structured information can be directly used by downstream businesses, reducing memory usage compared to strings. In addition, this structured information can interpret documents including complex nested objects, and based on this feature, it can be applied to more complex business scenarios.

[0111] Understandable Figure 2 The structure of the search engine described in this application is only an exemplary implementation in the embodiments of this application. The structure of the search engine in the embodiments of this application includes, but is not limited to, the above structure.

[0112] based on Figures 1A-1B The provided system architecture, and Figure 2 The provided search engine module, combined with the explanatory search method provided in this application, provides a specific analysis and solution to the technical problems raised in this application.

[0113] See Figure 3A , Figure 3A This is a flowchart illustrating a search method provided in an embodiment of this application. This method can be applied to the above-mentioned... Figures 1A-1B In the system architecture described herein, search engine 01 can be used to support and execute... Figure 3A The method flow shown consists of steps S300-S303. The following will combine... Figure 3A Described from the perspective of search engine 01. The method may include the following steps S300-S303.

[0114] Step S300: Receive search criteria and generate the first structured object.

[0115] Specifically, when a search engine receives a user's search criteria, it first parses and processes them, converting them into a query statement object that the search engine can understand and execute (for example, the Lucene search engine converts the query statements contained in the search criteria into a first structured object). This query statement object can include the query statement, as well as possible additional conditions and parameters, such as filtering conditions and sorting conditions. This query statement object is the core of the search process, defining the scope and method of the search. For example, through the generated query statement object, the Lucene search engine can accurately locate and retrieve documents (e.g., candidate documents) that match the search criteria in the index, thereby quickly and accurately retrieving search results.

[0116] In one possible implementation, K subqueries are obtained from the query statement, where K is a positive integer greater than 0. Based on these K subqueries, K third-structured objects are generated, where each third-structured object includes a corresponding subquery statement. The first-structured object includes the aforementioned K third-structured objects. Specifically, if the search condition includes K subqueries, the query1 object (i.e., the query statement object) converted based on the search condition can include K query2 objects (i.e., subquery statement objects), where the aforementioned K query2 objects are sub-objects stored in the nested query1 object. Furthermore, the independent structuring of each subquery allows the search engine to process each subquery individually. (Additionally, multiple conditions can be combined within a query to filter data; for example, logical operators such as "OR" and "AND" can be used to combine multiple subqueries into a single query.) This enables nested queries, retrieving sub-documents within nested objects and returning parent documents that are matched by the sub-document search criteria, thus improving the relevance and accuracy of search results. Simultaneously, breaking down complex queries into combinations of subqueries allows the search engine to employ different strategies and algorithms for different parts of the query conditions, enhancing its adaptability to complex information needs and its ability to handle complex queries. This contributes to improved information retrieval efficiency, especially when dealing with queries containing logical operators (such as AND / OR / NOT) or nested queries, resulting in more accurate search results.

[0117] Step S301: Create a scorer based on the first structured object.

[0118] Specifically, a scorer is created based on a first structured object. This scorer may include the first structured object, M document identifiers matching the query statement, and one or more scoring components, where M is a positive integer greater than 0. After the search engine parses the query statement in the search conditions to generate a query statement object (i.e., the first structured object), a scorer (e.g., the scorer in the Lucene search engine) is created based on this query statement object. This scorer may include M document identifiers matching the query statement object (i.e., the IDs of candidate documents) and scoring components (e.g., the scoring components in the scorer). For example, subsequently, the corresponding M candidate documents can be found based on the above M document identifiers, and then the scorer can be used to score and filter the above M candidate documents.

[0119] In one possible implementation, K corresponding sub-scorers are created based on K third-structured objects. Each sub-scorer includes a corresponding third-structured object, L document identifiers matching the corresponding third-structured object, and a scoring component. A scorer is created based on these K corresponding sub-scorers, where L is a positive integer greater than 0. Specifically, in this embodiment, a corresponding sub-scorer can be created based on each sub-query statement in the query statement. These sub-scorers also include the corresponding sub-query statement, scoring component, and document identifier. The scorer generated based on the query statement includes all sub-scorers generated based on the sub-query statements. For example, please refer to [link to example]. Figure 4A Step 3 in the provided search process diagram is related to... Figure 4B A schematic diagram of a scorer structure is provided. If the query1 object converted from parsing search conditions is a nested object (i.e., the query1 object is a nested object structure including N query2 objects, where the N query2 objects are sub-objects generated based on sub-queries in the query statement), a corresponding parentscorer is first created based on the query1 object. The parentscorer can include the query1 object, N childscorers (i.e., sub-scorers) generated based on the query2 objects, the ID information of each candidate document in the parentscorer, and a scoring component. Furthermore, a corresponding sub-scorer can be created for each sub-query statement. The sub-scorers exist in the corresponding main scorer (i.e., the scorer corresponding to the first structured object generated based on the query statement). Each sub-scorer operates on a specific query segment (e.g., it can score and sort each individually), enabling the search engine to better interpret and optimize query results, while improving the ability to handle nested queries and multi-condition queries.

[0120] Step S302: Obtain M first documents that match the M document identifiers, and generate rating description information for each of the M first documents through the rating component.

[0121] Specifically, the search engine can find M corresponding candidate documents based on M document identifiers (for example, it can find the candidate document corresponding to each ID from the inverted index), and then obtain the rating description information corresponding to each candidate document based on the rating component (for example, the rating description information can include hit information and rating information; the hit information can be one or more of the hit words, hit fields, and hit positions that match the search conditions obtained by the rating component based on each candidate document; the rating information can be the rating of the matching degree between each candidate document and the query statement object by the rating component).

[0122] In one possible implementation, the search engine includes an inverted index, which comprises a mapping between document identifiers and their corresponding documents. The process involves retrieving M first documents matching M document identifiers, and searching the inverted index for the M first documents corresponding to the M document identifiers. Specifically, in the search engine, data retrieval can be achieved through a query object (query1) generated from a query statement. A key step in this process is to use the identifiers (i.e., IDs) of candidate documents in the inverted index. The inverted index is a data structure in Lucene where each document has a unique ID, serving as its identification code within the document collection. For example, after the scorer identifies candidate documents matching the query conditions in Lucene, it can use the ID of each document to retrieve the specific candidate document from the inverted index. Furthermore, using these IDs, the scorer can query the index and retrieve document information related to each keyword in the query statement, including their storage location, the frequency and distribution of keywords within the documents, etc. This information can be used as a scoring component in the scorer to calculate the relevance score of the document, thereby determining its ranking in the search results.

[0123] In one possible implementation, the rating description information corresponding to each first document includes hit information and rating information; the rating component generates the rating description information corresponding to each of the M first documents, including: for the scorer, selecting Q first documents with nested structures and J first documents with non-nested structures from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q; the rating component generates the hit information and rating information corresponding to each of the M first documents, where the rating information is the score of the matching degree between the corresponding first document and the search conditions. Specifically, in this embodiment, the scorer can be used to identify nested and non-nested document structures (i.e., when processing documents containing multi-level data, such as nested JSON or XML documents, the scorer can distinguish and identify whether the document is a nested structure), which allows the search engine to process complex queries more accurately. Furthermore, the scorer can generate score description information (i.e., hit information and score information) for each candidate document through the scoring component in the scorer, and the score information is a score based on the matching degree between the corresponding candidate document and the search conditions, thereby improving the accuracy of search results. For example, the score for each candidate document can be obtained by the scoring component in the scorer based on the hit information of the corresponding candidate document obtained from the inverted index, and each candidate document is scored based on this hit information. For example, it can be calculated using TF-IDF or BM25 to determine the relevance score of a document to a query. Further, the scoring component in the scorer first creates weights for multiple terms included in each search condition in the query statement. Based on the weight of the terms, their frequency in the document, document length, and other relevance signals, the scoring component scores each candidate document one by one until all relevant documents are scored.

[0124] In one possible implementation, the hit information in the J first documents includes one or more of the following: hit fields, hit words, and hit locations that match the search criteria in the corresponding first documents. Specifically, for each candidate document that matches the query statement, the hit information for each candidate document may include, but is not limited to, one or more of the following: hit fields (e.g., fields in the candidate document where the query word appears), hit words (e.g., words or phrases that actually match the query statement), and hit locations (e.g., the specific location of the hit words parsed from the query statement in the candidate document or in a specific field). By collecting the hit information for each candidate document, this application embodiment can indicate the specific location of the query word in the document based on the hit information, and can also be used to calculate scoring information together with other information in the matching situation (e.g., document frequency (DF), term frequency (TF), document length, query word weight, etc.), and the hit information can be used as a kind of explanatory information to indicate the specific hit situation of the query statement in the corresponding document.

[0125] In one possible implementation, the first document, which has a nested structure, includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, a set of document identifier sequences of one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document. Specifically, first, it is confirmed whether the candidate documents include nested candidate documents (i.e., second documents). In search engines, documents may have complex nested structures to support richer information expression. For example, a report may contain multiple chapters, each containing multiple paragraphs; or a video may contain multiple paragraphs or highlights, each containing multiple images. Such nested documents can support richer content. In this embodiment, a nested document can be a nested structure including a main document and multiple sub-documents. If Q candidate documents are nested documents, the hit information that the scorer can obtain is modified so that each scorer (e.g., a scorer generated by a query statement or a sub-scorer generated by a sub-query statement) can obtain the ID of the main document (i.e., document identifier), a set of ID sequences of one or more sub-documents, the ID of the hit sub-document in the main document, and the hit position of the hit sub-document as hit information in each candidate document; further, please refer to Figure 4A Step 3 in the schematic diagram of a search process shown, and Figure 4C The diagram shows another scoring unit structure, specifically... Figure 4CIn the scorer shown, the main document IDs of frames.ocr_text that are hit are 10 and 20. The sub-document IDs with ID 10 are 7, 8, and 9, where 8 is the sub-document ID hit by frames.ocr_text. By parsing this information, it can be determined that the condition hits the second nested frame object in the main document. In this embodiment, by modifying the scorer, explanation information for complex nested documents can be obtained, allowing this embodiment to be applied to more application scenarios. It is understood that the nested document can be a nested structure composed of multiple layers of text, a nested document composed of each specific frame included in the video, or a rich text type nested structure. This embodiment does not specifically limit this, as long as the document can be shown to be a nested structure.

[0126] Step S303: Based on the rating description information and the first structured object corresponding to each first document, generate M second structured objects.

[0127] Specifically, please see Figure 4A Step 4, before collecting candidate documents from each scorer (i.e., before sorting and filtering the scoring information corresponding to each candidate document), generates a corresponding explanation information object (i.e., a second structured object) by combining the scoring description information of each candidate document with the corresponding query statement object (e.g., a first structured object generated by the query statement, or a third structured object generated by the subquery statement) as detailed information (e.g., explanation information). For example, each explanation information object can correspond to a candidate document, and each explanation information object can include the query object (i.e., the first or third structured object) converted from the query statement corresponding to the scorer, the scoring information of the corresponding candidate document, and the hit information. This embodiment of the application, by storing the explanation information of each candidate document as an explanation object during the data retrieval process, eliminates the need for multiple explanation queries when explanation information is subsequently required; the explanation information can be obtained simply by finding the explanation query object corresponding to the target candidate document.

[0128] In one possible implementation, based on the Lucene search engine, the storage method for detailed information (such as explanatory information) objects of complex nested documents can be as follows: nested sub-objects can be stored as independent index documents, with their fields constructed in the format of "parent field."child field; in this way, the sub-objects also have independent retrieval capabilities; please refer to [link to relevant documentation]. Figure 4DThe diagram illustrates a nested document structure. When it is necessary to retrieve the parent document through a child document, the child document can be retrieved first by searching fields such as frames.tag_name. Then, the first parent document (i.e., the parent document of this record) is searched backward. In the fetcher, the nested objects in the frames field are restored sequentially backward by using the position of the parent document and the number of child documents recorded in the frames field.

[0129] Optionally, after the above method steps S300-S303, in the process of data retrieval, obtain detailed information (such as explanatory information) for each candidate document and generate a corresponding structured object based on the explanatory information of each candidate document, the following steps can be referred to... Figure 4E Before collecting hit documents during the search process, an interpretation collector is injected to collect various hit conditions for each document, and an interpretation object is constructed in the final fetch stage. Compared to existing technologies, interpretations can be obtained simultaneously with a single search, such as... Figure 3B As shown, it may also include the following steps S400-S403:

[0130] Step S400: Save the second structured object of the candidate documents in each scorer to a preset storage component.

[0131] Specifically, see Figure 4A Step 4 involves acquiring detailed information (such as explanation information) for each candidate document during data retrieval. After generating the corresponding structured object, each explanation information object (i.e., the second structured object) can be saved to the `explainquerycolletcor` (a pre-defined storage component). Once all candidate documents are collected, matching documents that meet the preset criteria can be selected based on their scores. By pre-storing the explanation information of all candidate documents before selecting matching documents, the required explanation information can be retrieved through the `explainquerycolletcor` when outputting search results. This eliminates the need to perform explanation queries on each matching document in the search results, thus avoiding resource waste and reducing latency.

[0132] Step S401: Based on the results of the first sorting, select N first documents whose scoring information meets the preset conditions from the first documents corresponding to each scorer.

[0133] Specifically, for each scorer, the corresponding first documents are sorted based on the score description information. Based on the first sorting result, N first documents whose score description information meets preset conditions are selected from the first documents corresponding to each scorer, where N is a positive integer less than or equal to M. For example, after the scoring component calculates the score information of the candidate documents in each scorer (e.g., the scorer in Lucene), the corresponding candidate documents are sorted according to the score information, placing the more relevant candidate documents first. Then, based on preset conditions, the candidate documents in each scorer are filtered to select N first documents whose score information meets the preset conditions, where N is a positive integer less than or equal to M. Through this process, documents with higher scores, better matching the search conditions, and meeting the preset conditions are selected from the original candidate document set as the hit documents. For example, if the business only needs the top ten most accurate documents as output results, all candidate documents can be sorted based on the score information, and then pagination can be set to display only the top ten, thus meeting the business requirements. In this embodiment of the application, by comparing with preset scoring conditions, the search engine can effectively narrow the search scope, improve search efficiency and the accuracy of results, thereby meeting more business needs.

[0134] Step S402: Based on the document identifiers of the N first documents, retrieve the corresponding N second structured objects from the storage component and generate N corresponding structured information.

[0135] Specifically, after sorting the candidate documents based on their rating information, the candidate documents with the highest relevance to the corresponding query are displayed first. Then, the N candidate documents with the highest relevance are selected as the hit documents based on preset conditions. For example, see [link to example]. Figure 4A In step 6, after filtering out the matching documents that meet the business requirements through scoring, the corresponding matching document can be found from the explainquerycollector by its ID during the fetch stage (i.e., the stage of retrieving paginated results). For example, see... Figure 4AIn step 6, during the fetch phase (i.e., the phase of retrieving paginated results), since the document ID and its corresponding explanation query stored in the explanation query collector are a dictionary structure (e.g., a hash set), with the key being the ID and the value being the explanation query, it is not necessary to iterate through each explanation query during the fetch phase. Instead, the corresponding explanation query can be found directly by the document ID, and then the explanation information can be extracted from each explanation query (e.g., it may include query objects constructed based on the corresponding search conditions, hit fields, hit words, source query, score, the location of the hit document, or the location of nested hit documents). The extracted explanation information is organized according to a specific format (e.g., XML, JSON, or database tables). If there is a hierarchical or linked relationship between the target hit documents (e.g., the hit document is a nested hit document), these relationships are correctly expressed in the structured information, which better describes the explanation information. Downstream businesses can then use this information to perform more advanced logic.

[0136] In one possible implementation, based on the structured information constructed from the detailed information (such as explanatory information) of each hit document, the search results page can display detailed information about the record (such as the reason for the hit, hit details, detailed content, explanatory information, etc.). For example, when the searched image is hit because of text, a text recognition badge can be displayed; when the searched image is hit because of a tag, a tag badge can be displayed; when the image is hit because of a location, a location badge can be displayed. Taking contacts as an example, the search results display often does not have enough space to include the contact's index information. The information is displayed in this way, so through the explanation, the corresponding data can be displayed according to different hit reasons during the display. The explanation includes the hit field and the hit word. Through this information, the user can obtain the position information of the hit word in the original text. Through this information, the hit word can be highlighted during the search display to prompt the user. For cases where the input word is not hit in the original text, the explanation provides the position information of the hit word in the original text. For example, hit scenarios such as first letter, pinyin, synonyms, etc. Compared with the traditional string containing explanation information, which cannot accurately display the hit reason, this application embodiment can achieve highlight prompts.

[0137] In one possible implementation, more advanced display effects can be applied through the interpretation of nested objects. Specifically, for video and highlight moment retrieval; a video can contain multiple nested sub-segments, and a highlight moment includes multiple sub-photos and sub-video clips; the content of each nested sub-segment in the video can be searched; for example, tags, semantic vectors, subtitles, etc., can be returned for the video; at the same time, interpretation will return which field and which attribute hit the target; further, assuming there is a video, when a user searches for the word "protect" in a certain text in a certain frame of the video based on local data, if the keyword exists in a certain segment of a video, the video containing the keyword can be returned, and through interpretation, it can be found that it hits the second segment of the video; in this way, the hit position found above can be directly displayed as the cover, and playback can continue from that position; compared with existing video retrieval, the cover of the hit position and the hit content can be dynamically displayed, realizing the jump function.

[0138] For rich text structure retrieval: the rich text structure can be saved as a complex object containing paragraphs, tables, images, etc. When a search term is executed, the rich text document can be returned along with detailed information (explanatory information, such as which table, paragraph, or image in the rich text was matched). This information allows users to directly jump to the corresponding location. For note-taking applications, a similar rich text structure can be used. Notes contain multiple elements: titles, body text, handwriting, attachments, tables, audio recordings, images, etc., which can be constructed as a complex nested object. The search can search all elements of the note, and when a match is found, the explanation reveals which element was matched. This allows the user to directly locate the specified position and receive a prompt when the results are displayed.

[0139] Step S403: Perform a second sorting on the N first documents based on the N corresponding structured information.

[0140] Based on N second structured objects, generate N corresponding structured information.

[0141] Specifically, in one possible implementation, secondary filtering and sorting can be performed using detailed information (such as explanatory information). The explanatory information can include which search condition the record originated from and the score of that condition. For example, in the field of semantic retrieval, vector calculations are often used to obtain the similarity score between the search term and the hit documents. After recall, the similarity score for the specified semantic retrieval condition can be obtained through explanation. Based on this score, secondary sorting and relevance filtering can be performed. Alternatively, secondary sorting of hit documents can be performed using weighted percentage scores. For instance, when a user searches for "ni," the explanation reveals that two title fields, title1 and title2, are hit. Secondary sorting can be performed by comparing the length percentage of the hit word "ni" in title1 and title2, placing documents with shorter titles at the top.

[0142] based on Figures 3A-3B For the steps of the embodiments described herein, please refer to Figure 4E When a search engine receives a user's search query, it parses the query and determines whether it is a nested query. Nested queries retrieve child documents within nested objects and return the parent document found due to the child document search criteria. Non-nested queries retrieve non-nested objects or portions of the parent document of nested objects. The following sections will detail the cases of nested and non-nested queries.

[0143] Nested queries may include the following steps S511-S517:

[0144] Step S511: The search engine parses the query statement contained in the search conditions. The query statement may include one or more subquery statements. The independent structuring of each subquery statement allows the search engine to process each subquery statement separately. In addition, multiple conditions can be combined in the query statement to filter data. For example, logical operators such as "OR" and "AND" can be used to combine multiple subquery statements to form a combined query.

[0145] Step S512: Generate a query statement object query1 that the search engine can understand and execute based on the query statement (e.g., a structured object converted from the query statement contained in the search conditions by the Lucene search engine), and generate a corresponding subquery statement object for each subquery statement. The query statement object query1 can include multiple subquery statement objects query2. The main document matched by the query statement object query1 can be a sub-document hit by multiple subquery statement objects query2. The first parent document found forward is the parent document of the search record, so as to obtain a list of candidate documents.

[0146] Step S513: Create a corresponding child scorer for each subquery statement. The parent scorer (i.e., the scorer based on the query statement object query1) includes the child scorer. Each scorer in the parent scorer and the child scorer can include the query statement object of each candidate document and the document identifier (i.e., document ID) of each candidate document. The corresponding candidate document is obtained based on each document identifier. The scoring component generates corresponding scoring description information based on each candidate document (e.g., it can include the hit field, hit words, score information, and the location of the hit object (nested sub-object)).

[0147] Step S514: Generate a corresponding explanation information object explainquery by using the rating description information of each candidate document and the corresponding query object (e.g., the first structured object generated by the query statement, or the third structured object generated by the subquery statement) as explanation information.

[0148] Step S515: Before collecting candidate documents in each scorer (e.g., the parent scorer or the child scorer), (i.e., before sorting and filtering the scoring information corresponding to each candidate document), store each explanation information object explainquery in the scorer into the preset storage component Explain query collecor.

[0149] Step S516: After sorting the candidate documents based on the rating information, the candidate documents with the highest relevance to the corresponding query statement are displayed first. Then, the N candidate documents with the highest relevance are selected as the hit documents according to the preset conditions.

[0150] Step S517: In the output search results stage, based on the ID of the document that hit the search results, the corresponding explanation information object explanationquery is retrieved from the above-mentioned preset storage component Explain query collecor, and the corresponding structured information of the explanation is generated. Compared with the storage method of explanation information in the prior art, which concatenates the explanation information into a string and cannot be directly used, the explanation information in this embodiment is constructed as structured information, which is more convenient for downstream business to directly use.

[0151] Non-nested queries may include the following steps S521-S527:

[0152] Step S521: The search engine parses the query statements contained in the search criteria.

[0153] Step S522: Generate a query object (e.g., the first structured object that the Lucene search engine converts into a query statement contained in the search conditions) based on the query statement. The query object can only query documents of non-nested objects or the parent document portion of nested objects to obtain a list of candidate documents.

[0154] Step S523: Generate a corresponding scorer based on the query object query. The scorer may include the query object query of each candidate document and the document identifier (i.e. document ID) of each candidate document. Obtain the corresponding candidate document based on each document identifier. The scoring component generates corresponding scoring description information based on each candidate document (e.g., may include hit fields, hit words, score information, and the location of the hit object (nested sub-object)).

[0155] Step S524: Generate a corresponding explanation information object explainquery by using the rating description information of each candidate document and the corresponding query statement object query (e.g., the first structured object generated by the query statement, or the third structured object generated by the subquery statement) as explanation information.

[0156] Step S525: Before collecting candidate documents in each scorer (i.e. before sorting and filtering the scoring information corresponding to each candidate document), store each explanation information object explainquery in the scorer into the preset storage component Explain query collecor.

[0157] Step S526: After sorting the candidate documents based on the rating information, the candidate documents with the highest relevance to the corresponding query statement are displayed first. Then, the N candidate documents with the highest relevance are selected as the hit documents according to the preset conditions.

[0158] Step S527: In the output search results stage, based on the ID of the document that hit the search results, the corresponding explanation information object explanationquery is retrieved from the above-mentioned preset storage component Explain query collecor, and the corresponding structured information of the explanation is generated. Compared with the storage method of explanation information in the prior art, which concatenates the explanation information into a string and cannot be directly used, the explanation information in this embodiment is constructed as structured information, which is more convenient for downstream business to directly use.

[0159] For ease of understanding Figure 4EThe embodiments described herein, and the following exemplary examples illustrate the scenarios in which the data retrieval method in this application is applied, which may include the following three scenarios:

[0160] Scenario 1: On the search results page, the reason why the record was found is displayed:

[0161] Because the detailed information (such as explanatory information) obtained in this embodiment is structured rather than concatenated strings, the required explanatory information can be obtained more effectively. After retrieval based on search criteria, detailed information (such as the reason for the hit, the hit details, the detailed content, and the explanatory information) can be more accurately annotated. For example, when the user enters the search criteria "hello", the final search result contains images that include that text. The searched images (i.e., a hit document in the search results) are hit because of the text (i.e., the first or third structured object generated based on the search criteria). When the user enters the search criteria "happy", if the searched images (i.e., hit documents in the search results) are hit because of tags, a tag badge can be displayed because the search engine detected that the image was hit because of tags related to "happy". This badge can be a small icon or a visual cue indicating that the image contains the text the user searched for. This approach enhances the intuitiveness of the user interface, allowing users to easily identify which images were retrieved based on tag information. Similarly, if an image is found because its location matches the search criteria—for example, if a user searches for a specific geographical location—a "location badge" will be displayed next to the image in the search results. This badge typically takes the form of a map pin or a location icon, indicating that the image is related to a specific location. Location badges help users identify images that are relevant to their search criteria due to geographical information, providing a direct visual association that shows the image content has a real connection to a particular location.

[0162] Furthermore, in address book applications, taking a search by contact name as an example, the search results often lack sufficient space to display the contact's index information. Therefore, by providing explanatory information, the displayed data can be tailored to different reasons for the match. For instance, if a user searches for the name "Zhang San," the search engine might return contact records containing that name (i.e., matching documents in the search results). Next to each record, the system can display a small explanatory message (structured information generated based on a second structured object), such as "Name field match: Zhang San," meaning that the matching word "Zhang San" appears in the "Name" field of the contact record. Through this explanatory information, users not only understand why the record was retrieved but also the specific location of the matching word "Zhang San" in the original contact data. In addition, the search engine can further emphasize this match through highlighting. In the search results display, the name "Zhang San" can be highlighted to distinguish it from other text, prompting the user.

[0163] Scenario 2: Based on the explanatory information, perform secondary filtering and sorting on the hit documents:

[0164] When the explanatory information is more comprehensive and structured, the secondary sorting can better utilize this structure and obtain more explanatory information for in-depth filtering, so as to filter out documents that match the query conditions. In this embodiment, the explanatory information (i.e., the structured information of the explanatory information generated based on the second structured object) may include which search condition the record or the nested document in the record originates from (i.e., which query or subquery statement matched the document of a certain nested structure in the search results), and the scoring information of the condition. In the field of semantic retrieval, the similarity score between the search term and the matched document can be calculated using vectors. After recall, the similarity score of the specified semantic retrieval condition can be obtained through explanation. Based on the score, secondary sorting and relevance filtering can be performed, thereby improving the accuracy of data retrieval. For example, when the search condition such as "renewable energy" is obtained from the explanatory information, the system will convert the search condition into a semantic vector. Next, the system calculates the similarity score between the vector and the document vectors stored in the database, which have also been converted into vector form. This score reflects the semantic similarity between the search term and the document, with a high score indicating a high relevance. Once all documents have been assigned a similarity score relative to the search term, the system performs a "recall" operation, selecting a set of documents with the highest scores to form the initial search results set. After the recall phase, the search results can be further sorted and filtered for relevance.

[0165] The search results can also be further sorted and filtered based on the weight of the proportion of the hit words included in the explanatory information within the hit field. For example, if two documents, A and B, both contain "ni" in their title field (i.e., the search condition), but document A's title is "Nike Shoes" (i.e., the hit field in document A), while document B's title is "NikonCamera Lens" (i.e., the hit field in document B), then document A will be given a higher ranking priority because "ni" has a larger proportion in its title. This sorting method enhances the accuracy of the search results, ensuring that when faced with numerous documents containing short query terms, the most relevant content can be found more quickly.

[0166] Scenario 3 is suitable for obtaining explanatory information in complex business scenarios:

[0167] Compared to existing technologies, the embodiments of this application can obtain explanatory information from documents with nested structures. Therefore, it can be applied to complex business scenarios to obtain explanatory information. For nested documents, it can specifically locate the root cause of the hit. For example, when searching for videos, highlights, etc., a video may include multiple nested sub-segments, and a highlight may include multiple sub-photos and sub-video clips. The embodiments of this application can return the video by searching the content of each nested sub-segment in the video, such as tags, semantic vectors, subtitles, etc. At the same time, the explanation will return the specific attribute of which field hit the target. Figure 4F As shown, when searching for a search term (i.e., the first structured object generated by parsing the search criteria), the rich text document can be returned, along with the number of tables, paragraphs, and images in the rich text that were hit (i.e., the hit position in the interpreted structured information). With this information, you can directly jump to the corresponding position.

[0168] like Figure 4G As shown, note-taking applications can be analogous to the rich text structure above. A note contains multiple elements: title, body text, handwriting, attachments, tables, audio recordings, images, etc., which can be constructed into a complex nested object (i.e., a nested structured document). The search can search all elements of the note. When a match is found, the explanatory information (i.e., the structured information of the explanatory information) tells us which element was matched and which position it is. In this way, when displaying the note, we can directly locate the specified position (i.e., the match position in the match information) and provide a prompt.

[0169] like Figure 4HAs shown, when we search for "protection red line" (i.e., the first structured object constructed based on the search criteria), we can return the video (i.e., the main document in the nested structure's hit document). At the same time, through the explanation information, we can know that it is the text of the second segment of the video that is hit (i.e., the sub-document in the nested structure's hit document). In this way, the cover shown above can be displayed directly, and clicking it can continue playing from that position. Compared with existing video retrieval, it can dynamically display the cover of the hit position and the hit content.

[0170] It is understood that the above three application scenarios are only a few exemplary implementation methods in the embodiments of this application, and the application scenarios in the embodiments of this application include, but are not limited to, the above application scenarios.

[0171] The methods of the embodiments of this application have been described in detail above. The related apparatus of the embodiments of this application is provided below.

[0172] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a search engine device provided in an embodiment of this application. The search engine device 01-C may include a first parsing unit 601, a first creation unit 602, an information acquisition unit 603, a first generation unit 604, a first sorting unit 605, a filtering unit 606, an extraction unit 607, an information generation unit 608, and a second sorting unit 609. The detailed description of each unit is as follows.

[0173] The first parsing unit 601 is used to receive search conditions and generate a first structured object, wherein the first structured object includes the query statement contained in the search conditions.

[0174] The first creation unit 602 creates a scorer based on the first structured object. The scorer includes the first structured object and M document identifiers that match the first structured object, where M is a positive integer greater than 0.

[0175] In one possible implementation, the first creation unit 602 is specifically used for:

[0176] K corresponding sub-scoring units are created based on the K third structured objects. Each sub-scoring unit includes a corresponding third structured object, L document identifiers that match the corresponding third structured object, and the scoring component.

[0177] The scorer is created based on the K corresponding sub-scorers, where L is a positive integer greater than 0.

[0178] The information acquisition unit 603 acquires M first documents that match the M document identifiers, and generates rating description information corresponding to each of the M first documents through the rating component;

[0179] In one possible implementation, the search engine includes an inverted index that includes a mapping between document identifiers and corresponding documents;

[0180] The information acquisition unit 603 is specifically used for:

[0181] Find the M first documents corresponding to the M document identifiers from the inverted index.

[0182] In one possible implementation, the rating description information corresponding to each of the first documents includes hit information and rating information;

[0183] The information acquisition unit 603 is specifically used for:

[0184] For each scorer, Q first documents with nested structures and J first documents with non-nested structures are selected from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q.

[0185] The scoring component generates hit information and score information for each of the M first documents, where the score information is a score indicating the degree of matching between the corresponding first document and the search criteria.

[0186] In one possible implementation, the hit information in the J first documents includes one or more of the hit fields, hit words, and hit locations that match the search criteria in the corresponding first documents.

[0187] In one possible implementation, the first document, which is a nested structure, includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, a set of document identifier sequences of the one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document.

[0188] The first generation unit 604 generates M second structured objects based on the rating description information corresponding to each first document and the first structured object, wherein one second structured object corresponds to one first document;

[0189] In one possible implementation, the first generating unit 604 is specifically used for:

[0190] Obtain the K subqueries included in the query statement, where K is a positive integer greater than 0;

[0191] Based on the K subqueries, K third structured objects are generated, wherein each third structured object includes a corresponding subquery, and the first structured object includes the K third structured objects.

[0192] The first sorting unit 605 sorts the corresponding first document based on the scoring description information for each scorer.

[0193] The filtering unit 606, based on the result of the first sorting, filters out N first documents from the first documents corresponding to each scorer whose scoring description information meets the preset conditions, where N is a positive integer less than or equal to M;

[0194] Extraction unit 607 extracts N corresponding second structured objects from the storage component based on the document identifiers of the N first documents;

[0195] The information generation unit 608 generates N corresponding structured information items based on N second structured objects.

[0196] The second sorting unit 609 performs a second sorting on the N corresponding structured information on the N first documents, and outputs the sorting results to the search results.

[0197] In one possible implementation, the second sorting unit is specifically used for:

[0198] Based on the similarity score between each of the N structured information pieces and the corresponding first document, the N corresponding first documents are sorted in a second manner.

[0199] Alternatively, based on the percentage score of the hit words in each of the N structured information pieces in the corresponding hit field, the N corresponding first documents are sorted in a second manner.

[0200] It should be noted that the functions of each functional unit in the search engine device 01-C described in this application embodiment are as described above. Figure 3A The descriptions of steps S300-S302 in the embodiments of the method described above, and Figure 3B Steps S400-S403 in the method embodiment described herein will not be repeated here.

[0201] The terminal device provided in the embodiments of this application is described below.

[0202] Figure 6A A schematic diagram of the hardware structure of a terminal device 03 provided in an embodiment of this application is shown. This terminal device 03 is used to execute the image recommendation method provided in the preceding method embodiments.

[0203] Terminal device 03 may include a processor 101, a memory 102, a wireless communication module 103, a mobile communication module 104, an antenna 103A, an antenna 104A, a power switch 105, a sensor module 106, a focusing motor 107, a camera 108, a display screen 109, etc. The sensor module 106 may include a gyroscope sensor 106A, an accelerometer sensor 106B, an ambient light sensor 106C, an image sensor 106D, a proximity sensor 106E, etc. The wireless communication module 103 may include a WLAN communication module, a Bluetooth communication module, etc. All of the above components can transmit data via a bus.

[0204] Processor 101 may include multiple processing units to optimize the performance of the Lucene search engine when processing local data retrieval. For example, these processing units may include a main application processor (AP) responsible for complex data processing tasks, a graphics processing unit (GPU) to optimize the display of search results, and specially designed digital signal processors (DSPs) and neural network processors (NPUs) to accelerate the execution of search algorithms. These units work together to improve the efficiency and accuracy of the Lucene search engine in processing queries, indexing, and ranking search results. Furthermore, this multi-processing-unit design allows the Lucene search engine to process multiple tasks in parallel, thereby providing fast response and accurate search results when processing large datasets.

[0205] Memory 102 stores the operating system, application code, and other data of the terminal device 03. For example, it may include program code and data required by local applications (such as various business applications, games, or utility applications) and the Lucene search engine. Processor 101 implements device functions and data processing tasks by executing the program code stored in memory 102. The program storage area of ​​memory 102 is used to house the operating system and application code, while the data storage area is used to store data generated during application execution. In addition to containing fast-access random access memory (RAM), memory 102 may also include one or more forms of non-volatile memory, such as hard disk drives, solid-state drives, or flash memory devices, to provide a persistent storage solution.

[0206] The wireless communication function of the terminal device 03 can be implemented through antenna 103A, antenna 104A, mobile communication module 104, wireless communication module 103, modem processor, and baseband processor.

[0207] Antennas 103A and 104A can be used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 03 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0208] The mobile communication module 104 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 03. The mobile communication module 104 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 104 can receive electromagnetic waves via antenna 104A, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 104 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 104A.

[0209] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device or displays an image or video through the display screen 109.

[0210] The wireless communication module 103 can provide solutions for wireless communication applications on the terminal device 03, including wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). The wireless communication module 103 can be one or more devices integrating at least one communication processing module. The wireless communication module 103 receives electromagnetic waves via antenna 103A, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 101. The wireless communication module 103 can also receive signals to be transmitted from processor 101, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 103A.

[0211] The power switch 105 can be used to control the power supply to the terminal device 03.

[0212] The gyroscope sensor 106A can be used to determine the motion posture of the terminal device 03. In some embodiments, the gyroscope sensor 106A can determine the angular velocity of the terminal device 03 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 106A can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 106A detects the angle of the shaking of the terminal device 03, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shaking of the terminal device 03 through reverse movement, thus achieving image stabilization. The gyroscope sensor 106A can also be used in navigation and motion-sensing game scenarios.

[0213] Accelerometer 106B can detect the magnitude of acceleration in various directions (generally three axes) of terminal device 03. When terminal device 03 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of terminal device. For example, accelerometer 106B can be applied to applications such as landscape / portrait screen switching and pedometers.

[0214] The ambient light sensor 106C is used to sense the ambient light intensity. The terminal device 03 can adaptively adjust the brightness of the display screen 109 according to the sensed ambient light intensity. The ambient light sensor 106C can also be used to automatically adjust the white balance when taking pictures.

[0215] The 106D image sensor, also known as a photosensitive element, uses the photoelectric conversion function of an optoelectronic device to convert a light image on a photosensitive surface into an electrical signal proportional to the light image. Image sensors can be either charge-coupled device (CCD) sensors or complementary metal-oxide-semiconductor (CMOS) sensors.

[0216] The distance sensor 106E can be used to measure distance. The terminal device 03 can measure distance via infrared or laser. In some shooting scenarios, the terminal device 03 can use the distance sensor 106E to measure distance for fast focusing.

[0217] The focusing motor 107 can be used for rapid focusing. The terminal device 03 can control the movement of the lens via the focusing motor 107 to achieve autofocus.

[0218] Terminal device 03 can perform shooting functions through ISP, camera 108, video codec, GPU, display 109 and application processor.

[0219] The ISP is used to process data fed back by the camera 108. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 108.

[0220] Camera 108 can be used to capture still images or videos. An object is projected onto an image sensor through the lens, generating an optical image. The image sensor converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP can output the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, terminal device 03 may include one or N cameras 108, where N is a positive integer greater than 1.

[0221] Video codecs are used to compress or decompress digital images. Terminal device 03 can support one or more image codecs. Thus, terminal device 03 can open or save images or videos in various encoding formats.

[0222] Terminal device 03 can implement display functions through a GPU, display screen 109, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 109 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 101 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0223] The display screen 109 is used to display images, videos, etc. The display screen 109 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 03 may include one or N display screens 109, where N is a positive integer greater than 1.

[0224] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal device 03. In other embodiments of this application, the terminal device 03 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0225] For details on the operations performed by each device in terminal device 03, please refer to [link / reference]. Figures 3A-3B The relevant descriptions of the embodiments are not elaborated here.

[0226] The software system of terminal device 03 can adopt one or more of the following architectures: layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, and can support local search functionality based on the Lucene search engine. This application embodiment uses a layered architecture mobile operating system as an example to exemplify the software structure of terminal device 03.

[0227] Figure 6B This is a software structure block diagram of the terminal device 03 according to an embodiment of this application.

[0228] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, a mobile operating system is divided into four layers, from top to bottom: the application layer, the application framework / core service layer, the system libraries and runtime, and the kernel layer.

[0229] The application layer can include a series of application packages.

[0230] like Figure 6B As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0231] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0232] like Figure 6B As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0233] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0234] Content providers can be used to store and retrieve data, making this data accessible to applications. This data can then be used to access and index local data such as phone books, browsing history, and bookmarks, providing a data source for Lucene. The data can include videos, images, audio, made and received phone calls, browsing history, bookmarks, phone books, etc.

[0235] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0236] A phone manager is used to provide communication functions for terminal devices. For example, it manages call status (including connection and disconnection).

[0237] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0238] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the device, and flashing indicator lights.

[0239] Runtime can refer to all the code libraries, frameworks, etc., required for a program to run. For example, it provides the Java Virtual Machine and core libraries, which provide the necessary environment for the Lucene search engine to run.

[0240] System libraries can include multiple functional modules. Examples include: a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL). Lucene can utilize these libraries to index and search image, audio, and video files on the device.

[0241] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0242] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0243] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0244] A 2D graphics engine is a graphics engine for 2D drawing.

[0245] The kernel layer is the layer between hardware and software. At a minimum, it includes display drivers, camera drivers, audio drivers, and sensor drivers. For example, it can provide low-level hardware support for search engines, enabling the Lucene search engine to access and index files in the file system.

[0246] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0247] This application also provides a terminal device, which may include a memory and a processor. The memory may be used to store computer programs; the processor may be used to invoke the computer programs in the memory, so that the terminal device executes the methods executed on the terminal device side in any of the above embodiments.

[0248] This application also provides a terminal device, which may include a memory and a processor. The memory may be used to store computer programs; the processor may be used to invoke the computer programs in the memory, so that the terminal device executes the methods executed on the terminal device side in any of the above embodiments.

[0249] This application also provides a chip system, which includes at least one processor for implementing the functions involved on the terminal device side in any of the above embodiments.

[0250] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0251] The chip system can consist of chips or include chips and other discrete components.

[0252] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0253] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0254] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0255] This application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method executed on the terminal device side in any of the above embodiments.

[0256] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed on the terminal device side in any of the above embodiments.

[0257] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0258] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0259] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0260] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A data retrieval method, characterized in that, Applied to search engines, the method includes: Receive search criteria and generate a first structured object, wherein the first structured object includes the query statement contained in the search criteria; A scorer is created based on the first structured object. The scorer includes the first structured object, M document identifiers that match the query statement, and a scoring component, where M is a positive integer greater than 0. Obtain M first documents that match the M document identifiers, and generate rating description information for each of the M first documents through the rating component; Based on the rating description information corresponding to each of the first documents and the first structured object, M second structured objects are generated, wherein one second structured object corresponds to one first document.

2. The method according to claim 1, characterized in that, The search engine includes an inverted index, which includes a mapping relationship between document identifiers and corresponding documents; obtaining the M first documents that match the M document identifiers includes: Find the M first documents corresponding to the M document identifiers from the inverted index.

3. The method according to claim 1 or 2, characterized in that, The rating description information corresponding to each of the first documents includes hit information and rating information; the step of generating the rating description information corresponding to each of the M first documents through the rating component includes: For the scoring device, Q first documents with nested structures and J first documents with non-nested structures are selected from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q; The scoring component generates hit information and score information for each of the M first documents, where the score information is a score indicating the degree of matching between the corresponding first document and the search criteria.

4. The method according to claim 3, characterized in that, The hit information in the J first documents includes one or more of the hit fields, hit words, and hit locations that match the search criteria in the corresponding first documents.

5. The method according to claim 3, characterized in that, The first document, which is a nested structure, includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, the set of document identifier sequences of the one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document.

6. The method according to any one of claims 1-2, characterized in that, The generation of the first structured object includes: Obtain the K subqueries included in the query statement, where K is a positive integer greater than 0; Based on the K subqueries, K third structured objects are generated, wherein each third structured object includes a corresponding subquery, and the first structured object includes the K third structured objects.

7. The method according to claim 6, characterized in that, The step of creating a score calculator based on the first structured object includes: K corresponding sub-scoring units are created based on the K third structured objects. Each sub-scoring unit includes a corresponding third structured object, L document identifiers that match the corresponding third structured object, and the scoring component. The scorer is created based on the K corresponding sub-scorers, where L is a positive integer greater than 0.

8. The method according to any one of claims 1-2, characterized in that, The method further includes: For each scorer, the corresponding first document is sorted first based on the score description information; Based on the results of the first sorting, N first documents whose rating description information meets the preset conditions are selected from the first documents corresponding to each scorer, where N is a positive integer less than or equal to M.

9. The method according to claim 8, characterized in that, The search engine includes a pre-defined storage component; For each of the first documents in the scorer, the corresponding second structured object is saved to the storage component; The method further includes: Based on the document identifiers of the N first documents, retrieve the corresponding N second structured objects from the storage component; Based on the N second structured objects, N corresponding structured information items are generated.

10. The method according to claim 9, characterized in that, The method further includes: The N first documents are sorted a second time based on the N corresponding structured information, and the sorting results are output to the search results.

11. The method according to claim 10, characterized in that, The second sorting of N corresponding first documents based on N structured information includes: Based on the similarity score between each of the N structured information pieces and the corresponding first document, the N corresponding first documents are sorted in a second manner. Alternatively, based on the percentage score of the hit words in each of the N structured information pieces in the corresponding hit field, the N corresponding first documents are sorted in a second manner.

12. A data retrieval device, characterized in that, Applied to a search engine, the device includes: The first parsing unit is used to receive search conditions and generate a first structured object, wherein the first structured object includes the query statement contained in the search conditions; The first creation unit creates a scorer based on the first structured object. The scorer includes the first structured object, M document identifiers matching the first structured object, and a scoring component, where M is a positive integer greater than 0. The information acquisition unit acquires M first documents that match the M document identifiers, and generates rating description information corresponding to each of the M first documents through the rating component; The first generation unit generates M second structured objects based on the rating description information corresponding to each first document and the first structured object, wherein one second structured object corresponds to one first document.

13. The apparatus according to claim 12, characterized in that, The search engine includes an inverted index, which includes a mapping relationship between document identifiers and corresponding documents. The information acquisition unit is specifically used for: Find the M first documents corresponding to the M document identifiers from the inverted index.

14. The apparatus according to claim 12 or 13, characterized in that, The rating description information for each of the first documents includes hit information and rating information; The information acquisition unit is specifically used for: For each scorer, Q first documents with nested structures and J first documents with non-nested structures are selected from the M first documents, where Q is an integer not greater than M and not less than 0, and J is the difference between M and Q. The scoring component generates hit information and score information for each of the M first documents, where the score information is a score indicating the degree of matching between the corresponding first document and the search criteria.

15. The apparatus according to claim 14, characterized in that, The hit information in the J first documents includes one or more of the hit fields, hit words, and hit locations that match the search criteria in the corresponding first documents.

16. The apparatus according to claim 14, characterized in that, The first document, which is a nested structure, includes a main document and one or more sub-documents; the hit information in the Q first documents includes one or more of the following: the document identifier of the main document of each of the Q first documents, the set of document identifier sequences of the one or more sub-documents, the document identifier of the hit sub-document in the main document, and the hit position of the hit sub-document.

17. The apparatus according to any one of claims 12-13, characterized in that, The first generation unit is specifically used for: Obtain the K subqueries included in the query statement, where K is a positive integer greater than 0; Based on the K subqueries, K third structured objects are generated, wherein each third structured object includes a corresponding subquery, and the first structured object includes the K third structured objects.

18. The apparatus according to claim 17, characterized in that, The first creation unit is specifically used for: K corresponding sub-scoring units are created based on the K third structured objects. Each sub-scoring unit includes a corresponding third structured object, L document identifiers that match the corresponding third structured object, and the scoring component. The scorer is created based on the K corresponding sub-scorers, where L is a positive integer greater than 0.

19. The apparatus according to any one of claims 12-13, characterized in that, The device further includes: The first sorting unit sorts the corresponding first document based on the scoring description information for each scorer. The filtering unit, based on the result of the first sorting, filters out N first documents from the first documents corresponding to each scorer whose scoring description information meets preset conditions, where N is a positive integer less than or equal to M.

20. The apparatus according to claim 19, characterized in that, The search engine includes a pre-defined storage component; For each of the first documents in the scorer, the corresponding second structured object is saved to the storage component; The device further includes: The extraction unit extracts N corresponding second structured objects from the storage component based on the document identifiers of the N first documents. The information generation unit generates N corresponding structured information items based on N second structured objects.

21. The apparatus according to claim 20, characterized in that, The device further includes: The second sorting unit performs a second sorting on the N corresponding structured information on the N first documents, and outputs the sorting results to the search results.

22. The apparatus according to claim 21, characterized in that, The second sorting unit is specifically used for: Based on the similarity score between each of the N structured information pieces and the corresponding first document, the N corresponding first documents are sorted in a second manner. Alternatively, based on the percentage score of the hit words in each of the N structured information pieces in the corresponding hit field, the N corresponding first documents are sorted in a second manner.

23. A computer storage medium, characterized in that, The computer storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

24. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-11.

Images