Data searching method and device, electronic equipment and storage medium

Abstract

The application discloses a data search method and device, electronic equipment and a storage medium, and relates to the technical field of electronic equipment. The method comprises the following steps: receiving a search request input for a data set, wherein the search request carries search information, the data set comprises a plurality of data, performing entity recognition and entity linking on the search information, obtaining index field information corresponding to the search information, performing search intention recognition on the index field information, obtaining an intention recognition result, obtaining a query expression according to the intention recognition result and determining a target retrieval mode from a vector retrieval mode and a label retrieval mode, and retrieving target data corresponding to the query expression from the plurality of data through the target retrieval mode. By introducing the entity recognition and entity linking technology, the application can optimize the search efficiency of the data and reduce the time consumption of the data search.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and more specifically, to a data search method, apparatus, electronic device, and storage medium. Background Technology

[0002] With advancements in the camera technology of electronic devices (such as smartphones), these devices have become the primary means of taking photos. The number and variety of photos taken are increasing, including people, animals, plants, and landscapes. Therefore, efficient image retrieval techniques in photo albums are becoming increasingly important. However, current image retrieval technologies are relatively inefficient, resulting in lengthy search times. Summary of the Invention

[0003] In view of the above problems, this application proposes a data search method, apparatus, electronic device, and storage medium to solve the above problems.

[0004] In a first aspect, embodiments of this application provide a data search method, the method comprising: receiving a search request input for a dataset, wherein the search request carries search information, and the dataset includes multiple data; performing entity recognition and entity linking on the search information to obtain index field information corresponding to the search information; performing search intent recognition on the index field information to obtain intent recognition results; obtaining a query expression based on the intent recognition results and determining a target retrieval method from vector retrieval method and tag retrieval method; and retrieving target data corresponding to the query expression from the multiple data using the target retrieval method.

[0005] Secondly, embodiments of this application provide a data search apparatus, comprising: a search request receiving module, configured to receive a search request input to a dataset, wherein the search request carries search information and the dataset includes multiple data items; an index field information obtaining module, configured to perform entity recognition and entity linking on the search information to obtain index field information corresponding to the search information; an intent recognition module, configured to perform search intent recognition on the index field information to obtain intent recognition results; a retrieval method determining module, configured to obtain a query expression based on the intent recognition results and determine a target retrieval method from vector retrieval method and tag retrieval method; and a data search module, configured to retrieve target data corresponding to the query expression from the multiple data items using the target retrieval method.

[0006] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0008] The data search method, apparatus, electronic device, and storage medium provided in this application embodiment receive a search request for a dataset input, wherein the search request carries search information, the dataset includes multiple data, entity recognition and entity linking are performed on the search information to obtain index field information corresponding to the search information, search intent recognition is performed on the index field information to obtain intent recognition results, a query expression is obtained based on the intent recognition results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, and target data corresponding to the query expression is retrieved from multiple data using the target retrieval method. Thus, by introducing entity recognition and entity linking technology, the data search efficiency can be optimized and the data search time can be reduced. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a data search method provided in an embodiment of this application is shown;

[0011] Figure 2 A flowchart illustrating a data search method provided in an embodiment of this application is shown;

[0012] Figure 3 A flowchart illustrating a data search method provided in an embodiment of this application is shown;

[0013] Figure 4 A flowchart illustrating a data search method provided in an embodiment of this application is shown;

[0014] Figure 5 A flowchart illustrating a data search method provided in an embodiment of this application is shown;

[0015] Figure 6This illustration shows a schematic diagram of the data search method provided in this application when applied to a photo album application;

[0016] Figure 7 A block diagram of a data search apparatus provided in one embodiment of this application is shown;

[0017] Figure 8 A block diagram of an electronic device for performing a data search method according to an embodiment of this application is shown;

[0018] Figure 9 A storage unit for storing or carrying program code implementing a data search method according to an embodiment of the present application is shown. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0020] In the field of edge image search, search solutions include two methods: tag retrieval and vector retrieval.

[0021] Tag retrieval involves using a computer vision (CV) tagging model to identify content tags in images / videos, storing these tags in a database, and then retrieving them from the database. When a user enters a single search term, the search information is typically segmented into multiple tags, which are then used to query the database. However, tag retrieval is limited by the number of categories in the tag system. When the user's query keywords do not belong to the tag system, no results are found, resulting in insufficient recall. Furthermore, tag retrieval requires querying multiple attribute fields such as time, location, and CV tags separately, leading to relatively low query efficiency.

[0022] Vector retrieval uses a multimodal model to map both text and images into vectors, then calculates the distance between text and image vectors to find text-related images. Vector retrieval typically supports search information lengths of 80 Chinese characters or more, supporting most photo album search requests. However, because it maps user-input search information into vectors using a multimodal model, it is not limited by tag systems. However, the vectorization process is more complex and power-consuming than traditional database queries. Furthermore, because images possess rich semantic information, a single vector cannot fully represent the image's content; therefore, results obtained through multimodal vector retrieval are prone to partial matching errors, resulting in lower accuracy compared to tag-based retrieval.

[0023] Furthermore, a significant proportion of mobile phone photo album users search using handwriting input. Due to the limitations of handwriting input recognition accuracy, search results are prone to typos or incomplete characters. In such cases, neither tag-based nor vector-based retrieval can be effectively performed.

[0024] To address the aforementioned problems, the inventors, through long-term research, discovered and proposed the data search method, apparatus, electronic device, and storage medium provided in the embodiments of this application. By introducing entity recognition and entity linking technologies, the data search efficiency can be optimized and the time consumption of data search can be reduced. The specific data search method will be described in detail in subsequent embodiments.

[0025] Please see Figure 1 , Figure 1 A flowchart illustrating a data search method according to an embodiment of this application is shown. This method optimizes data search efficiency and reduces search time by introducing entity recognition and entity linking technologies. In a specific embodiment, this data search method is applied to, for example... Figure 7 The data search device 200 and the electronic device 100 equipped with the data search device 200 are shown. Figure 8 The following will use an electronic device as an example to illustrate the specific process of this embodiment. Of course, it is understood that the electronic device used in this embodiment may include smartphones, tablets, wearable electronic devices, etc., and is not limited thereto. The following will focus on... Figure 1 The process shown will be described in detail, and the data search method may specifically include the following steps:

[0026] Step S110: Receive a search request for the input dataset, wherein the search request carries search information and the dataset includes multiple data.

[0027] Optionally, the electronic device may include a dataset, which may contain multiple data sets. For example, the dataset may include a photo album, and the multiple data sets may be multiple images and / or videos from the album. Alternatively, the multiple data sets may include multiple audio recordings, multiple tags, multiple documents, etc., without limitation.

[0028] In this embodiment, when a user wishes to view certain data from multiple datasets, they can search for those specific data. The user can trigger a search request for the dataset, and the electronic device can receive the search request, which carries the search information, as input for the dataset.

[0029] In some implementations, users can input search requests carrying search information via voice, text, or touch, etc., without limitation. As an example, an electronic device can be configured with a "text input box" for a dataset, allowing users to input search requests carrying search information, such as a search request for "puppy".

[0030] Step S120: Perform entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information.

[0031] In this embodiment, when an electronic device receives a search request, it can respond to the search request by performing entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information.

[0032] Understandably, entity recognition and entity linking are two key tasks in natural language processing and text mining. They involve identifying and linking entities in text to extract useful information. Entity recognition, a task in natural language processing, aims to identify specific entities, such as names of people, places, and organizations, from text. The purpose of entity recognition is to classify these entities into predefined categories or to detect new entities. During entity recognition, natural language processing techniques, such as word segmentation, part-of-speech tagging, and syntactic analysis, can be used to extract features and information from the text. Entity linking, on the other hand, is the process of linking entity references in text to corresponding entities in a knowledge base. This task typically involves determining the relationships between entities appearing in the text and entities stored in the knowledge base.

[0033] In essence, index field information refers to the metadata information of each field obtained by indexing a data table or text in a database or search engine. This information describes attributes such as the field's name, data type, length, and index type. In a database, each table has multiple fields, each with its specific data type and constraints. Indexing these fields improves query speed and data retrieval efficiency. Index field information includes attributes such as field name, data type, whether nullables are allowed, and default values. In a search engine, after indexing text, information such as the weight, frequency, and position of each word or phrase can be obtained. This information can be used for tasks such as sorting, filtering, and recommendation. Index field information includes attributes such as word or phrase, weight, frequency, and position. In short, index field information is the result of indexing a data table or text; it describes the metadata of each field or word and can be used for subsequent data processing and analysis tasks.

[0034] In some implementations, when an electronic device receives a search request, it can input the search information carried in the search request into a natural language processing (NLP) module. The NLP module will then perform word segmentation on the search information and complete entity recognition and entity linking based on the word segmentation. The text information in the search information will be parsed into index field information.

[0035] Step S130: Perform search intent recognition on the index field information to obtain intent recognition results.

[0036] In this embodiment, given the index field information, search intent recognition can be performed on the index field information to obtain intent recognition results. Intent recognition refers to a technical task of automatically identifying a user's intent or purpose through natural language text. In search engines, intent recognition can analyze a user's core search needs.

[0037] As an feasible approach, once the index field information is obtained, it can be input into the intent recognition module. The intent recognition module can then perform semantic understanding and expansion based on the literal parsing of the index field information to obtain the intent recognition result.

[0038] As another feasible approach, given the index field information, it can be input into the intent recognition model to obtain the intent recognition result output by the model. Optionally, given the index field information, feature selection can be performed on the index field information (selecting features related to the intent to improve the accuracy of the intent recognition model) to obtain target features. These target features can then be input into the intent recognition model to obtain the intent recognition result output by the model. Optionally, during the intent recognition process using the index field information, the model parameters of the intent recognition model can be adjusted and optimized to improve its accuracy and robustness. Furthermore, the intent recognition model can be continuously updated with new data; this is not limited here.

[0039] Step S140: Based on the intent recognition result, obtain the query expression and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0040] In this embodiment, upon obtaining the intent recognition result, a query expression can be obtained based on the intent recognition result, and the target retrieval method can be determined from vector retrieval and tag retrieval. Tag retrieval is a keyword-based retrieval method that represents and stores text information in the form of tags by tagging the text content, thereby achieving fast tag retrieval. Common tagging methods in tag retrieval include rule-based tagging and machine learning-based tagging. Vector retrieval is a vector-based retrieval method that converts text content into vector representations and uses the similarity between vectors to measure the similarity between texts, thereby achieving efficient vector retrieval. Common vector representation methods in vector retrieval include TF-IDF, word2vec, and BERT.

[0041] As one feasible approach, electronic devices can pre-set and store a first condition for satisfying the vector retrieval method and a second condition for satisfying the tag retrieval method. Based on this, upon obtaining the intent recognition result, it can be determined whether the intent recognition result satisfies the first condition and whether it satisfies the second condition. Specifically, if it is determined that the intent recognition result satisfies the first condition, the vector retrieval method can be determined as the target retrieval method and the corresponding query expression can be obtained; if it is determined that the intent recognition result satisfies the second condition, the tag retrieval method can be determined as the target retrieval method and the corresponding query expression can be obtained.

[0042] As another feasible approach, electronic devices can pre-set and store target conditions that satisfy the tag retrieval method. Based on this, upon obtaining the intent recognition result, it can be determined whether the intent recognition result meets the target conditions. Specifically, if it is determined that the intent recognition result meets the target conditions, the tag retrieval method can be determined as the target retrieval method and the corresponding query expression can be obtained; if it is determined that the intent recognition result does not meet the target conditions, the vector retrieval method can be determined as the target retrieval method and the corresponding query expression can be obtained.

[0043] In some implementations, if the tag retrieval method is determined as the target retrieval method from vector retrieval and tag retrieval methods based on the intent recognition results, a query statement in SQLite or an index can be generated based on the attribute fields (index field information) corresponding to the entity link to obtain a query expression. If the vector retrieval method is determined as the target retrieval method from vector retrieval and tag retrieval methods based on the intent recognition results, information such as time, location, and tags can be extracted from the entity link results to obtain a query expression.

[0044] Step S150: Using the target retrieval method, retrieve the target data corresponding to the query expression from the plurality of data.

[0045] In this embodiment, given a defined target retrieval method, the target data corresponding to the query expression can be retrieved from multiple datasets using this method. By introducing entity recognition and entity linking technologies, the search efficiency can be optimized and the time required for data retrieval can be reduced.

[0046] In some implementations, if the tag retrieval method is determined to be the target retrieval method, the search service corresponding to the tag retrieval method can be used to search the query expression to retrieve the target data corresponding to the query expression from multiple data; if the vector retrieval method is determined to be the target retrieval method, the search service corresponding to the vector retrieval method can be used to search the query expression to retrieve the target data corresponding to the query expression from multiple data.

[0047] As an feasible approach, if the target retrieval method is determined to be tag-based retrieval, the target data corresponding to the query expression can be retrieved through SQL search service or index search service. If the target retrieval method is vector-based retrieval, the target data corresponding to the query expression can be retrieved through multimodal search service. SQL search service is a search service based on structured data, using the SQL query language to retrieve data from the database. This search service is suitable for scenarios requiring precise queries of structured data, such as enterprise databases and relational databases. Index search service is an index-based search service that improves search efficiency by building indexes. This search service is suitable for scenarios requiring fast retrieval of large-scale text data, such as search engines and knowledge question-answering systems. Multimodal search service is a search service that simultaneously supports multiple media data (such as text, images, and video). This search service is suitable for scenarios requiring comprehensive retrieval of multiple media data, such as multimedia databases and digital libraries.

[0048] Optionally, the electronic device may be equipped with a multi-path retrieval module, which can request atomic capabilities such as SQLite database, index search, and multimodal search in parallel based on the results of the intent recognition module. Due to differences in computing power among different models of electronic devices and differences in system configuration among different users, the availability of each atomic retrieval capability is not entirely consistent. The multi-path retrieval module supports dynamic access to atomic retrieval capabilities and can dynamically adjust the priority of each atomic retrieval capability according to retrieval efficiency. In this embodiment, when the target retrieval method is determined to be tag retrieval, the multi-path retrieval module can request the atomic capabilities of SQL search service or index search service based on the search expression to perform the search; when the target retrieval method is determined to be vector retrieval, the multi-path retrieval module can request the atomic capabilities of multimodal search service based on the search expression to perform the search.

[0049] In some implementations, when the target retrieval method is determined to be tag retrieval, it can be determined whether the electronic device supports index search service. If it does, the target data corresponding to the query expression is retrieved through the index search service; if it does not, the target data corresponding to the query expression is retrieved through the SQL search service.

[0050] An embodiment of this application provides a data search method that receives a search request input for a dataset, wherein the search request carries search information, the dataset includes multiple data, entity recognition and entity linking are performed on the search information to obtain index field information corresponding to the search information, search intent recognition is performed on the index field information to obtain intent recognition results, a query expression is obtained based on the intent recognition results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, and target data corresponding to the query expression is retrieved from multiple data using the target retrieval method. Thus, by introducing entity recognition and entity linking technology, the data search efficiency can be optimized and the time consumption of data search can be reduced.

[0051] Please see Figure 2 , Figure 2 A flowchart illustrating a data search method provided in an embodiment of this application is shown. The following will focus on... Figure 2 The process shown will be described in detail, and the data search method may specifically include the following steps:

[0052] Step S210: Receive a search request for the input dataset, wherein the search request carries search information and the dataset includes multiple data.

[0053] For a detailed description of step S210, please refer to step S110, which will not be repeated here.

[0054] Step S220: Perform error correction processing on the search information to obtain the corrected search information.

[0055] In this embodiment, when a search request carrying search information is received for input of a dataset, the search information can be corrected to obtain the corrected search information.

[0056] In some implementations, when a search request carrying search information is received for input to a dataset, the accuracy of the search information can be judged. If it is determined that the search information contains errors, the search information can be corrected to obtain corrected search information. If it is determined that the search information does not contain errors, the search information can be kept without correction.

[0057] In some implementations, correcting the search information to obtain corrected search information may include: acquiring the preceding text information of the search information, performing error correction processing on the search information based on the preceding text information and preset error correction candidates, and obtaining corrected search information, wherein the preset error correction candidates are generated based on the stroke order of Chinese characters and pinyin. Optionally, the electronic device may include an NLP module. Upon receiving a search request carrying search information for input to a dataset, the electronic device may input the search information into the NLP module. The NLP module will then adapt the search information to common text in handwriting input methods, generate error correction candidates based on the stroke order of Chinese characters and pinyin as preset error correction candidates, and perform error correction based on the preceding text information of the search information.

[0058] As an feasible approach, correcting search information based on pre-existing text information and preset error correction candidates to obtain corrected search information can include the following steps: First, preprocessing: Performing necessary preprocessing on the input search information, including removing irrelevant information, standardizing data, and word segmentation. Second, error identification: Identifying errors in the text by analyzing the pre-existing text information. Common error types include spelling errors, grammatical errors, and semantic errors. Third, error correction: Based on the identified error types and preset error correction candidates, performing error correction processing to obtain corrected search information.

[0059] Step S230: Perform entity recognition and entity linking on the corrected search information to obtain the index field information corresponding to the corrected search information.

[0060] In this embodiment, after obtaining the corrected search information, the electronic device can perform entity recognition and entity linking on the corrected search information to obtain the index field information corresponding to the corrected search information, thereby improving the accuracy of subsequent searches by correcting input errors.

[0061] Step S240: Perform search intent recognition on the index field information to obtain intent recognition results.

[0062] Step S250: Based on the intent recognition result, obtain the query expression and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0063] Step S260: Using the target retrieval method, retrieve the target data corresponding to the query expression from the plurality of data.

[0064] For a detailed description of steps S240-S260, please refer to steps S130-S150, which will not be repeated here.

[0065] An embodiment of this application provides a data search method that receives a search request for a dataset input, wherein the search request carries search information, the dataset includes multiple data points, the search information is corrected to obtain corrected search information, entity recognition and entity linking are performed on the corrected search information to obtain index field information corresponding to the corrected search information, search intent is identified on the search field information to obtain intent identification results, a query expression is obtained based on the intent identification results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, and target data corresponding to the query expression is retrieved from multiple data points using the target retrieval method. Compared to Figure 1 The data search method shown in this embodiment also performs error correction processing on the input search information, which can improve the accuracy of data search and enhance the user's search experience.

[0066] Please see Figure 3 , Figure 3 A flowchart illustrating a data search method provided in an embodiment of this application is shown. The following will focus on... Figure 3 The process shown will be described in detail, and the data search method may specifically include the following steps:

[0067] Step S310: Receive a search request for the input dataset, wherein the search request carries search information and the dataset includes multiple data.

[0068] For a detailed description of step S310, please refer to step S110, which will not be repeated here.

[0069] Step S320: Perform completion processing on the search information to obtain the completed search information.

[0070] In this embodiment, when a search request carrying search information is received for input of a dataset, the search information can be completed to obtain the completed search information.

[0071] In some implementations, when a search request carrying search information is received for input to a dataset, the completeness of the search information can be determined. If the search information is determined to be incomplete, it can be completed to obtain the completed search information; if the search information is determined to be complete, it can be left uncompleted and the search information can be retained.

[0072] In some implementations, completing the search information to obtain the completed search information may include: completing the search information through prefix matching. Optionally, the electronic device may include an NLP module. Upon receiving a search request carrying search information for a dataset, the NLP module can input the search information into the NLP module. The NLP module, based on business requirements, completes the search information into one or more business-defined search terms through prefix matching, enabling retrieval even when the user inputs incomplete search information.

[0073] As an implementable approach, completing search information using prefix matching to obtain the completed search information can include the following steps: Upon receiving a search request carrying search information, a prefix matching algorithm can be used to find search results that match the prefix of the search information. This prefix matching algorithm can be implemented using fuzzy matching, data structures, natural language processing, or heuristics. Then, the search results that match the prefix of the search information, i.e., the completed search information, are returned. These search results may include relevant web pages, articles, products, etc., without limitation.

[0074] Step S330: Perform entity recognition and entity connection on the completed search information to obtain the index field information corresponding to the completed search information.

[0075] In this embodiment, when the electronic device obtains the completed search information, it can perform entity recognition and entity linking on the completed search information to obtain the index field information corresponding to the completed search information, thereby improving the accuracy of subsequent searches by improving the incomplete input.

[0076] Step S340: Perform search intent recognition on the index field information to obtain intent recognition results.

[0077] Step S350: Based on the intent recognition result, obtain the query expression and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0078] Step S360: Using the target retrieval method, retrieve the target data corresponding to the query expression from the plurality of data.

[0079] For a detailed description of steps S340-S360, please refer to steps S130-S140, which will not be repeated here.

[0080] An embodiment of this application provides a data search method that receives a search request for a dataset input, wherein the search request carries search information, the dataset includes multiple data points, the search information is completed to obtain completed search information, entity recognition and entity linking are performed on the completed search information to obtain index field information corresponding to the completed search information, search intent is identified on the search field information to obtain intent identification results, a query expression is obtained based on the intent identification results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, and target data corresponding to the query expression is retrieved from multiple data points using the target retrieval method. Compared to Figure 1 The data search method shown in this embodiment also performs completion processing on the input search information, which can improve the accuracy of data search and enhance the user's search experience.

[0081] Please see Figure 4 , Figure 4 A flowchart illustrating a data search method provided in an embodiment of this application is shown. The following will focus on... Figure 4 The process shown will be described in detail, and the data search method may specifically include the following steps:

[0082] Step S410: Receive a search request for the input dataset, wherein the search request carries search information and the dataset includes multiple data.

[0083] Step S420: Perform entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information.

[0084] Step S430: Perform search intent recognition on the index field information to obtain intent recognition results.

[0085] Step S440: Based on the intent recognition result, obtain the query expression and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0086] Step S450: Using the target retrieval method, retrieve the target data corresponding to the query expression from the plurality of data.

[0087] For a detailed description of steps S410-S450, please refer to steps S110-S150, which will not be repeated here.

[0088] Step S460: Filter the target data to obtain filtered target data.

[0089] In this embodiment, in order to further improve the accuracy of the searched data, when the target data corresponding to the query expression is retrieved from multiple data, the target data can be filtered to obtain filtered target data. By filtering, data with low accuracy in the target data is removed, while data with high accuracy in the target data is retained, thereby improving the accuracy of the searched data.

[0090] In some implementations, the electronic device can set different filtering conditions for tag-based retrieval and vector-based retrieval. Optionally, if the target retrieval method is tag-based, the target data is filtered using a first filtering condition to obtain filtered target data; if the target retrieval method is vector-based, the target data is filtered using both a first and a second filtering condition to obtain filtered target data. The first filtering condition may include: filtering rules supporting business logic (e.g., directly filtering deleted data); and filtering based on a dynamic relevance threshold of the semantics of the search information (e.g., directly filtering data with an accuracy rate below 80%). The second filtering condition may include: cross-validation using CV tags and multimodal vector retrieval (e.g., time tags, location tags, etc.).

[0091] As an feasible approach, if the target retrieval method is vector retrieval, target information is extracted from the index field information, and a second filtering condition is generated based on the target information. Specifically, if the target retrieval method is vector retrieval, information such as time, location, and tags can be extracted from the entity link results, and multimodal retrieval result filtering conditions can be generated as the second filtering condition, thereby reducing the error rate of vector retrieval.

[0092] In some implementations, the electronic device may include a search result filtering module. When retrieving target data corresponding to a query expression from multiple datasets, the search result filtering module can filter the target data to obtain filtered target data. Specifically, the search result filtering module filters the recall results from each of the multiple retrieval modules based on the second filtering conditions generated by the intent recognition module. The search result filtering module incorporates three layers of filtering to improve search accuracy: 1) supporting filtering rules related to business logic; 2) using dynamic relevance thresholds based on query semantics; and 3) using cross-validation with CV tags and multimodal vector retrieval.

[0093] Therefore, in this embodiment, if the target retrieval method is determined to be tag retrieval, the target data can be filtered by the search result filtering module based on 1) filtering rules related to business logic and 2) dynamic relevance thresholds of query semantics to obtain filtered target data; if the target retrieval method is determined to be vector retrieval, the target data can be filtered by the search result filtering module based on 1) filtering rules related to business logic; 2) dynamic relevance thresholds of query semantics; and 3) cross-validation using CV tags and multimodal vector retrieval to obtain filtered target data.

[0094] An embodiment of this application provides a data search method that receives a search request input for a dataset, wherein the search request carries search information, the dataset includes multiple data items, entity recognition and entity linking are performed on the search information to obtain index field information corresponding to the search information, search intent recognition is performed on the index field information to obtain intent recognition results, a query expression is obtained based on the intent recognition results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, target data corresponding to the query expression is retrieved from multiple data items using the target retrieval method, and the target data is filtered to obtain filtered target data. Compared to Figure 1 The data search method shown in this embodiment also filters the target data, which can improve the accuracy of the final searched data.

[0095] Please see Figure 5 , Figure 5 A flowchart illustrating a data search method provided in an embodiment of this application is shown. The following will focus on... Figure 5 The process shown will be described in detail, and the data search method may specifically include the following steps:

[0096] Step S510: Receive a search request for the input dataset, wherein the search request carries search information and the dataset includes multiple data.

[0097] Step S520: Perform entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information.

[0098] Step S530: Perform search intent recognition on the index field information to obtain intent recognition results.

[0099] Step S540: Based on the intent recognition result, obtain the query expression and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0100] Step S550: Using the target retrieval method, retrieve the target data corresponding to the query expression from the plurality of data.

[0101] For a detailed description of steps S510-S550, please refer to steps S110-S150, which will not be repeated here.

[0102] Step S560: Obtain the attributes corresponding to each of the multiple target data.

[0103] In this embodiment, when retrieving multiple target data corresponding to a query expression from multiple data sources, the attributes corresponding to each of the multiple target data can be obtained. These attributes may include: the time corresponding to each of the multiple target data, the location corresponding to each of the multiple target data, the relevance corresponding to each of the multiple target data, etc., and are not limited thereto.

[0104] Step S570: Merge and sort the multiple target data based on the attributes corresponding to each of the multiple target data.

[0105] In this embodiment, given the attributes corresponding to each of the multiple target data, the multiple target data can be fused and sorted based on the attributes corresponding to each of the multiple target data to meet diverse business display needs.

[0106] In some implementations, the electronic device may include a fusion sorting module, which is a result encapsulation module that interfaces with the business interaction device. This fusion sorting module can support result aggregation and sorting based on different attributes such as time, location, and relevance, and can be dynamically adjusted according to configuration information to meet diverse business display needs. Therefore, when retrieving multiple target data corresponding to a query expression from multiple datasets, the fusion sorting module can obtain the attributes corresponding to each of the multiple target data sets, and then fuse and sort the multiple target data sets based on these attributes.

[0107] An embodiment of this application provides a data search method that receives a search request input for a dataset, wherein the search request carries search information, the dataset includes multiple data sets, entity recognition and entity linking are performed on the search information to obtain index field information corresponding to the search information, search intent recognition is performed on the index field information to obtain intent recognition results, a query expression is obtained based on the intent recognition results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, target data corresponding to the query expression is retrieved from the multiple data sets using the target retrieval method, attributes corresponding to each of the multiple target data sets are obtained, and the multiple target data sets are fused and sorted based on their respective attributes. Compared to Figure 1The data search method shown in this embodiment also integrates and sorts the target data, which can improve the display effect of the searched data and enhance the user experience.

[0108] Please see Figure 6 , Figure 6 This illustration shows a diagram of the data search method provided in this application when applied to a photo album application. For example... Figure 6 As shown, this solution, when applied to photo album applications, can be compatible with different atomic search capabilities and supports customized business needs. The search process mainly includes five modules: NLP processing, intent recognition, multi-path retrieval, search result filtering, and fusion ranking.

[0109] The NLP processing module handles query-level error correction and completion, and performs entity recognition and linking based on word segmentation, parsing the text information in the query into index fields. The query error correction function is adapted to common handwriting input methods, generating error correction candidates based on the stroke order and pinyin of Chinese characters and correcting errors based on the preceding text information of the query. Completion, based on business needs, uses prefix matching to complete the query into one or more business-defined search terms, enabling retrieval even when the user inputs an incomplete query. Entity recognition and linking are responsible for identifying proper nouns such as time, location, name, and tags in the query and aligning them to attribute fields in the database.

[0110] The intent recognition module achieves semantic understanding and expansion based on literal query parsing. First, it separates vector retrieval from tag retrieval. Second, for tag retrieval requests, it generates SQLite or indexed query statements based on the attribute fields corresponding to the entity links. For vector retrieval requests, the intent recognition module extracts information such as time, location, and tags from the entity link results and generates filtering conditions for multimodal search results, thereby reducing bad cases in multimodal search results.

[0111] The multi-path retrieval module requests atomic capabilities such as SQLite database, index search, and multimodal search in parallel based on the results of the intent recognition module. Due to differences in computing power across different machine models and system configurations among different users, the availability of each atomic retrieval capability is not entirely consistent. The multi-path retrieval module supports dynamic access to atomic retrieval capabilities and can dynamically adjust the priority of each atomic retrieval capability based on retrieval efficiency.

[0112] The search result filtering module filters the recall results from each of the multi-path retrieval modules based on the filtering conditions generated by the intent recognition module. The filtering module incorporates three layers of filtering to improve search accuracy: 1) supporting filtering rules related to business logic; 2) using dynamic relevance thresholds based on query semantics; and 3) using cross-validation with CV tags and multimodal vector retrieval.

[0113] The fusion sorting module is a result encapsulation module that integrates with business interaction design. It supports the aggregation and sorting of results based on different attributes such as time, location, and relevance, and can be dynamically adjusted according to configuration information to meet diverse business display needs.

[0114] Please see Figure 7 , Figure 7 A block diagram of a data search apparatus provided in an embodiment of this application is shown. The following will focus on... Figure 7 The block diagram shown illustrates that the data search device 200 includes: a search request receiving module 210, an index field information obtaining module 220, an intent recognition module 230, a retrieval method determining module 240, and a data search module 250, wherein:

[0115] The search request receiving module 210 is used to receive a search request for input data set, wherein the search request carries search information and the data set includes multiple data.

[0116] The index field information acquisition module 220 is used to perform entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information.

[0117] Further, the index field information acquisition module 220 includes: an error correction processing submodule and a first index field information acquisition submodule, wherein:

[0118] The error correction processing submodule is used to perform error correction processing on the search information to obtain the error-corrected search information.

[0119] Furthermore, the error correction processing submodule includes: a pre-text information acquisition unit and an error correction processing unit, wherein:

[0120] A preliminary text information acquisition unit is used to acquire the preliminary text information of the search information.

[0121] The error correction processing unit is used to perform error correction processing on the search information based on the preceding text information and preset error correction candidates to obtain the error-corrected search information, wherein the preset error correction candidates are generated based on the stroke order of Chinese characters and pinyin.

[0122] The first index field information acquisition submodule is used to perform entity recognition and entity linking on the corrected search information to obtain the index field information corresponding to the corrected search information.

[0123] Further, the index field information acquisition module 220 includes: a completion processing submodule and a second index field information acquisition submodule, wherein:

[0124] The completion processing submodule is used to complete the search information to obtain the completed search information.

[0125] Furthermore, the completion processing submodule includes: a completion processing unit, wherein:

[0126] The completion processing unit is used to complete the search information by prefix matching to obtain the completed search information.

[0127] The second index field information acquisition submodule is used to perform entity recognition and entity connection on the completed search information to obtain the index field information corresponding to the completed search information.

[0128] The intent recognition module 230 is used to perform search intent recognition on the index field information and obtain intent recognition results.

[0129] The retrieval method determination module 240 is used to obtain a query expression based on the intent recognition result and determine the target retrieval method from the vector retrieval method and the tag retrieval method.

[0130] The data search module 250 is used to retrieve target data corresponding to the query expression from the plurality of data using the target retrieval method.

[0131] Furthermore, the data search module 250 includes: a first data search submodule and a second search submodule, wherein:

[0132] The first data search submodule is used to retrieve the target data corresponding to the query expression through SQL search service or index search service if the target retrieval method is the tag retrieval method.

[0133] The second data search submodule is used to retrieve the target data corresponding to the query expression through a multimodal search service if the target retrieval method is the vector retrieval method.

[0134] Furthermore, the data search device 200 further includes a data filtering module, wherein:

[0135] The data filtering module is used to filter the target data to obtain filtered target data.

[0136] Furthermore, the data filtering module includes: a first data filtering submodule and a second data filtering submodule, wherein:

[0137] The first data filtering submodule is used to filter the target data through the first filtering condition if the target retrieval method is the tag retrieval method, so as to obtain the filtered target data.

[0138] The second data filtering submodule is used to filter the target data using the first filtering condition and the second filtering condition if the target retrieval method is the vector retrieval method, so as to obtain the filtered target data.

[0139] Furthermore, the data filtering module includes: a target information extraction submodule and a filter condition generation submodule, wherein:

[0140] The target information extraction submodule is used to extract target information from the index field information if the target retrieval method is the vector retrieval method.

[0141] The filter condition generation submodule is used to generate the second filter condition based on the target information.

[0142] Furthermore, the data search device 200 further includes: an attribute acquisition module and a data fusion module, wherein:

[0143] The attribute retrieval module is used to retrieve the attributes corresponding to each of multiple target data.

[0144] The data fusion module is used to fuse and sort the multiple target data based on the attributes corresponding to each of the multiple target data.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0146] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0147] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0148] Please see Figure 8This document illustrates a structural block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications can be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0149] The processor 110 may include one or more processing cores. The processor 110 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120. Optionally, the processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 110 and may be implemented separately using a communication chip.

[0150] The memory 120 may include random access memory (RAM) or read-only memory (ROM). The memory 120 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing functions (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0151] Please see Figure 9This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0152] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0153] In summary, the data search method, apparatus, electronic device, and storage medium provided in this application receive a search request for a dataset input, wherein the search request carries search information, the dataset includes multiple data, entity recognition and entity linking are performed on the search information to obtain index field information corresponding to the search information, search intent recognition is performed on the index field information to obtain intent recognition results, a query expression is obtained based on the intent recognition results, and a target retrieval method is determined from vector retrieval method and tag retrieval method, and target data corresponding to the query expression is retrieved from multiple data through the target retrieval method. Thus, by introducing entity recognition and entity linking technology, the data search efficiency can be optimized and the data search time can be reduced.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data search method, characterized in that, The method includes: Receive a search request for input to a dataset, wherein the search request carries search information and the dataset includes multiple data items; Entity identification and entity linking are performed on the search information to obtain the index field information corresponding to the search information; The search intent is identified by analyzing the index field information to obtain the intent identification result; Based on the intent recognition results, a query expression is obtained, and the target retrieval method is determined from the vector retrieval method and the tag retrieval method; Using the target retrieval method, target data corresponding to the query expression is retrieved from the plurality of data; The step of performing entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information includes: Obtain the preceding text information of the search information, and identify errors in the search information based on the preceding text information; The search information is corrected according to the error type and preset error correction candidates to obtain the corrected search information. The preset error correction candidates are generated based on the stroke order of Chinese characters and pinyin. Entity recognition and entity linking are performed on the corrected search information to obtain the index field information corresponding to the corrected search information.

2. The method according to claim 1, characterized in that, The step of performing entity recognition and entity linking on the search information to obtain the index field information corresponding to the search information includes: The search information is completed to obtain the completed search information; Entity recognition and entity connection are performed on the completed search information to obtain the index field information corresponding to the completed search information.

3. The method according to claim 2, characterized in that, The step of completing the search information to obtain the completed search information includes: The search information is completed by prefix matching to obtain the completed search information.

4. The method according to any one of claims 1-3, characterized in that, After retrieving the target data corresponding to the query expression from the plurality of data using the target retrieval method, the method further includes: The target data is filtered to obtain filtered target data.

5. The method according to claim 4, characterized in that, The step of filtering the target data to obtain filtered target data includes: If the target retrieval method is the tag retrieval method, then the target data is filtered using the first filtering condition to obtain the filtered target data; or If the target retrieval method is the vector retrieval method, then the target data is filtered by the first filtering condition and the second filtering condition to obtain the filtered target data.

6. The method according to claim 5, characterized in that, After obtaining the query expression based on the intent recognition result and determining the target retrieval method from the vector retrieval method and the tag retrieval method, the method further includes: If the target retrieval method is the vector retrieval method, then the target information is extracted from the index field information; The second filtering condition is generated based on the target information.

7. The method according to any one of claims 1-3, characterized in that, The number of target data is multiple. After retrieving the target data corresponding to the query expression from the multiple data using the target retrieval method, the method further includes: Retrieve the attributes corresponding to each of multiple target data; The multiple target data are fused and sorted based on their respective attributes.

8. The method according to any one of claims 1-3, characterized in that, The step of retrieving target data corresponding to the query expression from the plurality of data using the target retrieval method includes: If the target retrieval method is the tag retrieval method, then the target data corresponding to the query expression is retrieved through SQL search service or index search service; or If the target retrieval method is the vector retrieval method, then the target data corresponding to the query expression is retrieved through the multimodal search service.

9. A data search device, characterized in that, The device includes: The search request receiving module is used to receive search requests for input datasets, wherein the search requests carry search information and the datasets include multiple data sets. The index field information acquisition module is used to acquire the preceding text information of the search information, and identify errors in the search information based on the preceding text information; perform error correction processing on the search information according to the error type and preset error correction candidates to obtain the corrected search information, wherein the preset error correction candidates are generated based on the stroke order of Chinese characters and pinyin; perform entity recognition and entity linking on the corrected search information to obtain the index field information corresponding to the corrected search information; The intent recognition module is used to recognize the search intent of the index field information and obtain the intent recognition result; The retrieval method determination module is used to obtain a query expression based on the intent recognition result and determine the target retrieval method from vector retrieval method and tag retrieval method; The data search module is used to retrieve target data corresponding to the query expression from the plurality of data using the target retrieval method.

10. An electronic device, characterized in that, The method includes a memory and a processor, the memory being coupled to the processor, the memory storing instructions that, when executed by the processor, the processor performs the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-8.

Images