Data retrieval method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of data processing, and relates to a data retrieval method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining standard data information; performing feature extraction and semantic representation on the standard data information to obtain semantic vector representation, and constructing a semantic index according to the semantic vector representation; obtaining input retrieval information, and performing semantic understanding and expansion on the input retrieval information to obtain retrieval information representation; converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set; and performing priority ranking on the candidate data set to obtain a retrieval result data set, and returning the retrieval result data set to the display interface for display. According to the method and the device, the retrieval result data set with high content correlation and high retrieval accuracy can be effectively obtained according to the input retrieval information.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, specifically to the field of digital medicine, and in particular to a data retrieval method, device, computer equipment and storage medium. Background Art

[0002] Unstructured databases face the technical bottleneck of semantic understanding when managing and retrieving massive amounts of heterogeneous information. The semantic representation of unstructured data such as text, images, audio and video in the database is diverse and implicit, and it is difficult to describe it with a unified data model and index structure. Traditional retrieval methods based on keyword matching cannot deeply understand the semantic connotation of unstructured data, and the relevance and accuracy of retrieval results cannot be guaranteed. At the same time, there is an expression difference between the retrieval requirements proposed by users in natural language and the semantic representation of unstructured data, which makes it difficult to accurately match the retrieval intent to relevant data.

[0003] This semantic understanding problem is particularly prominent in the management of unstructured databases in the medical field. Massive unstructured data such as medical literature, medical records, and medical images contain a large number of professional terms, abbreviations, synonyms, etc., and the semantic representation is extremely complex. When searching, doctors often use professional terms and implied logics, and these search intentions are difficult to directly match the semantic representation in text data, resulting in a deviation between the search results and actual needs. The difficulty in semantic understanding further leads to low search efficiency. Doctors need to manually browse a large number of irrelevant results to find the information they really need, which seriously affects the value of medical data. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data retrieval method, apparatus, computer device and storage medium to solve the problem that it is impossible to effectively obtain retrieval result data with high content relevance and high retrieval accuracy based on input retrieval information.

[0005] In order to solve the above technical problems, the present application provides a data retrieval method, which adopts the following technical solution:

[0006] Obtain standard data information;

[0007] Performing feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructing a semantic index based on the semantic vector representation;

[0008] Acquire input search information, perform semantic understanding and expansion on the input search information, and obtain search information representation;

[0009] Converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set;

[0010] The candidate data sets are prioritized to obtain a search result data set, and the search result data set is returned to a display interface for display.

[0011] Furthermore, the step of obtaining standard data information specifically includes:

[0012] Get information extraction identification;

[0013] Extracting initial data information from an unstructured database according to the information extraction identifier;

[0014] The initial data information is preprocessed to obtain the standard data information.

[0015] Furthermore, the step of performing feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructing a semantic index according to the semantic vector representation specifically includes:

[0016] Identify the data type of the standard data information, wherein the data type includes: text data, image data, audio data, and video data;

[0017] Inputting the text data, the image data, the audio data, and the video data into a pre-trained feature extraction model for feature extraction to obtain a text data feature vector, an image data feature vector, an audio data feature vector, and a video data feature vector;

[0018] Mapping the text data feature vector, the image data feature vector, the audio data feature vector, and the video data feature vector to a unified semantic vector space to obtain the semantic vector representation;

[0019] Performing similarity calculation on the semantic vector representation to obtain semantic similarity;

[0020] Performing cluster analysis based on the semantic similarity to obtain semantic association relationships;

[0021] The semantic index is constructed according to the semantic association relationship and the semantic vector representation.

[0022] Furthermore, the step of obtaining input search information, semantically understanding and expanding the input search information, and obtaining the search information representation specifically includes:

[0023] Obtain a knowledge graph extraction identifier, and extract a pre-constructed knowledge graph from a database according to the knowledge graph extraction identifier;

[0024] Performing semantic understanding on the input search information based on the pre-built knowledge graph to obtain input search semantic information;

[0025] The input search semantic information is semantically expanded based on the pre-constructed knowledge graph to obtain the search information representation.

[0026] Furthermore, the step of converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set specifically includes:

[0027] Mapping the retrieval information representation to a low-dimensional semantic space to obtain the retrieval semantic vector;

[0028] Extracting a semantic vector representation from the semantic index, and performing cosine similarity calculation on the retrieved semantic vector and the semantic vector representation to obtain a similarity score;

[0029] Searching for corresponding valid data information from the standard data information according to the semantic vector representation;

[0030] The valid data information is sorted according to the similarity score to obtain a sorted data information set, and a preset information set threshold is obtained. The sorted data information set is screened according to the preset information set threshold to obtain the candidate data set.

[0031] Furthermore, the step of prioritizing the candidate data sets to obtain the search result data set specifically includes:

[0032] Obtaining the input search information, and calculating a semantic relevance score between the input search information and the candidate data set;

[0033] Obtaining authoritative information and timeliness information of the candidate data set, and calculating a data ranking score of the candidate data set according to the authoritative information, the timeliness information, and the semantic relevance score;

[0034] The candidate data sets are prioritized according to the data ranking scores to obtain the retrieval result data set.

[0035] Furthermore, after the step of prioritizing the candidate data sets according to the data ranking scores to obtain the search result data set, the following steps are also included:

[0036] Obtaining a preset relevance score threshold, and determining whether the semantic relevance score corresponding to the search result data set is less than the preset relevance score threshold;

[0037] If the semantic relevance score corresponding to the retrieval result data set is less than the preset relevance score threshold, semantic expansion and inference are performed on the input retrieval information to obtain an extended retrieval term, and then the corresponding extended retrieval result data is searched in the retrieval result data set according to the extended retrieval term, and the extended retrieval result data is re-sorted based on the data sorting score to obtain an extended retrieval result data set, and the extended retrieval result data set is returned to the display interface for display.

[0038] In order to solve the above technical problems, the embodiment of the present application further provides a data retrieval device, which adopts the following technical solution:

[0039] An information acquisition module is used to acquire standard data information;

[0040] An index building module, used to perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and to build a semantic index based on the semantic vector representation;

[0041] An information expansion module is used to obtain input search information, perform semantic understanding and expansion on the input search information, and obtain a search information representation;

[0042] An information screening module, used for converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set;

[0043] The data set display module is used to prioritize the candidate data sets to obtain a search result data set, and return the search result data set to a display interface for display.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0045] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above data retrieval methods when executing the computer-readable instructions.

[0046] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0047] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data retrieval method described in any one of the above items are implemented.

[0048] Compared with the prior art, the embodiment of the present application has the following beneficial effects: the embodiment obtains standard data information; performs feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructs a semantic index based on the semantic vector representation; obtains input search information, performs semantic understanding and expansion on the input search information to obtain a search information representation; converts the search information representation into a search semantic vector, and calculates the similarity between the search semantic vector and the semantic index, and screens the standard data information based on the similarity to obtain a candidate data set; prioritizes the candidate data sets to obtain a search result data set, and returns the search result data set to the display interface for display. Thus, a search result data set with high content relevance and high search accuracy can be effectively obtained based on the input search information. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0051] Figure 2 A flowchart of an embodiment of a data retrieval method according to the present application;

[0052] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;

[0053] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;

[0054] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S30;

[0055] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S40;

[0056] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S50;

[0057] Figure 8 is a schematic structural diagram of an embodiment of a data retrieval device according to the present application;

[0058] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to non-related or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0062] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0063] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0064] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0065] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0066] It should be noted that the data retrieval method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the data retrieval device is generally arranged in the server / terminal device.

[0067] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0068] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for data retrieval according to the present application. The data retrieval method comprises the following steps:

[0069] Step S10, obtaining standard data information;

[0070] In this embodiment, the standard data information is information obtained after preprocessing based on the initial data information. The initial data information is a variety of data information stored in an unstructured database. The initial data information includes initial text data, initial image data, initial audio data, and initial video data. In this embodiment, the initial data information can be collected medical text data, medical image data, medical audio data, and medical video data. Among them, the medical text data can be obtained from information sources such as hospital information systems, electronic medical record systems, scientific research databases, government or industry reports, academic papers and journals; medical image data can be obtained from information sources such as medical imaging equipment, hospital imaging departments, and medical imaging databases; medical audio data can be obtained from information sources such as medical recording equipment, telemedicine platforms, and medical education and training systems; medical video data can be obtained from information sources such as live or recorded surgeries and medical documentaries.

[0071] Step S20, performing feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructing a semantic index according to the semantic vector representation;

[0072] In this embodiment, feature extraction is the conversion of standard data information into a representation containing useful information. The standard data information is converted into a corresponding feature vector after feature extraction. Semantic representation refers to a method of converting the feature vector corresponding to the standard data information into a corresponding semantic vector representation. Semantic indexing refers to organizing the feature vector corresponding to the standard data information into a structured form for realizing data retrieval function in the database.

[0073] Step S30, obtaining input search information, performing semantic understanding and expansion on the input search information, and obtaining a search information representation;

[0074] In this embodiment, the input search information is data information in the form of media such as search terms, sentences (phrases or sentences), pictures, voices, videos, etc. entered into the database. Semantic understanding refers to a method of understanding the semantic expression of the input search information including the above data information based on a pre-constructed knowledge graph. Semantic extension is also an information extension of the input search information including the above data information based on the pre-constructed knowledge graph. In this embodiment, the database can be an unstructured database, and the pre-constructed knowledge graph can be a knowledge representation method for medical knowledge construction by organizing and expressing professional knowledge in the medical field in the form of a graph. The pre-constructed knowledge graph represents entities in the medical field (such as diseases, symptoms, drugs, examinations, etc.) and the relationships between them (such as causes, treatments, side effects, etc.) in the form of nodes and edges, forming a complex network structure.

[0075] Step S40, converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set;

[0076] In this embodiment, the retrieval information representation is converted into a retrieval semantic vector by mapping the retrieval information representation to a low-dimensional semantic space to form vector information. The low-dimensional semantic space is a space that simplifies complex, high-dimensional data or text information into a lower-dimensional vector representation. In this low-dimensional space, the distance and relationship between data points can more intuitively reflect the semantic similarity between the original data or text. The similarity between the retrieval semantic vector and the semantic index can be obtained by cosine similarity calculation. The candidate data set is a data set obtained by sorting and filtering according to the score corresponding to the similarity and the preset information set threshold. The preset information set threshold is the initial set data set screening quantity value. In this embodiment, the preset information set threshold is initially set to 20, which can be set and adjusted according to actual conditions.

[0077] Step S50 , prioritizing the candidate data sets to obtain a search result data set, and returning the search result data set to a display interface for display.

[0078] In this embodiment, the candidate data sets are prioritized by inputting the semantic relevance scores between the search information and the candidate data sets, as well as the authoritative information and timeliness information corresponding to the candidate data sets. The data are sorted in order based on the data ranking scores calculated based on the comprehensive semantic relevance scores, authoritative information, and timeliness information, so as to prioritize the candidate data sets and effectively obtain the search result data set for display on the display interface of the searching user.

[0079] In this embodiment, the above method can be applied to the database of the medical service system, and data retrieval is performed in the database of the medical service system by inputting retrieval information to query and obtain corresponding retrieval result information. Specifically, in this embodiment, the medical service system can be one or more of a medical insurance system and a disease insurance system, and the standard data information is data information collected by the medical service system and stored in the system database, and the standard data information is stored in the medical insurance system and the disease insurance system and obtained from the database of the above system, and the retrieval result data set is generated by the above system through the method of this embodiment, and finally output to the display interface of the system for display.

[0080] This embodiment obtains standard data information; performs feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructs a semantic index based on the semantic vector representation; obtains input search information, performs semantic understanding and expansion on the input search information to obtain a search information representation; converts the search information representation into a search semantic vector, and calculates the similarity between the search semantic vector and the semantic index, and screens the standard data information based on the similarity to obtain a candidate data set; prioritizes the candidate data sets to obtain a search result data set, and returns the search result data set to the display interface for display. In this way, a search result data set with high content relevance and high search accuracy can be effectively obtained based on the input search information.

[0081] Continue to refer Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:

[0082] Step S101, obtaining an information extraction identifier;

[0083] In this embodiment, the information extraction identifier is a unique information identifier corresponding to the initial data information, and the information extraction identifier may be a serial number, a UUID (Universally Unique Identifier), a hash value, or a unique code generated by a specific algorithm.

[0084] Step S102, extracting initial data information from the unstructured database according to the information extraction identifier;

[0085] In this embodiment, a traversal query is performed in an unstructured database through an information extraction identifier to extract the matched information, thereby obtaining initial data information. The unstructured database is used to store and manage data that is not convenient to represent with a fixed structure, including various types of data including text, images, audio, video, documents, e-mails, etc.

[0086] Step S103, preprocessing the initial data information to obtain the standard data information.

[0087] In this embodiment, preprocessing includes data cleaning: outlier processing, missing value processing, duplicate value processing and data conversion: normalization and standardization. By preprocessing the initial data information, standard data information that meets the system processing requirements is obtained.

[0088] This embodiment obtains an information extraction identifier; extracts initial data information from an unstructured database according to the information extraction identifier; and preprocesses the initial data information to obtain standard data information that meets processing standards and is valid, so as to provide reliable data support for subsequent feature extraction and semantic representation.

[0089] Continue to refer Figure 4 In some optional implementations of this embodiment, step S20 includes the following steps:

[0090] Step S201, identifying the data type of the standard data information, wherein the data type includes: text data, image data, audio data, and video data;

[0091] In this embodiment, text data can be identified by querying information stored in the form of a character string, image data can be identified by querying pixel information arranged in the form of a two-dimensional array, audio data can be identified by querying a sample representation of the amplitude of a sound waveform changing over time, and video data can be identified by querying a series of continuous image frames.

[0092] Step S202, inputting the text data, the image data, the audio data, and the video data into a pre-trained feature extraction model for feature extraction, to obtain a text data feature vector, an image data feature vector, an audio data feature vector, and a video data feature vector;

[0093] In this embodiment, the pre-trained feature extraction model adopts a Transformer model and a convolutional neural network model, wherein the Transformer model specifically adopts a BERT model, the BERT model is used to extract feature vectors of text data, and the convolutional neural network model can adopt a three-dimensional convolutional neural network model, and the convolutional neural network model is used to extract feature vectors of image data, audio data, and video data.

[0094] Step S203, mapping the text data feature vector, the image data feature vector, the audio data feature vector, and the video data feature vector to a unified semantic vector space to obtain the semantic vector representation;

[0095] In this embodiment, the semantic vector representation is used to represent the semantic relationship between text data feature vectors, image data feature vectors, audio data feature vectors, and video data feature vectors. The unified semantic vector space is a space used to represent and process the semantic information of multimodal data (such as text, images, audio, video, etc.). In the unified semantic vector space, data of different modalities are mapped to vector representations, and the semantic relationship between these vector representations can be evaluated by calculating similarity or distance. In this embodiment, before performing feature vector mapping, the feature vectors of each modality need to be normalized to ensure comparability in the unified semantic vector space.

[0096] Step S204, performing similarity calculation on the semantic vector representation to obtain semantic similarity;

[0097] In this embodiment, semantic similarity is used to measure the closeness of semantic vector representations corresponding to text, image, audio or video data at the semantic level, and the closeness of semantic vector representations at the semantic level can be obtained by cosine similarity calculation.

[0098] Step S205, performing cluster analysis according to the semantic similarity to obtain a semantic association relationship;

[0099] In this embodiment, according to the calculated semantic similarity, data with high semantic similarity is determined as having high association, and data with low semantic similarity is determined as having low association, thereby obtaining the semantic association relationship between the data after cluster analysis.

[0100] Step S206: construct the semantic index according to the semantic association relationship and the semantic vector representation.

[0101] In this embodiment, a semantic index is constructed based on the semantic vector representation and the semantic association relationship. The semantic index includes not only the vocabulary index information, but also the semantic vector representation and the semantic association relationship information of the data.

[0102] This embodiment identifies the data type of the standard data information, wherein the data type includes: text data, image data, audio data, and video data; inputs the text data, image data, audio data, and video data into a pre-trained feature extraction model for feature extraction to obtain a text data feature vector, an image data feature vector, an audio data feature vector, and a video data feature vector; maps the text data feature vector, the image data feature vector, the audio data feature vector, and the video data feature vector to a unified semantic vector space to obtain the semantic vector representation; performs similarity calculation on the semantic vector representation to obtain semantic similarity; performs cluster analysis based on the semantic similarity to obtain semantic association relationships; and constructs the semantic index based on the semantic association relationships and the semantic vector representation. This effectively obtains a semantic index related to the semantic similarity, semantic association relationships, and semantic vector representation of the standard data information to facilitate subsequent semantic understanding and expansion processing.

[0103] Continue to refer Figure 5 In some optional implementations of this embodiment, step S30 includes the following steps:

[0104] Step S301, obtaining a knowledge graph extraction identifier, and extracting a pre-constructed knowledge graph from a database according to the knowledge graph extraction identifier;

[0105] In this embodiment, the knowledge graph extraction identifier is the unique identification information corresponding to the pre-built knowledge graph. The knowledge graph extraction identifier can be a serial number, UUID (Universally Unique Identifier), hash value, or a unique code generated by a specific algorithm.

[0106] Step S302, performing semantic understanding on the input search information based on the pre-built knowledge graph to obtain input search semantic information;

[0107] In this embodiment, semantic understanding of the input search information includes the following steps: Named Entity Recognition (NER): identifying entities with specific meanings from the input search information, such as names of people, places, and institutions; Entity Linking: matching the identified entities with entities in the knowledge graph to establish links between entities; Relationship Extraction: extracting relationships between entities from the input text. These relationships can be explicit (such as "A is the founder of B") or implicit (need to be derived through contextual reasoning); Semantic Role Labeling: analyzing the predicate-argument structure in the input text and clarifying the semantic relationships in the sentence, such as agent, patient, and instrument; Constructing a query statement: constructing a statement for querying in the knowledge graph based on the results of the above semantic role labeling, and executing the query statement in the knowledge graph to obtain entity, relationship, and attribute information related to the user query, thereby obtaining input search semantic information.

[0108] Step S303: semantically expand the input search semantic information based on the pre-constructed knowledge graph to obtain the search information representation.

[0109] In this embodiment, the step of semantically expanding the input search semantic information includes analyzing the initial result set: conducting an in-depth analysis of the input search semantic information to identify the key entities, relationships and connection paths involved therein; associating related entities and relationships: using the semantic relationships in the knowledge graph to extract information about other entities and relationships associated with the entities in the input search semantic information. The extracted information may include directly related entities (such as entities connected by specific relationships), parent or child entities in the category hierarchy, entities with similar or related attributes, etc.; evaluating relevance: evaluating the relevance of the expanded entities and relationships to ensure that they maintain a certain degree of relevance with the user's original query intent, integrating the information that has passed the evaluation, and generating a search information representation.

[0110] This embodiment obtains a knowledge graph extraction identifier, extracts a pre-constructed knowledge graph from a database according to the knowledge graph extraction identifier; performs semantic understanding on the input search information based on the pre-constructed knowledge graph to obtain input search semantic information; performs semantic expansion on the input search semantic information based on the pre-constructed knowledge graph, thereby effectively obtaining a search information representation based on the semantic understanding and semantic expansion of the input search information, so as to facilitate subsequent conversion processing of the search semantic vector.

[0111] Continue to refer Figure 6 In some optional implementations of this embodiment, step S40 includes the following steps:

[0112] Step S401, mapping the retrieval information representation to a low-dimensional semantic space to obtain the retrieval semantic vector;

[0113] In this embodiment, the retrieval semantic vector is used to reflect the semantic similarities and differences between words. For each word (or entity) in the retrieval information identifier, an embedding model is used to map it to a low-dimensional semantic space, thereby obtaining a retrieval semantic vector represented by a low-dimensional vector. Among them, an embedding model refers to a method of converting high-dimensional, discrete or symbolic data into a low-dimensional continuous vector representation, which is implemented by a neural network or other machine learning algorithm, and aims to capture the intrinsic features and relationships in the original data that are difficult to directly measure and calculate. In this embodiment, the embedding model can use the Word2Vec word embedding model.

[0114] Step S402, extracting a semantic vector representation from the semantic index, and performing cosine similarity calculation on the search semantic vector and the semantic vector representation to obtain a similarity score;

[0115] In this embodiment, the semantic index itself contains a semantic vector representation, which can be obtained from the semantic index through a query. The cosine similarity calculation is a measure of the degree of similarity between the semantic vector representation and the retrieval semantic vector in the direction. For the semantic vector representation A and the retrieval semantic vector B, the cosine similarity cosine_similarity(A, B) can be calculated by the following formula:

[0116] cosine_similarity(A,B)=A·B / ∥A∥∥B∥

[0117] Among them, A·B represents the dot product (inner product) of vectors A and B, and ∥A∥ and ∥B∥ respectively represent the modulus (length) of vectors A and B. After calculating the cosine similarity, the cosine similarity is converted into a corresponding similarity score, which can be a percentage value or a decimal value.

[0118] Step S403, searching corresponding valid data information from the standard data information according to the semantic vector representation;

[0119] In this embodiment, the semantic vector representation and the standard data information are in a corresponding relationship, and a matching query can be performed in the standard data information through the information identifier corresponding to the semantic vector representation to obtain valid data information.

[0120] Step S404, sorting the valid data information according to the similarity score to obtain a sorted data information set, obtaining a preset information set threshold, and screening the sorted data information set according to the preset information set threshold to obtain the candidate data set.

[0121] In this embodiment, valid data information is sorted in order from high to low according to the similarity score to obtain a sorted data information set. The preset information set threshold is an initially set data set screening quantity value. The preset information set threshold is used to perform quantitative screening on the sorted data information set. For example, the preset information set threshold is initially set to 20, then the first 20 data are selected from the sorted data information set in the sorted order and saved, and the data after 20 in the sorted order are discarded to obtain a candidate data set.

[0122] This embodiment obtains the retrieval semantic vector by mapping the retrieval information representation to a low-dimensional semantic space; extracts the semantic vector representation from the semantic index, calculates the cosine similarity between the retrieval semantic vector and the semantic vector representation, and obtains a similarity score; searches for corresponding valid data information from the standard data information according to the semantic vector representation; sorts the valid data information according to the similarity score to obtain a sorted data information set, obtains a preset information set threshold, and filters the sorted data information set according to the preset information set threshold, thereby effectively obtaining a candidate data set filtered according to the similarity between the corresponding vector of the retrieval information representation and the corresponding vector of the standard data information, so as to facilitate subsequent priority sorting processing.

[0123] Continue to refer Figure 7 In some optional implementations of this embodiment, step S50 includes the following steps:

[0124] Step S501, obtaining the input search information, and calculating the semantic relevance score between the input search information and the candidate data set;

[0125] In this embodiment, the attention mechanism is used to calculate the semantic relevance score of the input retrieval information and the candidate data set. Specifically, the retrieval semantic vector corresponding to the input retrieval information and the semantic vector representation corresponding to the candidate data set can be used for calculation, and each vector of the retrieval semantic vector is dot-producted with each vector of the semantic vector representation, and the attention weight is obtained by normalization through the softmax function. The calculated attention weight is used to perform weighted summation on the vector representation of the candidate data set to obtain the weighted candidate data set vector, and then the cosine similarity of the weighted candidate data set vector and the retrieval semantic vector is calculated, and the calculated cosine similarity value is used as the semantic relevance score.

[0126] Step S502, obtaining the authoritative information and timeliness information of the candidate data set, and calculating the data ranking score of the candidate data set according to the authoritative information, the timeliness information, and the semantic relevance score;

[0127] In this embodiment, authoritative information can be obtained by checking the data publishing platform, checking the specific content of the data, referring to academic literature and journals, etc., and timeliness information can be obtained by checking the data release date, checking the data update record, referring to industry reports and dynamics, evaluating data application scenarios, etc. The authoritative information and timeliness information can be quantified, and the authoritative information weight, timeliness information weight, and semantic relevance score corresponding to the authoritative information, timeliness information, and semantic relevance score can be used for weighted calculation to obtain the data ranking score. In this embodiment, the authoritative information weight is set to 0.3, the timeliness information weight is set to 0.3, and the semantic relevance score weight is set to 0.4, which can be set and adjusted accordingly according to actual conditions.

[0128] Step S503: Prioritize the candidate data sets according to the data ranking scores to obtain the search result data set.

[0129] In this embodiment, the data in the candidate data set are sorted in order from high to low according to the data sorting scores to obtain a sorted retrieval result data set.

[0130] This embodiment obtains the input search information, calculates the semantic relevance score between the input search information and the candidate data set; obtains the authoritative information and timeliness information of the candidate data set, and calculates the data ranking score of the candidate data set based on the authoritative information, the timeliness information, and the semantic relevance score; and prioritizes the candidate data sets based on the data ranking score, thereby effectively obtaining a search result data set that is sorted according to the authoritative information, timeliness information, and semantic relevance corresponding to the candidate data set, so as to facilitate returning the search result data set obtained by searching according to the input search information to a preset display interface for display.

[0131] In some optional implementations of this embodiment, after step S50, the following steps are further included:

[0132] Obtaining a preset relevance score threshold, and determining whether the semantic relevance score corresponding to the search result data set is less than the preset relevance score threshold;

[0133] In this embodiment, the preset relevance score threshold is a preset threshold used to determine whether the relevance between the search result data set and the input search information meets the required value. In this embodiment, the preset relevance score threshold is 70, which can be set and adjusted accordingly according to actual conditions.

[0134] If the semantic relevance score corresponding to the retrieval result data set is less than the preset relevance score threshold, semantic expansion and inference are performed on the input retrieval information to obtain an extended retrieval term, and then the corresponding extended retrieval result data is searched in the retrieval result data set according to the extended retrieval term, and the extended retrieval result data is re-sorted based on the data sorting score to obtain an extended retrieval result data set, and the extended retrieval result data set is returned to the display interface for display.

[0135] In this embodiment, semantic expansion and reasoning of input search information is realized based on the pre-built knowledge graph, and the input search information is semantically expanded and reasoned twice through the pre-built knowledge graph to obtain the expanded search terms. Among them, semantic expansion includes synonym and near-synonym expansion: finding entities that are synonymous or near-synonymous with the original keywords; related concept expansion: based on the relationship in the knowledge graph, discover other concepts or entities related to the original keywords. For example, if the original keyword is "apple", it can be expanded to related concepts such as "fruit" and "healthy food"; context expansion: considering the context of the original query, generate more specific expanded search terms. For example, if the context is "medical", "apple" may be expanded to "the impact of apples on health". Reasoning refers to reasoning using the relationships in the knowledge graph. For example, if the knowledge graph contains relationships such as "A is a subclass of B" or "C is a component of D", new query terms or query conditions can be derived based on these relationships.

[0136] If the semantic relevance score corresponding to the retrieval result data set is greater than or equal to the preset relevance score threshold, the retrieval result data set is added to the data list of the display page to be sent, and the data list is waited to be sent.

[0137] In this embodiment, when the data list sends the search result data set to the display interface, the display interface performs data rendering according to a preset page data rendering rule to effectively display the search result data set on the display interface.

[0138] This embodiment obtains a preset relevance score threshold and determines whether the semantic relevance score corresponding to the retrieval result data set is less than the preset relevance score threshold; if the semantic relevance score corresponding to the retrieval result data set is less than the preset relevance score threshold, semantic expansion and inference are performed on the input retrieval information to obtain an extended retrieval term, and then the corresponding extended retrieval result data is searched in the retrieval result data set according to the extended retrieval term, and the extended retrieval result data is re-sorted based on the data sorting score to obtain an extended retrieval result data set, and the extended retrieval result data set is returned to the display interface for display. Thus, when the semantic relevance corresponding to the retrieval result data set is low, an extended retrieval result data set that is more corresponding to the input retrieval information is obtained.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0140] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0141] Further references Figure 8 , as a response to the above Figure 1 In order to realize the method shown in the figure, the present application provides an embodiment of a data retrieval device, and the embodiment of the device is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0142] like Figure 8 As shown, the data retrieval device 600 described in this embodiment includes: an information acquisition module 601, an index construction module 602, an information expansion module 603, an information screening module 604, and a data set display module 605. Among them:

[0143] Information acquisition module 601, used to acquire standard data information;

[0144] An index building module 602 is used to perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and to build a semantic index according to the semantic vector representation;

[0145] The information expansion module 603 is used to obtain input search information, perform semantic understanding and expansion on the input search information, and obtain search information representation;

[0146] An information screening module 604 is used to convert the search information representation into a search semantic vector, calculate the similarity between the search semantic vector and the semantic index, and screen the standard data information according to the similarity to obtain a candidate data set;

[0147] The data set display module 605 is used to prioritize the candidate data sets, obtain a search result data set, and return the search result data set to the display interface for display.

[0148] This embodiment, by adopting the above-mentioned data retrieval device, can obtain standard data information; perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and construct a semantic index based on the semantic vector representation; obtain input retrieval information, perform semantic understanding and expansion on the input retrieval information to obtain a retrieval information representation; convert the retrieval information representation into a retrieval semantic vector, and calculate the similarity between the retrieval semantic vector and the semantic index, filter and process the standard data information based on the similarity to obtain a candidate data set; prioritize the candidate data sets to obtain a retrieval result data set, and return the retrieval result data set to the display interface for display. In this way, a retrieval result data set with high content relevance and high retrieval accuracy can be effectively obtained based on the input retrieval information.

[0149] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0150] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 7 with components 71-73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital signal processors (Digital Signal Processor, DSP), embedded devices, etc.

[0151] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0152] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 7. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions of the data retrieval method, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0153] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run the computer-readable instructions stored in the memory 71 or process data, such as computer-readable instructions for running the data retrieval method.

[0154] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0155] This embodiment, by using the above-mentioned computer device, can obtain standard data information; perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and construct a semantic index based on the semantic vector representation; obtain input search information, perform semantic understanding and expansion on the input search information to obtain a search information representation; convert the search information representation into a search semantic vector, and calculate the similarity between the search semantic vector and the semantic index, filter and process the standard data information based on the similarity to obtain a candidate data set; prioritize the candidate data sets to obtain a search result data set, and return the search result data set to the display interface for display. In this way, a search result data set with high content relevance and high search accuracy can be effectively obtained based on the input search information.

[0156] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data retrieval method as described above.

[0157] This embodiment, by using the above-mentioned computer-readable storage medium, can obtain standard data information; perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and construct a semantic index based on the semantic vector representation; obtain input search information, perform semantic understanding and expansion on the input search information to obtain a search information representation; convert the search information representation into a search semantic vector, and calculate the similarity between the search semantic vector and the semantic index, filter and process the standard data information based on the similarity to obtain a candidate data set; prioritize the candidate data sets to obtain a search result data set, and return the search result data set to the display interface for display. In this way, a search result data set with high content relevance and high search accuracy can be effectively obtained based on the input search information.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0159] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0160] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A data retrieval method, characterized in that: The steps include: Obtain standard data information; Performing feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructing a semantic index based on the semantic vector representation; Acquire input search information, perform semantic understanding and expansion on the input search information, and obtain search information representation; Converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set; The candidate data sets are prioritized to obtain a search result data set, and the search result data set is returned to a display interface for display.

2. The data retrieval method according to claim 1, characterized in that: The step of obtaining standard data information specifically includes: Get information extraction identification; Extracting initial data information from an unstructured database according to the information extraction identifier; The initial data information is preprocessed to obtain the standard data information.

3. The data retrieval method according to claim 1, characterized in that: The step of performing feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and constructing a semantic index according to the semantic vector representation specifically includes: Identify the data type of the standard data information, wherein the data type includes: text data, image data, audio data, and video data; Inputting the text data, the image data, the audio data, and the video data into a pre-trained feature extraction model for feature extraction to obtain a text data feature vector, an image data feature vector, an audio data feature vector, and a video data feature vector; Mapping the text data feature vector, the image data feature vector, the audio data feature vector, and the video data feature vector to a unified semantic vector space to obtain the semantic vector representation; Performing similarity calculation on the semantic vector representation to obtain semantic similarity; Performing cluster analysis based on the semantic similarity to obtain semantic association relationships; The semantic index is constructed according to the semantic association relationship and the semantic vector representation.

4. The data retrieval method according to claim 1, characterized in that: The step of obtaining input search information, semantically understanding and expanding the input search information, and obtaining the search information representation specifically includes: Obtain a knowledge graph extraction identifier, and extract a pre-constructed knowledge graph from a database according to the knowledge graph extraction identifier; Performing semantic understanding on the input search information based on the pre-built knowledge graph to obtain input search semantic information; The input search semantic information is semantically expanded based on the pre-constructed knowledge graph to obtain the search information representation.

5. The data retrieval method according to claim 1, characterized in that: The step of converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set specifically includes: Mapping the retrieval information representation to a low-dimensional semantic space to obtain the retrieval semantic vector; Extracting a semantic vector representation from the semantic index, and performing cosine similarity calculation on the retrieved semantic vector and the semantic vector representation to obtain a similarity score; Searching for corresponding valid data information from the standard data information according to the semantic vector representation; The valid data information is sorted according to the similarity score to obtain a sorted data information set, and a preset information set threshold is obtained. The sorted data information set is screened according to the preset information set threshold to obtain the candidate data set.

6. The data retrieval method according to claim 1, characterized in that: The step of prioritizing the candidate data sets to obtain the search result data set specifically includes: Obtaining the input search information, and calculating a semantic relevance score between the input search information and the candidate data set; Obtaining authoritative information and timeliness information of the candidate data set, and calculating a data ranking score of the candidate data set according to the authoritative information, the timeliness information, and the semantic relevance score; The candidate data sets are prioritized according to the data ranking scores to obtain the retrieval result data set.

7. The data retrieval method according to claim 1, characterized in that: After the step of prioritizing the candidate data sets according to the data ranking scores to obtain the search result data set, the following steps are also included: Obtaining a preset relevance score threshold, and determining whether the semantic relevance score corresponding to the search result data set is less than the preset relevance score threshold; If the semantic relevance score corresponding to the retrieval result data set is less than the preset relevance score threshold, semantic expansion and inference are performed on the input retrieval information to obtain an extended retrieval term, and then the corresponding extended retrieval result data is searched in the retrieval result data set according to the extended retrieval term, and the extended retrieval result data is re-sorted based on the data sorting score to obtain an extended retrieval result data set, and the extended retrieval result data set is returned to the display interface for display.

8. A data retrieval device, characterized in that: include: An information acquisition module is used to acquire standard data information; An index building module, used to perform feature extraction and semantic representation on the standard data information to obtain a semantic vector representation, and to build a semantic index based on the semantic vector representation; An information expansion module is used to obtain input search information, perform semantic understanding and expansion on the input search information, and obtain a search information representation; An information screening module, used for converting the retrieval information representation into a retrieval semantic vector, calculating the similarity between the retrieval semantic vector and the semantic index, and screening the standard data information according to the similarity to obtain a candidate data set; The data set display module is used to prioritize the candidate data sets to obtain a search result data set, and return the search result data set to a display interface for display.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data retrieval method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data retrieval method according to any one of claims 1 to 7 are implemented.

Images