Data retrieval method and device, computer device and storage medium
By receiving retrieval requests, extracting and hashing the retrieval primary key, and querying the weighted linked list, the problem of reduced data retrieval speed is solved, and the effect of quickly locating important data is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA PING AN PROPERTY INSURANCE CO LTD
- Filing Date
- 2022-11-17
- Publication Date
- 2026-05-29
AI Technical Summary
As the amount of data increases, the speed of data retrieval decreases, and existing technologies are unable to effectively improve the efficiency of data retrieval.
By receiving a retrieval request, extracting the retrieval primary key and generating its hash value, querying the weighted linked list, determining the target node based on the retrieval configuration information and weights, and generating data retrieval results.
It enables rapid location of important data and improves the speed of data retrieval.
Smart Images
Figure CN115757682B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data technology, and in particular to a data retrieval method, apparatus, computer equipment, and storage medium. Background Technology
[0002] With the widespread application of computer technology, the amount of data generated in production and daily life is increasing, leading to an information explosion. Search engines can help people obtain information by pre-storing large amounts of data and then retrieving and returning the retrieved data based on search requests. However, as the amount of stored data increases, the amount of data that needs to be searched during data retrieval also increases, inevitably leading to a decrease in data retrieval speed. Summary of the Invention
[0003] The purpose of this application is to provide a data retrieval method, apparatus, computer device, and storage medium to improve the speed of data retrieval.
[0004] To address the aforementioned technical problems, this application provides a data retrieval method, employing the following technical solution:
[0005] Receive a search request, wherein the search request includes search information and search configuration information;
[0006] Extract the primary key from the search information;
[0007] Generate the retrieval primary key hash value of the retrieval primary key;
[0008] Query the linked list corresponding to the primary key hash value, wherein the primary key hash value of the linked list corresponds to the primary key hash value, and the nodes in the linked list are arranged according to weight;
[0009] Based on the retrieval configuration information and the weight, a target node is determined in the linked list, and data retrieval results are generated based on the stored data in the target node.
[0010] To address the aforementioned technical problems, this application also provides a data retrieval device, which employs the following technical solution:
[0011] A request receiving module is used to receive a search request, wherein the search request includes search information and search configuration information;
[0012] The primary key extraction module is used to extract the retrieval primary key from the retrieval information;
[0013] A hash value generation module is used to generate the retrieval primary key hash value of the retrieval primary key;
[0014] The linked list query module is used to query the linked list corresponding to the retrieval primary key hash value, wherein the linked list primary key hash value corresponds to the retrieval primary key hash value, and the nodes in the linked list are arranged according to weight;
[0015] The result generation module is used to determine the target node in the linked list according to the retrieval configuration information and the weight, and generate data retrieval results based on the stored data in the target node.
[0016] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0017] Receive a search request, wherein the search request includes search information and search configuration information;
[0018] Extract the primary key from the search information;
[0019] Generate the retrieval primary key hash value of the retrieval primary key;
[0020] Query the linked list corresponding to the primary key hash value, wherein the primary key hash value of the linked list corresponds to the primary key hash value, and the nodes in the linked list are arranged according to weight;
[0021] Based on the retrieval configuration information and the weight, a target node is determined in the linked list, and data retrieval results are generated based on the stored data in the target node.
[0022] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0023] Receive a search request, wherein the search request includes search information and search configuration information;
[0024] Extract the primary key from the search information;
[0025] Generate the retrieval primary key hash value of the retrieval primary key;
[0026] Query the linked list corresponding to the primary key hash value, wherein the primary key hash value of the linked list corresponds to the primary key hash value, and the nodes in the linked list are arranged according to weight;
[0027] Based on the retrieval configuration information and the weight, a target node is determined in the linked list, and data retrieval results are generated based on the stored data in the target node.
[0028] Compared with the prior art, the embodiments of this application have the following main advantages: Receiving a retrieval request containing retrieval information and retrieval configuration information, where the retrieval information is the content to be retrieved and the retrieval configuration information is used to filter the retrieval results; extracting the retrieval primary key representing key information from the retrieval information and generating its corresponding retrieval primary key hash value; the hash value enables rapid location, thus quickly finding the linked list corresponding to the retrieval primary key hash value, wherein the linked list primary key hash value corresponds to the retrieval primary key hash value, the linked list primary key hash value represents the hash value of key information in the linked list, and the nodes in the linked list are arranged according to weight, the weight reflecting the importance of the data stored in the node; based on the retrieval configuration information and weight, the target node containing important information can be quickly selected from the linked list, and data retrieval results are generated based on the stored data in the target node, greatly improving the speed of data retrieval. Attached Figure Description
[0029] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;
[0031] Figure 2 This is a flowchart of an embodiment of the data retrieval method according to this application;
[0032] Figure 3 This is a schematic diagram of the structure of one embodiment of the data retrieval device according to this application;
[0033] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation
[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0035] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0036] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0037] like Figure 1 As shown, system architecture 100 may include terminal devices 101 and 102, server 103, data storage servers 104, 105, and 106, and network 107. Network 107 serves as the medium for providing communication links between terminal devices 101 and 102, server 103, and data storage servers 104, 105, and 106. Network 107 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0038] Users can use terminal devices 101 and 102 to interact with server 103 via network 107 to receive or send messages, etc. For example, terminals 101 and 102 can generate business-related data to be processed and send the data to server 103. Various communication client applications can be installed on terminal devices 101 and 102, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.
[0039] Terminal devices 101 and 102 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.
[0040] Server 103 and data storage servers 104, 105, and 106 form a distributed node network, where server 103 and data storage servers 104, 105, and 106 can act as nodes in the network. Server 103 plays a control role, receiving data to be stored and storing it in the data storage servers, and retrieving data from data storage servers 104, 105, and 106 based on retrieval requests. In one embodiment, server 103 can be a server selected from the data storage servers, serving both a control role and data storage capabilities.
[0041] It should be noted that the data retrieval method provided in this application embodiment is generally executed by a server, and correspondingly, the data retrieval device is generally located in the server.
[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0043] Continue to refer to Figure 2 A flowchart of an embodiment of the data retrieval method according to this application is shown. The data retrieval method includes the following steps:
[0044] Step S201: Receive a search request, wherein the search request includes search information and search configuration information.
[0045] In this embodiment, the data retrieval method operates on an electronic device (e.g., Figure 1 The server shown can communicate with the terminal device via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, Wi-Fi connections, Zigbee connections, UWB (ultra-Width band) connections, and other currently known or future wireless connection methods.
[0046] Specifically, the server first receives a search request and then performs a data retrieval based on it. The search request includes search information and search configuration information. The search information can be the raw content to be searched; for example, a user entering the sentence "What functions does product A include?" can serve as the search information. The search configuration information can be generated based on the search information or pre-configured information. It can also be entered by the user; for example, the user can select the level of detail in the search. Higher levels of detail result in more data being displayed. The search configuration information is used to filter the initial search results.
[0047] Furthermore, before step S201 above, the method may include: obtaining the search text input by the user; generating a search request based on the search text; or, when a triggered recommendation instruction is received, obtaining user information; inputting the user information into a pre-trained recommendation model to obtain a recommendation evaluation result; and generating a search request based on the recommendation evaluation result.
[0048] Specifically, users can input search text through their terminals, and the server generates a search request based on the received search text. Users can also select the level of detail for the search through their terminals, and the server uses the search text as search information in the search request, generating search configuration information according to the level of detail selected by the user. In one embodiment, users may also choose not to directly select the level of detail, and the server will automatically determine the level of detail; the longer the search text, the higher the level of detail.
[0049] This application can also push information to users, perform searches based on user information, and push the search results to the user. The server receives a triggered recommendation instruction, which is associated with a specific user. Based on the recommendation instruction, the server obtains the user's information, which can be basic user information or a user profile. Then, the user information is input into a recommendation model. The recommendation model is built on a neural network and pre-trained to obtain a recommendation evaluation result. The recommendation evaluation result may include the predicted type of product or information to recommend to the user. Based on the recommendation evaluation result, a search request can be generated to retrieve information related to the recommendation evaluation result and push it to the user.
[0050] In this embodiment, a search request can be generated based on the search text input by the user, or based on the recommendation evaluation results of the recommendation model for the user, thus enriching the ways to generate search requests.
[0051] Step S202: Extract the search key from the search information.
[0052] Specifically, the primary key is extracted from the search information and can be key information within the search information.
[0053] Furthermore, step S202 above may include: segmenting the search information to obtain multiple sub-words; determining keywords among the multiple sub-words based on their TF-IDF information; and generating a search primary key based on the determined keywords.
[0054] Specifically, the retrieved information can be in text form. Artificial intelligence techniques can be used to segment the retrieved information into multiple sub-words, and then the TF-IDF value of each sub-word is calculated to obtain the TF-IDF information of the retrieved information. TF-IDF (Term Frequency-Inverse Document Frequency) is a commonly used weighting technique in information retrieval and data mining. TF stands for Term Frequency, which refers to the frequency of a given word appearing in a document. IDF is the Inverse Document Frequency, which can be obtained by dividing the total number of documents by the number of documents containing a certain word and then taking the logarithm.
[0055] TF-I DF is used to assess the importance of a term to the retrieved information. Keywords can be determined from the retrieved information based on the TF-I DF value. For example, the TF-I DF value of a term can be compared with a preset threshold. If the TF-I DF value is greater than the preset threshold, the term can be used as a keyword; or the term with the highest TF-I DF value can be selected as the keyword.
[0056] In this embodiment, the importance of each sub-word in the search information is evaluated based on IF-TDF information, thereby enabling the selection of keywords from the search information.
[0057] Step S203: Generate the retrieval primary key hash value.
[0058] Specifically, a hash operation is performed on the primary key to obtain the hash value of the primary key. A hash operation transforms an input of arbitrary length into a fixed-length output using a hash algorithm; this output is the hash value. This transformation is a compression mapping, a function that compresses a message of arbitrary length into a message digest of a fixed length.
[0059] Step S204: Query and retrieve the linked list corresponding to the primary key hash value, wherein the primary key hash value in the linked list corresponds to the retrieved primary key hash value, and the nodes in the linked list are arranged according to their weights.
[0060] Specifically, the stored data can be stored in a linked list. The linked list structure in this application is specially designed, and the linked list has a linked list primary key hash value, which is generated by hashing the linked list primary key. The linked list primary key can be key information within the linked list. The linked list includes multiple nodes, each containing node data and a pointer. The node data stores the data, and the pointer points to the position of the next node in the linked list. In this application, each node has a weight, which measures the importance of the data stored in the node, or the correlation between the data stored in the node and the linked list primary key. The nodes in the linked list are sorted according to their weights; the higher the weight, the closer the node is to the node storing the linked list primary key, thus allowing for faster retrieval of more important stored data.
[0061] The server performs a fast search across all linked lists based on the primary key hash value. The primary key hash value of the retrieved linked list corresponds to the primary key hash value. Since both the primary key hash value and the linked list primary key hash value are obtained through hash operations, fast location can be achieved using the hash value. The server can retrieve multiple linked lists by searching the primary key hash value.
[0062] Step S205: Based on the retrieval configuration information and weights, determine the target node in the linked list, and generate data retrieval results based on the stored data in the target node.
[0063] Specifically, the retrieval configuration information can include weight thresholds or weight conditions. Target nodes are determined in the linked list based on this configuration information. For example, when the retrieval configuration information is a weight threshold, nodes with weights greater than or equal to the threshold are selected as target nodes. When the retrieval configuration information is a weight condition, nodes whose weights meet the weight condition are selected as target nodes. For example, based on the weight condition, the weights of each node are sorted in descending order, and then the nodes ranking in the top N% are selected as target nodes. The higher the level of retrieval detail selected by the user, the larger the value of N, or the lower the weight threshold.
[0064] In one embodiment, the retrieval configuration information can also be type-related information. The data stored in the node has different types, such as personal information data or social information data. The node containing the specific type of data can be selected as the target node based on the retrieval configuration information.
[0065] The stored data in the selected target nodes is displayed according to a preset format to obtain the data retrieval results.
[0066] In this embodiment, a retrieval request containing retrieval information and retrieval configuration information is received. The retrieval information is the content to be retrieved, and the retrieval configuration information is used to filter the retrieval results. A retrieval primary key representing key information is extracted from the retrieval information, and its corresponding retrieval primary key hash value is generated. The hash value enables rapid location, allowing for quick retrieval of the linked list corresponding to the retrieval primary key hash value. The linked list primary key hash value corresponds to the retrieval primary key hash value, representing the hash value of key information within the linked list. Nodes in the linked list are arranged according to weights, reflecting the importance of the data stored in each node. Based on the retrieval configuration information and weights, the target node containing important information can be quickly selected from the linked list. Data retrieval results are generated based on the stored data in the target node, significantly improving the speed of data retrieval.
[0067] Furthermore, before step S204 above, the method may include: acquiring the data to be stored; generating the knowledge graph corresponding to the data to be stored; determining the master node and each slave node in the knowledge graph, and determining the weights corresponding to the master node and each slave node respectively; generating a circular doubly linked list based on the master node and its weight, and each slave node and its weight, wherein each slave node is sorted according to its weight, and the hash value of the primary key of the circular doubly linked list is generated based on the master node.
[0068] Specifically, the process involves acquiring the data to be stored, identifying the entities within the data, and determining the relationships between entities. Entities are then used as nodes, and relationships are used as connecting edges between nodes to construct a knowledge graph corresponding to the data to be stored.
[0069] Then, using a pre-defined node classification strategy, the nodes in the knowledge graph are divided into master nodes and slave nodes. Master nodes are key nodes in the knowledge graph and have higher importance. There must be at least one master node and at least one slave node. Then, based on a pre-defined weight determination strategy, the weights of master nodes and slave nodes are determined. The magnitude of the weight reflects the importance of the node; understandably, the weight of a master node is greater than that of a slave node. For slave nodes, the weight also reflects the degree of correlation between the slave node and the master node.
[0070] After obtaining the master node and its corresponding weight, and the slave nodes and their corresponding weights, a linked list can be constructed. The linked list in this application can be a circular doubly linked list, possessing characteristics of both circular and doubly linked lists. The linked list includes multiple nodes, with the master node located at the beginning of the list. Slave nodes are sorted according to their weights; the higher the weight, the closer it is to the master node. The circular doubly linked list has a primary key, generated from the master node. A hash operation is performed on the primary key to obtain its hash value, which can be used as the address and entry point of the linked list.
[0071] In one embodiment, after extracting the retrieval key from the retrieval information, a preset number of synonyms or near-synonyms of the retrieval key are searched according to a preset thesaurus. Then, the retrieval key and its corresponding synonyms or near-synonyms are sorted according to preset conditions to obtain a word sequence (e.g., sorted according to pinyin order). Then, a hash operation is performed on the word sequence to obtain the retrieval key hash value. A linked list key hash value can be generated based on the master node in the same way, so that the retrieval key hash value and the linked list key hash value can cover more key information.
[0072] In this embodiment, a knowledge graph corresponding to the data to be stored is generated, and the master node, slave node and their respective weights are determined in the knowledge graph. The weights represent the importance of the nodes. A circular doubly linked list is generated based on each node and its weight. In the linked list, the slave nodes are sorted according to their weights to ensure that important data can be retrieved quickly. The hash value of the primary key of the circular doubly linked list generated based on the master node is used as the address and entry point of the linked list, so that the linked list can be located quickly.
[0073] Furthermore, the steps of determining the master node and each slave node in the knowledge graph, and determining the weights corresponding to the master node and each slave node respectively, may include: calculating the node evaluation value of each node in the knowledge graph based on a preset ranking algorithm; determining the master node and each slave node in the knowledge graph based on the node evaluation value; determining the weights corresponding to the master node and each slave node respectively based on the node evaluation value; or, determining the weights corresponding to the master node and each slave node respectively based on the node type of the master node and each slave node.
[0074] Specifically, a preset ranking algorithm is used to calculate the node evaluation value of each node in the knowledge graph. The node evaluation value reflects the importance and influence of the node. In one embodiment, the ranking algorithm can be the PageRank algorithm. PageRank, also known as webpage ranking, webpage level, Google left-side ranking, or PageRank, is a technique that calculates based on the hyperlinks between webpages. The PageRank algorithm can calculate the influence of nodes in the network using a random walk method.
[0075] The master and slave nodes in the knowledge graph can be determined based on the node evaluation value. The node corresponding to the highest node evaluation value can be directly determined as the master node; or, a preset evaluation value threshold can be obtained, and the nodes whose evaluation value is greater than the evaluation value threshold can be determined as the master nodes; or, the nodes whose evaluation value ranks in the top N% (N is a positive integer) can be selected as the master nodes, and the remaining nodes in the knowledge graph can be determined as slave nodes.
[0076] When master and slave nodes are stored as nodes in a linked list, they are assigned weights. These weights can be determined by the node evaluation values of the master and slave nodes within the knowledge graph; alternatively, they can be identified by their node types. A node type refers to the type of data the node represents; for example, when the master node represents a person's name, the node type could be "person," and when the slave node represents gender, the node type could be "sex." The server can be pre-configured with weights corresponding to different node types, thus allowing the determination of the weights for master and slave nodes.
[0077] In this embodiment, a ranking algorithm is used to calculate the node evaluation value of each node in the knowledge graph. The node evaluation value reflects the importance of the node, thereby determining the master node and slave node. The weight of a node when it is stored in a linked list can be determined based on the node evaluation value or the node type, which enriches the way the weight value is determined.
[0078] Furthermore, the steps described above for generating a circular doubly linked list based on the master node and its weight, and each slave node and its weight may include: storing the master node as the master node of the initial linked list, and storing each slave node as a slave node of the initial linked list; generating pointers to each node based on the storage locations and weights of the master node and each slave node, wherein the nodes include the master node and each slave node, and the pointers include storage pointers and weights; in the predecessor and successor linked lists of the master node, each slave node is sorted in descending order according to the weight of its corresponding slave node; determining the master node as the primary key of the linked list, and generating a hash value of the primary key of the linked list, thus obtaining a circular doubly linked list.
[0079] The initial linked list can be a linked list that has not yet been fully constructed.
[0080] Specifically, the linked list includes a master node and slave nodes. The master node stores the data represented by the master node in the knowledge graph, and the slave nodes store the data represented by the slave nodes in the knowledge graph. In the knowledge graph, nodes are connected by edges, which represent the relationships between nodes. These edges are also stored in the nodes to ensure data integrity.
[0081] The master and slave nodes have actual storage locations in storage resources (such as hard disks) and also have weights. Pointers to each node can be generated based on the storage location and weight. The pointers in this application are divided into two parts: a storage pointer and a weight. The storage pointer is used to point to the actual storage locations of the master and slave nodes.
[0082] In a linked list, the master node is located at the beginning or center, and the slave nodes are arranged on either side of the master node, forming the predecessor linked list and the successor linked list, respectively. In the predecessor and successor linked lists, the slave nodes are arranged in descending order of their weight values. That is, in either the predecessor or successor linked list, the smaller the weight value of a slave node, the more slave node pointers need to be consulted to find that node, starting from the master node. When constructing the initial linked list, the master node is obtained first, and then slave nodes are generated sequentially according to their weight values. Slave nodes in the predecessor and successor linked lists can be generated alternately.
[0083] The data in the master node is used as the primary key of the linked list, and a hash value of the primary key is generated. The hash value of the primary key is used as the entry point of the circular doubly linked list.
[0084] In this embodiment, the master node is stored as the master node of the initial linked list, and each slave node is stored as a slave node. Pointers to each node are generated based on the storage location and weight of the master node and each slave node. Each pointer includes a storage pointer and a weight; the storage pointer represents the actual storage location of the node, and the weight is used to sort the nodes. In the predecessor and successor linked lists of the master node, each slave node is sorted in descending order according to its corresponding slave node weight, so that the node storing important data can be quickly found during retrieval. The master node is determined as the primary key of the linked list, and a hash value of the primary key is generated to obtain the linked list entry point, thus completing the construction of the circular doubly linked list.
[0085] Furthermore, the above method may also include: compressing the data to be stored to obtain compressed data; and storing the compressed data in the form of a hi ve table into the nodes of the linked list.
[0086] Specifically, the server can compress the data to be stored to reduce its size, and then store the compressed data in the form of a hi ve table in the nodes of a linked list. In this case, the core of each node is the address of the hi ve table.
[0087] In this embodiment, the data to be stored is compressed and then stored in the nodes of the linked list in the form of a hi ve table, which reduces the volume occupied by data storage.
[0088] It should be emphasized that, to further ensure the privacy and security of the aforementioned chained lists, they can also be stored in a node of a blockchain.
[0089] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0090] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0091] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0093] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0094] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data retrieval device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0095] like Figure 3 As shown, the data retrieval device 300 described in this embodiment includes: a request receiving module 301, a primary key extraction module 302, a hash value generation module 303, a linked list query module 304, and a result generation module 305, wherein:
[0096] The request receiving module 301 is used to receive a search request, wherein the search request includes search information and search configuration information.
[0097] The primary key extraction module 302 is used to extract the primary key from the search information.
[0098] The hash value generation module 303 is used to generate the hash value of the primary key for retrieval.
[0099] The linked list query module 304 is used to query and retrieve the linked list corresponding to the primary key hash value. The primary key hash value in the linked list corresponds to the primary key hash value to be retrieved, and the nodes in the linked list are arranged according to their weights.
[0100] The result generation module 305 is used to determine the target node in the linked list based on the retrieval configuration information and weight, and generate data retrieval results based on the stored data in the target node.
[0101] In this embodiment, a retrieval request containing retrieval information and retrieval configuration information is received. The retrieval information is the content to be retrieved, and the retrieval configuration information is used to filter the retrieval results. A retrieval primary key representing key information is extracted from the retrieval information, and its corresponding retrieval primary key hash value is generated. The hash value enables rapid location, allowing for quick retrieval of the linked list corresponding to the retrieval primary key hash value. The linked list primary key hash value corresponds to the retrieval primary key hash value, representing the hash value of key information within the linked list. Nodes in the linked list are arranged according to weights, reflecting the importance of the data stored in each node. Based on the retrieval configuration information and weights, the target node containing important information can be quickly selected from the linked list. Data retrieval results are generated based on the stored data in the target node, significantly improving the speed of data retrieval.
[0102] In some optional implementations of this embodiment, the data retrieval device 300 may further include: a text acquisition module, a first generation module, and an information acquisition module, or an information input module and a second generation module, wherein:
[0103] The text acquisition module is used to acquire the search text input by the user.
[0104] The first generation module is used to generate a search request based on the search text.
[0105] The information acquisition module is used to acquire user information when a triggered recommendation instruction is received.
[0106] The information input module is used to input user information into a pre-trained recommendation model to obtain recommendation evaluation results.
[0107] The second generation module is used to generate search requests based on the recommendation evaluation results.
[0108] In this embodiment, a search request can be generated based on the search text input by the user, or based on the recommendation evaluation results of the recommendation model for the user, thus enriching the ways to generate search requests.
[0109] In some optional implementations of this embodiment, the primary key extraction module 302 may include: a retrieval and word segmentation submodule, a keyword determination submodule, and a primary key generation submodule.
[0110] in:
[0111] The retrieval and word segmentation submodule is used to segment the retrieved information into multiple sub-words.
[0112] The keyword determination submodule is used to determine the keywords among multiple sub-words based on their TF-IDF information.
[0113] The primary key generation submodule is used to generate a retrieval primary key based on a given set of keywords.
[0114] In this embodiment, the importance of each sub-word in the search information is evaluated based on IF-TDF information, thereby enabling the selection of keywords from the search information.
[0115] In some optional implementations of this embodiment, the data retrieval device 300 may further include: a data acquisition module, a map generation module, a node determination module, and a linked list generation module, wherein:
[0116] The data acquisition module is used to acquire data to be stored.
[0117] The knowledge graph generation module is used to generate the knowledge graph corresponding to the data to be stored.
[0118] The node determination module is used to determine the master node and each slave node in the knowledge graph, and to determine the weights corresponding to the master node and each slave node.
[0119] The linked list generation module is used to generate a circular doubly linked list based on the master node and its weight, and each slave node and its weight. The slave nodes are sorted according to their weights, and the primary key hash value of the circular doubly linked list is generated based on the master node.
[0120] In this embodiment, a knowledge graph corresponding to the data to be stored is generated, and the master node, slave node and their respective weights are determined in the knowledge graph. The weights represent the importance of the nodes. A circular doubly linked list is generated based on each node and its weight. In the linked list, the slave nodes are sorted according to their weights to ensure that important data can be retrieved quickly. The hash value of the primary key of the circular doubly linked list generated based on the master node is used as the address and entry point of the linked list, so that the linked list can be located quickly.
[0121] In some optional implementations of this embodiment, the node determination module may include: an evaluation value determination submodule, a node determination submodule, a first determination submodule, and a second determination submodule, wherein:
[0122] The evaluation value determination submodule is used to calculate the node evaluation value of each node in the knowledge graph based on a preset ranking algorithm.
[0123] The node determination submodule is used to determine the master node and each slave node in the knowledge graph based on the node evaluation value.
[0124] The first determination submodule is used to determine the weights corresponding to the master node and each slave node by taking the node evaluation values corresponding to each master node and each slave node respectively.
[0125] The second determination submodule is used to determine the weights corresponding to the master node and each slave node based on the node types of the master node and each slave node.
[0126] In this embodiment, a ranking algorithm is used to calculate the node evaluation value of each node in the knowledge graph. The node evaluation value reflects the importance of the node, thereby determining the master node and slave node. The weight of a node when it is stored in a linked list can be determined based on the node evaluation value or the node type, which enriches the way the weight value is determined.
[0127] In some optional implementations of this embodiment, the linked list generation module may include: a node storage submodule, a pointer generation submodule, and a primary key determination submodule, wherein:
[0128] The node storage submodule is used to store the master node as the master node of the initial linked list and store each slave node as a slave node of the initial linked list.
[0129] The pointer generation submodule is used to generate pointers to each node based on the storage location and weight of the master node and each slave node. The node includes the master node and each slave node, and the pointer includes the storage pointer and weight. In the predecessor and successor linked lists of the master node, each slave node is sorted in descending order according to the weight of its corresponding slave node.
[0130] The primary key determination submodule is used to determine the primary key of the linked list and generate the hash value of the primary key of the linked list, resulting in a circular doubly linked list.
[0131] In this embodiment, the master node is stored as the master node of the initial linked list, and each slave node is stored as a slave node. Pointers to each node are generated based on the storage location and weight of the master node and each slave node. Each pointer includes a storage pointer and a weight; the storage pointer represents the actual storage location of the node, and the weight is used to sort the nodes. In the predecessor and successor linked lists of the master node, each slave node is sorted in descending order according to its corresponding slave node weight, so that the node storing important data can be quickly found during retrieval. The master node is determined as the primary key of the linked list, and a hash value of the primary key is generated to obtain the linked list entry point, thus completing the construction of the circular doubly linked list.
[0132] In some optional implementations of this embodiment, the data retrieval device 300 may further include: a data compression module and a compressed storage module, wherein:
[0133] The data compression module is used to compress the data to be stored, resulting in compressed data.
[0134] The compressed storage module is used to store compressed data in the form of hi ve tables in the nodes of a linked list.
[0135] In this embodiment, the data to be stored is compressed and then stored in the nodes of the linked list in the form of a hi ve table, which reduces the volume occupied by data storage.
[0136] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0137] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital digital processors (DSPs), embedded devices, etc.
[0138] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0139] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data retrieval methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.
[0140] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the data retrieval method.
[0141] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.
[0142] The computer device provided in this embodiment can execute the data retrieval method described above. The data retrieval method here can be any of the data retrieval methods described in the various embodiments above.
[0143] In this embodiment, a retrieval request containing retrieval information and retrieval configuration information is received. The retrieval information is the content to be retrieved, and the retrieval configuration information is used to filter the retrieval results. A retrieval primary key representing key information is extracted from the retrieval information, and its corresponding retrieval primary key hash value is generated. The hash value enables rapid location, allowing for quick retrieval of the linked list corresponding to the retrieval primary key hash value. The linked list primary key hash value corresponds to the retrieval primary key hash value, representing the hash value of key information within the linked list. Nodes in the linked list are arranged according to weights, reflecting the importance of the data stored in each node. Based on the retrieval configuration information and weights, the target node containing important information can be quickly selected from the linked list. Data retrieval results are generated based on the stored data in the target node, significantly improving the speed of data retrieval.
[0144] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data retrieval method described above.
[0145] In this embodiment, a retrieval request containing retrieval information and retrieval configuration information is received. The retrieval information is the content to be retrieved, and the retrieval configuration information is used to filter the retrieval results. A retrieval primary key representing key information is extracted from the retrieval information, and its corresponding retrieval primary key hash value is generated. The hash value enables rapid location, allowing for quick retrieval of the linked list corresponding to the retrieval primary key hash value. The linked list primary key hash value corresponds to the retrieval primary key hash value, representing the hash value of key information within the linked list. Nodes in the linked list are arranged according to weights, reflecting the importance of the data stored in each node. Based on the retrieval configuration information and weights, the target node containing important information can be quickly selected from the linked list. Data retrieval results are generated based on the stored data in the target node, significantly improving the speed of data retrieval.
[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0147] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. A data retrieval method, characterized in that, Includes the following steps: Receive a search request, wherein the search request includes search information and search configuration information; Extract the primary key from the search information; Generate the retrieval primary key hash value of the retrieval primary key; Query the linked list corresponding to the primary key hash value, wherein the primary key hash value of the linked list corresponds to the primary key hash value, and the nodes in the linked list are arranged according to weight; Based on the retrieval configuration information and the weight, a target node is determined in the linked list, and data retrieval results are generated based on the stored data in the target node; Prior to the step of querying the linked list corresponding to the retrieved primary key hash value, the method further includes: Retrieve the data to be stored; Generate a knowledge graph corresponding to the data to be stored; Determine the master node and each slave node in the knowledge graph, and determine the weights corresponding to the master node and each slave node respectively; A circular doubly linked list is generated based on the master node and its weight, and each slave node and its weight. The slave nodes are sorted according to their weights, and the hash value of the primary key of the circular doubly linked list is generated based on the master node.
2. The data retrieval method according to claim 1, characterized in that, Prior to the step of receiving the retrieval request, the method further includes: Get the search text entered by the user; A search request is generated based on the search text; or, When a recommendation instruction is received, user information is retrieved; The user information is input into a pre-trained recommendation model to obtain recommendation evaluation results; A search request is generated based on the recommended evaluation results.
3. The data retrieval method according to claim 1, characterized in that, The step of extracting the retrieval primary key from the retrieval information includes: The retrieved information is segmented into multiple sub-words; Based on the TF-IDF information of the multiple sub-words, the keywords among the multiple sub-words are determined; Generate a search primary key based on the determined keywords.
4. The data retrieval method according to claim 1, characterized in that, The steps of determining the master node and each slave node in the knowledge graph, and determining the weights corresponding to the master node and each slave node respectively, include: The node evaluation value of each node in the knowledge graph is calculated based on a preset ranking algorithm. The master node and each slave node in the knowledge graph are determined based on the node evaluation value; The node evaluation values corresponding to the master node and each slave node are determined as the weights corresponding to the master node and each slave node, respectively. or, The weights corresponding to the master node and each slave node are determined based on their node types.
5. The data retrieval method according to claim 1, characterized in that, The step of generating a circular doubly linked list based on the master node and its weight, and each slave node and its weight includes: Store the master node as the master node of the initial linked list, and store each slave node as a slave node of the initial linked list respectively; Based on the storage location and weight of the master node and each slave node, a pointer to each node is generated. The node includes the master node and each slave node, and the pointer includes a storage pointer and a weight. In the predecessor and successor linked lists of the master node, each slave node is sorted in descending order according to the weight of its corresponding slave node. The master node is determined as the primary key of the linked list, and the hash value of the primary key of the linked list is generated to obtain a circular doubly linked list.
6. The data retrieval method according to claim 1, characterized in that, The method further includes: The data to be stored is compressed to obtain compressed data; The compressed data is stored in the nodes of a linked list as a Hive table.
7. A data retrieval device, characterized in that, include: A request receiving module is used to receive a search request, wherein the search request includes search information and search configuration information; The primary key extraction module is used to extract the retrieval primary key from the retrieval information; A hash value generation module is used to generate the retrieval primary key hash value of the retrieval primary key; The linked list query module is used to query the linked list corresponding to the retrieval primary key hash value, wherein the linked list primary key hash value corresponds to the retrieval primary key hash value, and the nodes in the linked list are arranged according to weight; The result generation module is used to determine the target node in the linked list according to the retrieval configuration information and the weight, and generate data retrieval results according to the stored data in the target node; The data retrieval device further includes: a data acquisition module, a graph generation module, a node determination module, and a linked list generation module, wherein: The data acquisition module is used to acquire data to be stored. The knowledge graph generation module is used to generate the knowledge graph corresponding to the data to be stored. The node determination module is used to determine the master node and each slave node in the knowledge graph, and to determine the weights corresponding to the master node and each slave node respectively. The linked list generation module is used to generate a circular doubly linked list based on the master node and its weight, and each slave node and its weight. The slave nodes are sorted according to their weights, and the primary key hash value of the circular doubly linked list is generated based on the master node.
8. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the data retrieval method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data retrieval method as described in any one of claims 1 to 6.