A data retrieval method, device, storage medium and electronic device
By retrieving and filtering candidate information units in databases with different storage modalities, the problems of high retrieval complexity and data omission caused by structural and semantic differences are solved, and efficient multimodal data query is achieved.
Patent Information
- Application Number
- CN202611138059.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-29
- Publication Date
- 2026-11-06
AI Technical Summary
When retrieving data from databases with different storage modalities, structural and semantic differences lead to high retrieval complexity and are prone to data omissions or errors.
By responding to query commands, candidate information units are retrieved from databases of different storage modalities to form a sequence of candidate information units. Target information units are then filtered based on confidence conditions. The target information content is determined in each database modality using a unique identifier, and the query results are output.
It reduces the complexity of user searches, eliminates the risk of data omissions and errors, and enables efficient querying of multimodal data.
Smart Images
Figure CN123048591A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a data retrieval method, apparatus, storage medium, and electronic device. Background Technology
[0002] With the continuous development of science and information technology, data is often stored in databases with different storage modes to meet the needs of storing and processing different types of data, such as vector storage mode, key-value storage mode, and graph storage mode.
[0003] In related technologies, when users retrieve data according to their needs, they usually need to initiate searches in different storage modalities to obtain all the data related to their needs from the databases of different storage modalities, so as to provide a complete data foundation for users to view and analyze the data in the future.
[0004] However, due to the differences in structure, presentation and semantics of data stored in databases of different storage modalities, users need to perform secondary verification of the relevant data after obtaining it, which greatly increases the complexity of retrieval and is prone to data omissions or errors due to human misunderstanding. Summary of the Invention
[0005] To address the aforementioned technical problems, embodiments of this application provide a data retrieval method, apparatus, storage medium, and electronic device.
[0006] According to one aspect of the embodiments of this application, a data retrieval method is provided, comprising: retrieving candidate information units related to the query instruction from databases corresponding to different storage modalities in response to a query instruction, to form a candidate information unit sequence; determining target information units whose confidence levels meet preset confidence conditions from the candidate information unit sequence; determining target information content related to the unique identifier in the database of each storage modal based on the unique identifier of the target information unit, so as to output a query result corresponding to the query instruction according to the target information content.
[0007] Through the above embodiments, during the retrieval process, candidate information units related to the query command are retrieved from the databases corresponding to different storage modalities in response to the query command, forming a sequence of candidate information units. Then, target information units whose confidence levels meet the preset confidence conditions are determined from the candidate information unit sequence. Finally, based on the unique identifier of the target information unit, the target information content related to the unique identifier is determined in the database of each storage modal, and the query result corresponding to the query command is output according to the target information content. This integrates information scattered in different storage modalities into a consistent query result, so that users do not need to perform secondary verification after obtaining the query result, which greatly reduces the user's retrieval complexity, eliminates the risk of data omission or error caused by human comparison and misunderstanding, and realizes efficient querying of multimodal data.
[0008] In some embodiments of this application, based on the foregoing scheme, in response to a query instruction, candidate information units related to the query instruction are retrieved from the databases corresponding to the different storage modes to form a sequence of candidate information units, including: for each storage mode, generating a retrieval command corresponding to the storage mode based on the query instruction; and retrieving relevant information units from the databases corresponding to the storage mode through the retrieval command as candidate information units.
[0009] Through the above embodiments, for each storage mode, a retrieval command corresponding to the storage mode is generated based on the query instruction, and then relevant information units are retrieved from the database corresponding to the respective storage mode as candidate information units through their respective retrieval commands, so as to achieve the purpose of determining the information units related to the query instruction under different storage modes.
[0010] In some embodiments of this application, based on the aforementioned scheme, different storage modes include vector storage mode, key-value pair storage mode, and relational graph storage mode.
[0011] In some embodiments of this application, based on the foregoing scheme, generating a retrieval command corresponding to the storage modality based on the query instruction includes: in the vector storage modality, obtaining a query instruction vector corresponding to the query instruction, and generating a retrieval command corresponding to the query instruction in the vector storage modality based on the query instruction vector; in the key-value pair storage modality, obtaining a query instruction key in the query instruction, and generating a retrieval command corresponding to the query instruction in the key-value pair storage modality based on the query instruction key; and in the relational graph storage modality, obtaining a query instruction node and a query instruction relationship corresponding to the query instruction, and generating a retrieval command corresponding to the query instruction in the relational graph storage modality based on the query instruction node and the query instruction relationship.
[0012] The above embodiments are used to generate retrieval commands for query instructions under different storage modes.
[0013] In some embodiments of this application, based on the aforementioned scheme, determining a target information unit whose confidence level meets a preset confidence condition from the candidate information unit sequence includes: determining the confidence level corresponding to each candidate information unit in the candidate information unit sequence; and determining a candidate information unit whose confidence level reaches the confidence threshold in the preset confidence condition from the candidate information unit sequence as the target information unit.
[0014] Through the above embodiments, candidate information units that meet the retrieval accuracy requirements are selected from the candidate information unit sequence and used as target information units for subsequent output of query results, thereby improving the overall retrieval accuracy.
[0015] In some embodiments of this application, based on the foregoing scheme, determining the confidence level corresponding to each candidate information unit in the candidate information unit sequence includes: determining the confidence score of the candidate information unit in each storage mode; determining the weight score of the candidate information unit in each storage mode; and determining the confidence level of the candidate information unit based on the confidence score and the weight score.
[0016] The confidence level of each candidate information unit is calculated through the above embodiments.
[0017] In some embodiments of this application, based on the foregoing scheme, the data retrieval method further includes: obtaining multiple token fragments corresponding to the long document to be stored; configuring a unique identifier corresponding to each of the token fragments, and generating a storage information unit for each of the token fragments under different storage modes based on the unique identifier; storing the storage information unit for each of the token fragments under different storage modes in the databases corresponding to the different storage modes, so as to store the long document to be stored.
[0018] Through the above embodiments, each information unit under different storage modes can be configured with its own unique identifier, thereby facilitating unified management and retrieval across storage modes based on the unique identifier.
[0019] According to one aspect of the embodiments of this application, a data retrieval apparatus is provided, comprising: a retrieval module configured to retrieve candidate information units related to the query instruction from databases corresponding to different storage modes in response to a query instruction, thereby forming a candidate information unit sequence; a filtering module configured to determine target information units whose confidence levels meet preset confidence conditions from the candidate information unit sequence; and an output module configured to determine target information content related to the unique identifier in the database of each storage mode based on the unique identifier of the target information unit, thereby outputting a query result corresponding to the query instruction based on the target information content.
[0020] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the data retrieval method as described in the above embodiments.
[0021] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the data retrieval method as described in the above embodiments. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a flowchart illustrating a data retrieval method in an exemplary embodiment of this application; Figure 2 This is a block diagram illustrating a data retrieval device in an exemplary embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application. Detailed Implementation
[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.
[0024] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0026] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0027] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "OR" relationship.
[0028] The technical solution of this application embodiment proposes a data retrieval method, which is detailed below. Figure 1 As shown. The method includes at least steps S110 to S130, which are described in detail below: In step S110, in response to the query command, candidate information units related to the query command are retrieved from the databases corresponding to different storage modes to form a sequence of candidate information units.
[0029] In the embodiments of this application, after a user initiates a query command, the system can retrieve candidate information units related to the query command from the databases corresponding to different storage modes in response to the query command, so as to form a sequence of candidate information units.
[0030] In some embodiments of this application, the process of retrieving candidate information units related to the query command from the databases corresponding to different storage modes in response to the query command can be carried out by generating a retrieval command corresponding to each storage mode based on the query command, and then retrieving relevant information units from the databases corresponding to the storage mode as candidate information units through the retrieval command.
[0031] For example, under different storage modalities, including vector storage modality, key-value pair storage modality, and graph storage modality, generating a retrieval command corresponding to the storage modality based on the query instruction may include the following steps: In vector storage mode, obtain the query instruction vector corresponding to the query instruction, and generate the retrieval command corresponding to the query instruction in vector storage mode based on the query instruction vector; In the key-value pair storage mode, the query instruction key value in the query instruction is obtained, and the corresponding retrieval command in the key-value pair storage mode is generated based on the query instruction key; In the relational graph storage mode, the query instruction node and query instruction relationship corresponding to the query instruction are obtained, and the retrieval command corresponding to the query instruction in the relational graph storage mode is generated based on the query instruction node and query instruction relationship.
[0032] In the above steps, a pre-defined language parsing model can parse the query command into a semantic vector, which serves as the query command vector and can be directly used as a retrieval command in the vector storage mode. Subsequently, a pre-defined rule parsing model extracts the query constraints and query intent from the query command. Based on the query constraints, the query command key and query command node are determined, and based on the query intent, the query command relationship is determined. Finally, the determined query command key can be directly used as a retrieval command in the key-value pair storage mode, and the determined query command node and query command relationship, combined, can serve as a retrieval command in the relational graph storage mode.
[0033] Specifically, in one example, if the query is "What are the differences between the batteries of car model 1 and car model 2?", then the preset language parsing model and preset rule parsing model can be used to obtain " The query instruction vector is "=[0.12,-0.34,0.56,...]", the query intent is "comparison", and the query constraint is "battery of model 1, battery of model 2".
[0034] Based on this, " "The query is directly used as the retrieval command in the vector storage modality. Then, the query constraint "battery of model 1, battery of model 2" determines the key ("model 1" and "model 2") in the key-value pair storage modality. The field of interest is determined by "battery". The key and the field of interest are combined as the query command key, which is then used as the retrieval command in the key-value pair storage modality. Subsequently, the query constraint "battery of model 1, battery of model 2" determines the nodes ("battery of model 1" and "battery of model 2") in the relational graph storage modality, which are then used as the query command nodes. The relation type in the relational graph storage modality is determined by the query intent "comparison". The relation direction is determined by the order in the query constraints. The relation type and relation direction are combined as the query command relation. Finally, the query command node and the query command relation are combined as the retrieval command in the relational graph storage modality."
[0035] In some embodiments of this application, in order to form a candidate information unit sequence, a unique identifier for each candidate information unit can be determined first, and then clustering matching can be performed based on the unique identifier of each candidate information unit to deduplicate all candidate information units. Then, all candidate information units obtained after deduplication can be used as a candidate information unit sequence.
[0036] It should be noted that the unique identifier of each candidate information unit is configured when it is stored in the database of each storage mode. Specifically, in the embodiments of this application, multiple token fragments corresponding to the long document to be stored can be obtained, then a unique identifier corresponding to each token fragment can be configured, and information units to be stored under different storage modes can be generated based on the unique identifier. Finally, the information units to be stored under different storage modes can be stored in the databases corresponding to different storage modes to store the long document to be stored.
[0037] Among them, the long documents to be stored are the long documents that need to be stored in databases of different storage modes.
[0038] The multiple token fragments corresponding to the long document to be stored are obtained by segmenting the long document to be stored into a token sequence according to preset segmentation rules (such as the number of tokens, structural organization markers, etc.) after the token sequence is obtained by word encoding.
[0039] Based on the aforementioned scheme, under different storage modes, including vector storage mode, key-value pair storage mode, and relational graph storage mode, the process of generating the information unit to be stored for each token fragment under different storage modes based on a unique identifier may include the following steps: In the vector storage mode, the information vector features of the token fragment are obtained, and the information vector features and the unique identifier of the token fragment are used as the information unit to be stored corresponding to the token fragment in the vector storage mode. In the key-value pair storage mode, extract the key-value pair fields from the token fragment, and use the key-value pair fields and the unique identifier of the token fragment as the information unit to be stored corresponding to the token fragment in the key-value pair storage mode; In the relational graph storage mode, the information association features between token fragments and other token fragments are determined based on the information vector features and key-value pair fields of each token fragment. The information association features and the unique identifier of the token fragment are used as the information unit to be stored corresponding to the token fragment in the relational graph storage mode.
[0040] In step S120, target information units whose confidence levels meet preset confidence conditions are determined from the candidate information unit sequence.
[0041] In the embodiments of this application, after forming a sequence of candidate information units, target information units whose confidence levels meet preset confidence conditions can be determined from the candidate information unit sequence. The confidence level is used to characterize the degree of relevance between the candidate information unit and the query instruction.
[0042] In one example, the method of determining the target information unit whose confidence level meets the preset confidence condition from the candidate information unit sequence can be as follows: first, determine the confidence level corresponding to each candidate information unit in the candidate information unit sequence; then, determine the candidate information unit whose confidence level reaches the confidence threshold in the preset confidence condition from the candidate information unit sequence, and use it as the target information unit.
[0043] For example, if the confidence threshold is set to 0.7, and the candidate information unit sequence includes candidate information units A, B, and C, with corresponding confidence levels of 0.85, 0.62, and 0.91 respectively, then the confidence levels of candidate information units A and C reach the confidence threshold of 0.7, and therefore candidate information units A and C can be determined as target information units.
[0044] In some embodiments of this application, the process of determining the confidence level of each candidate information unit in the candidate information unit sequence can first determine the confidence score of the candidate information unit in each storage mode, then determine the weight score of the candidate information unit in each storage mode, and finally determine the confidence level of the candidate information unit based on the confidence score and the weight score.
[0045] For example, the confidence level of candidate information unit i can be calculated using the following formula: ; in, Let i be the confidence level of candidate information unit i. Let i be the weight score of candidate information unit i in the vector storage mode. Let i be the confidence score of candidate information unit i in the vector storage mode. This represents the weight score of candidate information unit i in the key-value pair storage mode. The confidence score of candidate information unit i in the key-value pair storage mode. Let i be the weight score of candidate information unit i in the relational graph storage mode. Let be the confidence score of candidate information unit i in the relational graph storage mode.
[0046] In the above process, the confidence score of a candidate information unit in the vector storage mode can be based on the cosine similarity between the information unit and the query command vector; the confidence score of a candidate information unit in the key-value pair storage mode can be based on the field hit rate of the information unit under the retrieval command in the key-value pair storage mode; and the confidence score of a candidate information unit in the relation graph storage mode can be based on the preset relation weight of the relation edge corresponding to the information unit under the retrieval command in the relation graph storage mode.
[0047] In the above process, the weight scores for different storage modes can be either directly preset to fixed values or matched according to the query type of the query instruction. Specifically, the query type of the query instruction can be determined first, and then the weights corresponding to that query type in the preset memory can be used as the weight scores. For example, when the query type is a single-point retrieval, the corresponding weight scores of the candidate information units under different storage modes can be... =0.4、 =0.4、 =0.2, to improve the weight score of candidate information units in both vector storage mode and key-value pair storage mode. When the query type is differential retrieval, the corresponding weight scores of candidate information units in different storage modes can be... =0.2、 =0.2、 =0.6, to improve the weight score of candidate information units in the relational graph storage mode.
[0048] In another example, the confidence level of each candidate information unit in the candidate information unit sequence can be determined first, and then the candidate information unit with the highest confidence level can be selected as the target information unit. In other words, the pre-set confidence condition is to select the candidate information unit with the highest confidence level.
[0049] For example, if the candidate information unit sequence includes candidate information units D, E, and F, and their corresponding confidence levels are 0.75, 0.88, and 0.65, respectively, then candidate information unit E has the highest confidence level, and can be selected as the target information unit.
[0050] In step S130, the target information content associated with the unique identifier is determined in the database of each storage mode based on the unique identifier of the target information unit, so as to output the query result corresponding to the query instruction according to the target information content.
[0051] In the embodiments of this application, after the target information unit is determined, the target information content related to the unique identifier can be determined in the database of each storage mode based on the unique identifier of the target information unit, so as to output the query result corresponding to the query instruction according to the target information content.
[0052] In some examples, the process of determining the target information content associated with the unique identifier in the database of each storage modality based on the unique identifier of the target information unit can use the unique identifier of the target information unit as a global retrieval command, and initiate a retrieval in the database of each storage modality based on the global retrieval command, so as to obtain the relevant information content from the databases of different storage modalities as the target information content.
[0053] For example, in vector storage mode, the information vector features corresponding to the information unit are obtained based on the unique identifier in the global retrieval command, thus retrieving the original text fragment. In key-value pair storage mode, the key-value pair fields corresponding to the information unit are obtained based on the unique identifier in the global retrieval command, thus retrieving the structured text fragment. In relational graph storage mode, the information association features corresponding to the information unit are obtained based on the unique identifier in the global retrieval command, thus retrieving the differentiated text fragment. Thus, the goal of retrieving relevant information content of the target information unit from databases with different storage modes under different storage modes is achieved.
[0054] In the above process, the method of outputting query results corresponding to the query command based on the target information content can be achieved through a preset language processing model. This model performs semantic understanding and information integration on the target information content, ultimately generating a natural language response to the query command, which is then output to the user as the query result. This preset language processing model can include LLM (Large Language Model), NLP (Natural Language Processing), NLG (Natural Language Generation), and others.
[0055] Through the above implementation method, in response to the query command, candidate information units related to the query command are first retrieved from the databases corresponding to different storage modalities to form a candidate information unit sequence. Then, target information units whose confidence levels meet the preset confidence conditions are determined from the candidate information unit sequence. Finally, based on the unique identifier of the target information unit, the target information content related to the unique identifier is determined in the database of each storage modal. The query result corresponding to the query command is output according to the target information content. In this way, information scattered in different storage modalities is integrated into a consistent query result, so that users do not need to perform secondary verification after obtaining the query result, which greatly reduces the user's retrieval complexity, eliminates the risk of data omission or error caused by human comparison and misunderstanding, and realizes efficient querying of multimodal data.
[0056] The following describes an embodiment of the apparatus described in this application, which can be used to execute the data retrieval method described above. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the data retrieval method described above.
[0057] Figure 2 A block diagram of a data retrieval apparatus 100 according to an embodiment of this application is shown.
[0058] Reference Figure 2 As shown, a data retrieval apparatus 100 according to an embodiment of this application includes: a retrieval module 110 configured to retrieve candidate information units related to the query command from databases corresponding to different storage modes in response to a query command, so as to form a candidate information unit sequence; a filtering module 120 configured to determine target information units whose confidence levels meet preset confidence conditions from the candidate information unit sequence; and an output module 130 configured to determine target information content related to the unique identifier in the database of each storage mode based on the unique identifier of the target information unit, so as to output the query result corresponding to the query command according to the target information content.
[0059] In some embodiments of this application, based on the foregoing scheme, the retrieval module 110 is further configured to: generate a retrieval command corresponding to each storage mode based on a query instruction; and retrieve relevant information units from the database corresponding to the storage mode using the retrieval command as candidate information units.
[0060] In some embodiments of this application, based on the aforementioned scheme, different storage modes include vector storage mode, key-value pair storage mode, and relational graph storage mode.
[0061] In some embodiments of this application, based on the foregoing scheme, the retrieval module 110 is further configured to: in vector storage mode, obtain the query instruction vector corresponding to the query instruction, and generate a retrieval command corresponding to the query instruction in vector storage mode based on the query instruction vector; in key-value pair storage mode, obtain the query instruction key in the query instruction, and generate a retrieval command corresponding to the query instruction in key-value pair storage mode based on the query instruction key; in relational graph storage mode, obtain the query instruction node and query instruction relationship corresponding to the query instruction, and generate a retrieval command corresponding to the query instruction in relational graph storage mode based on the query instruction node and query instruction relationship.
[0062] In some embodiments of this application, based on the aforementioned scheme, the screening module 120 is further configured to: determine the confidence level corresponding to each candidate information unit in the candidate information unit sequence; and determine the candidate information units whose confidence level reaches the confidence threshold in the preset confidence conditions from the candidate information unit sequence as target information units.
[0063] In some embodiments of this application, based on the foregoing scheme, the screening module 120 is further configured to: determine the confidence score of the candidate information unit in each storage mode; determine the weight score of the candidate information unit in each storage mode; and determine the confidence level of the candidate information unit based on the confidence score and the weight score.
[0064] In some embodiments of this application, based on the foregoing scheme, the data retrieval device may further include: a storage module configured to: acquire multiple token fragments corresponding to a long document to be stored; configure a unique identifier corresponding to each token fragment, and generate a storage information unit for each token fragment in different storage modes based on the unique identifier; and store the storage information unit for each token fragment in different storage modes in the databases corresponding to the different storage modes, so as to store the long document to be stored.
[0065] It should be noted that the data retrieval device 100 provided in the above embodiments and the data retrieval method provided in the above embodiments belong to the same concept. The specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.
[0066] Embodiments of this application also provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the data retrieval method as described above.
[0067] Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.
[0068] It should be noted that, Figure 3 The computer system 200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0069] like Figure 3 As shown, the computer system 200 includes a Central Processing Unit (CPU) 201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 202 or programs loaded from storage portion 208 into Random Access Memory (RAM) 203, such as performing the methods described in the above embodiments. The RAM 203 also stores various programs and data required for system operation. The CPU 201, ROM 202, and RAM 203 are interconnected via a bus 204. An Input / Output (I / O) interface 205 is also connected to the bus 204.
[0070] The following components are connected to I / O interface 205: an input section 206 including a keyboard, mouse, etc.; an output section 207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to I / O interface 205 as needed. Removable media 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 210 as needed so that computer programs read from them can be installed into storage section 208 as needed.
[0071] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by central processing unit (CPU) 201, it performs various functions defined in the system of this application.
[0072] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.
[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0074] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.
[0075] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.
[0076] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0077] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.
[0078] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.
[0079] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A data retrieval method, characterized in that, The method includes: In response to a query command, candidate information units related to the query command are retrieved from databases corresponding to different storage modes to form a sequence of candidate information units; From the candidate information unit sequence, target information units whose confidence levels meet the preset confidence conditions are determined; Based on the unique identifier of the target information unit, the target information content related to the unique identifier is determined in the database of each storage mode, so as to output the query result corresponding to the query instruction according to the target information content.
2. The method according to claim 1, characterized in that, The step of responding to a query command by retrieving candidate information units related to the query command from databases corresponding to different storage modes to form a sequence of candidate information units includes: For each of the aforementioned storage modes, a retrieval command corresponding to the storage mode is generated based on the query instruction; The search command retrieves relevant information units from the database corresponding to the storage mode, which are then used as candidate information units.
3. The method according to claim 2, characterized in that, The different storage modes include vector storage mode, key-value pair storage mode, and graph storage mode.
4. The method according to claim 3, characterized in that, The step of generating the retrieval command corresponding to the storage modality based on the query instruction includes: In the vector storage mode, the query instruction vector corresponding to the query instruction is obtained, and the retrieval command corresponding to the query instruction in the vector storage mode is generated based on the query instruction vector; In the key-value pair storage mode, the query instruction key in the query instruction is obtained, and the retrieval command corresponding to the query instruction in the key-value pair storage mode is generated based on the query instruction key; In the relational graph storage mode, the query instruction node and query instruction relationship corresponding to the query instruction are obtained, and the retrieval command corresponding to the query instruction in the relational graph storage mode is generated based on the query instruction node and the query instruction relationship.
5. The method according to claim 1, characterized in that, The step of determining the target information unit whose confidence level meets the preset confidence condition from the candidate information unit sequence includes: Determine the confidence level corresponding to each candidate information unit in the candidate information unit sequence; Candidate information units whose confidence level reaches the confidence threshold in the preset confidence condition are determined from the candidate information unit sequence and are used as the target information units.
6. The method according to claim 5, characterized in that, Determining the confidence level corresponding to each candidate information unit in the candidate information unit sequence includes: Determine the confidence score of the candidate information unit in each of the storage modes; Determine the weight score of the candidate information unit in each of the storage modes; The confidence level of the candidate information unit is determined based on the confidence score and the weight score.
7. The method according to claim 1, characterized in that, The method further includes: Retrieve multiple token fragments corresponding to the long document to be stored; Configure the unique identifier corresponding to each of the token fragments, and generate the information unit to be stored for each of the token fragments in different storage modes based on the unique identifier; Each token fragment is stored in a database corresponding to a different storage mode, and the information unit to be stored under each storage mode is stored in the database corresponding to the different storage modes, so as to store the long document to be stored.
8. A data retrieval device, characterized in that, include: The retrieval module is configured to retrieve candidate information units related to the query command from the databases corresponding to the different storage modes in response to the query command, so as to form a sequence of candidate information units; The filtering module is configured to determine target information units whose confidence levels meet preset confidence conditions from the candidate information unit sequence; The output module is configured to determine the target information content related to the unique identifier in the database of each storage mode based on the unique identifier of the target information unit, so as to output the query result corresponding to the query instruction according to the target information content.
9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the data retrieval method as described in any one of claims 1 to 7.
10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the steps of the data retrieval method as described in any one of claims 1 to 7.