Data retrieval method and device, storage medium and electronic equipment

Abstract

The invention relates to a data retrieval method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the steps that to-be-retrieved content is acquired, and multi-modal to-be-retrieved features of the to-be-retrieved content on multiple different dimensions are determined; determining target attention weights of the multi-modal to-be-retrieved features and an original document vector index in a preset three-dimensional feature vector container on a plurality of different dimensions; based on the target attention weights on the multiple different dimensions, determining a comprehensive matching score between the multi-modal to-be-retrieved feature and an original document vector index; and determining a target document vector index from the original document vector indexes based on the comprehensive matching score, and retrieving a result according to the target document vector index and the content corresponding to the to-be-retrieved content. According to the method and the device, the accuracy of the obtained content retrieval result is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of artificial intelligence technology, and more specifically, to a data retrieval method, a data retrieval device, a computer-readable storage medium, and a corresponding electronic device. Background Art

[0002] In existing data retrieval methods, a single semantic vector is usually used to match document retrieval results corresponding to the query content; however, in the specific matching process, due to the limitations of vector representation, the system may mistakenly match semantically similar documents and queries together, resulting in retrieval results that are inconsistent with the actual semantics, and thus making the accuracy of the obtained content retrieval results low.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a data retrieval method, a data retrieval device, a computer-readable storage medium and a corresponding electronic device, thereby overcoming, at least to a certain extent, the problem of low accuracy of content retrieval results obtained due to limitations and defects of related technologies.

[0005] According to one aspect of the present disclosure, there is provided a data retrieval method, comprising:

[0006] Acquire content to be retrieved, and determine multimodal features to be retrieved of the content to be retrieved in multiple different dimensions;

[0007] Determining target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions;

[0008] Determining a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights on the multiple different dimensions;

[0009] A target document vector index is determined from the original document vector index based on the comprehensive matching score, and a content retrieval result is generated according to the target document vector index and the content to be retrieved.

[0010] In an exemplary embodiment of the present disclosure, determining multimodal features of the content to be retrieved in multiple dimensions includes:

[0011] Performing embedding mapping processing on the content to be retrieved based on a preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension;

[0012] Extracting the current timestamp of the content to be retrieved, and generating a time feature of the content to be retrieved in a time dimension based on the current timestamp of the content to be retrieved;

[0013] The context level of the content to be retrieved is extracted based on a preset context level extraction model, and based on the context level of the content to be retrieved, context features to be retrieved of the content to be retrieved in a context dimension are generated.

[0014] In an exemplary embodiment of the present disclosure, the preset semantic embedding model includes multiple Transformer models and average pooling layers;

[0015] The embedding mapping process is performed on the content to be retrieved based on a preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension, including:

[0016] Performing word embedding on the content to be retrieved to obtain a word embedding vector, a word embedding matrix, and a position embedding matrix of the content to be retrieved;

[0017] Generate an embedding vector according to the word embedding vector, the word embedding matrix, and the position embedding matrix, and input the embedding vector into a first Transformer model to generate a first text semantic vector corresponding to the first Transformer model;

[0018] Inputting the first text semantic vector into a second Transformer model to generate a second text semantic vector corresponding to the second Transformer model, and repeating the generation process of the second text semantic vector in sequence to obtain text semantic vectors corresponding to other Transformer models;

[0019] A first current encoding vector of the content to be retrieved is generated based on the text semantic vector and embedding vector corresponding to each Transformer model, and the first current encoding vector is input into the average pooling layer to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension.

[0020] In an exemplary embodiment of the present disclosure, the original document vector index in the preset three-dimensional feature vector container includes an original semantic vector in a semantic dimension, an original time vector in a time dimension, and an original context vector in a context dimension;

[0021] Determining target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions includes:

[0022] Determining a semantic similarity value between the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector, and determining a first original attention weight on the semantic dimension according to the semantic similarity value;

[0023] Determining a time decay coefficient based on the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector, and determining a second original attention weight in the time dimension based on the time decay coefficient;

[0024] Determining a context difference coefficient between a context feature to be retrieved in the multimodal feature to be retrieved and the original context vector, and determining a third original attention weight on the context dimension according to the context difference coefficient;

[0025] The first original attention weight, the second original attention weight and the third original attention weight are normalized to obtain a first target attention weight, a second target attention weight and a third target attention weight.

[0026] In an exemplary embodiment of the present disclosure, normalizing the first original attention weight, the second original attention weight, and the third original attention weight to obtain a first target attention weight, a second target attention weight, and a third target attention weight includes:

[0027] Mapping the first original attention weight, the second original attention weight, and the third original attention weight to a quantum bit space;

[0028] Performing quantum state amplitude encoding on the first original attention weight, the second original attention weight, and the third original attention weight in the quantum bit space to obtain a quantum state;

[0029] A first target attention weight, a second target attention weight, and a third target attention weight are determined based on the quantum state.

[0030] In an exemplary embodiment of the present disclosure, determining a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights on the multiple different dimensions includes:

[0031] Determining an original semantic matching score on a semantic dimension based on the semantic features to be retrieved in the multimodal features to be retrieved and the original semantic vector in the original document vector index;

[0032] Determining an original time matching score in a time dimension according to a time feature to be retrieved in the multimodal feature to be retrieved and an original time vector in an original document vector index;

[0033] Determining an original context matching score on a context dimension based on the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector in the original document vector index;

[0034] The original semantic matching score and the first target attention weight, the original time matching score and the second target attention weight, the original context matching score and the third target attention weight are weightedly summed to obtain a comprehensive matching score between the multimodal feature to be retrieved and the original document vector index.

[0035] In an exemplary embodiment of the present disclosure, the content retrieval result corresponding to the target document vector index and the content to be retrieved includes:

[0036] Determining a target document corresponding to the target document vector index, and loading the target document;

[0037] The loaded target document and the content to be retrieved are input into a preset content generation model to obtain a content retrieval result corresponding to the content to be retrieved.

[0038] In an exemplary embodiment of the present disclosure, the original document vector index is obtained in the following manner:

[0039] Acquire an original document, and determine an original semantic vector in a semantic dimension, an original time vector in a time dimension, and an original context vector in a context dimension of the original document;

[0040] The original document vector index is generated according to the original semantic vector, the original time vector, and the original context vector, and the original document vector index is stored in a preset three-dimensional feature vector container.

[0041] In an exemplary embodiment of the present disclosure, determining the original semantic vector of the original document in the semantic dimension includes:

[0042] Performing embedding mapping processing on the original document based on a preset semantic embedding model to obtain a standard semantic vector of the original document in a semantic feature dimension;

[0043] Parameter perturbation is performed on the standard semantic vector to obtain an adversarially enhanced semantic vector, and an original semantic vector is generated based on the standard semantic vector and the adversarially enhanced semantic vector.

[0044] According to one aspect of the present disclosure, there is provided a data retrieval device, comprising:

[0045] A multimodal to-be-retrieved feature determination module, configured to obtain content to be retrieved and determine multimodal to-be-retrieved features of the content to be retrieved in multiple dimensions;

[0046] An attention weight determination module is used to determine target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions;

[0047] a comprehensive matching score determination module, configured to determine a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in the multiple different dimensions;

[0048] The content retrieval result generating module is used to determine a target document vector index from the original document vector index based on the comprehensive matching score, and generate a content retrieval result corresponding to the target document vector index and the content to be retrieved.

[0049] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned data retrieval methods.

[0050] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0051] processor; and

[0052] a memory for storing executable instructions of the processor;

[0053] The processor is configured to execute any one of the above-mentioned data retrieval methods by executing the executable instructions.

[0054] A data retrieval method provided by an embodiment of the present disclosure obtains content to be retrieved and determines multimodal features to be retrieved of the content to be retrieved in multiple different dimensions; then determines target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions; and then determines a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in multiple different dimensions; finally, determines a target document vector index from the original document vector index based on the comprehensive matching score; since the comprehensive matching score and the target document vector index can be determined from multiple different dimensions, the problem of semantic vector deviation caused by matching based on a single semantic vector, which results in a low accuracy of the obtained target document vector index, can be avoided; on the other hand, since the content retrieval result corresponding to the target document vector index and the content to be retrieved can be obtained, the accuracy of the obtained content retrieval result is improved.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0057] Figure 1 The following schematically shows a flow chart of a data retrieval method according to an exemplary embodiment of the present disclosure.

[0058] Figure 2 An exemplary diagram schematically illustrates a data retrieval system according to an exemplary embodiment of the present disclosure.

[0059] Figure 3 A structural example diagram of a preset semantic embedding model according to an example embodiment of the present disclosure is schematically shown.

[0060] Figure 4 An example diagram schematically illustrates an obtained original document vector index according to an example embodiment of the present disclosure.

[0061] Figure 5 A diagram schematically illustrates an example structure of a preset context-level extraction model according to an exemplary embodiment of the present disclosure.

[0062] Figure 6 A schematic diagram shows an example structure of a preset content generation model according to an exemplary embodiment of the present disclosure.

[0063] Figure 7 An exemplary diagram schematically illustrates a data retrieval device according to an exemplary embodiment of the present disclosure.

[0064] Figure 8 An exemplary diagram schematically illustrates an electronic device for implementing a data retrieval method according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0065] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0066] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0067] With the rapid development of artificial intelligence technology, Retrieval Augmented Generation (RAG) systems have been widely used in the field of natural language processing. However, existing RAG systems have the following technical drawbacks:

[0068] On the one hand, vector matching leads to semantic drift; that is, existing RAG systems usually use a single semantic vector to represent documents and queries, which easily leads to semantic drift, resulting in low accuracy of the semantic retrieval results; the semantic drift recorded here refers to the fact that in the vector matching process, due to the limitations of vector representation, the system may mistakenly match semantically similar documents and queries together, resulting in retrieval results that are inconsistent with the actual semantics. This problem not only reduces the accuracy of retrieval but also affects the quality of generated content.

[0069] On the other hand, static weight allocation cannot adapt to the query context; that is, the existing RAG system adopts a static method in weight allocation and cannot dynamically adjust the importance weights of each feature dimension according to different query contexts; further, in actual applications, since the query context will change dynamically, the importance of different feature dimensions will also change accordingly; under this premise, the static weight allocation method cannot adapt to such changes, which in turn leads to a decrease in the fit between the retrieval results and the query context.

[0070] On the other hand, there is a lack of adversarial sample defense mechanism; that is, the existing RAG system lacks an effective adversarial sample defense mechanism; the adversarial samples recorded here are specially designed input samples intended to cause the model to produce incorrect outputs; further, in actual applications, adversarial samples may be maliciously exploited, causing the system to output inaccurate or even harmful information; however, due to the lack of defense mechanism, the existing system is difficult to cope with such potential attacks, which in turn affects the robustness and reliability of the system.

[0071] Based on this, this exemplary embodiment first provides a data retrieval method, which can be run on a server, server cluster or cloud server, etc. Of course, those skilled in the art can also run the method disclosed in this disclosure on other platforms as needed, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the data retrieval method may include the following steps:

[0072] Step S110: Obtain the content to be retrieved, and determine the multimodal features to be retrieved of the content to be retrieved in multiple dimensions;

[0073] Step S120: determining target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions;

[0074] Step S130: Determine a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in the multiple different dimensions;

[0075] Step S140: Determine a target document vector index from the original document vector index based on the comprehensive matching score, and retrieve a content result corresponding to the target document vector index and the content to be retrieved.

[0076] In the data retrieval method described above, on the one hand, by obtaining the content to be retrieved and determining the multimodal features to be retrieved of the content to be retrieved in multiple different dimensions; then determining the target attention weights of the multimodal features to be retrieved and the original document vector index in the preset three-dimensional feature vector container in multiple different dimensions; and then determining the comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in multiple different dimensions; finally, determining the target document vector index from the original document vector index based on the comprehensive matching score; since the comprehensive matching score and the target document vector index can be determined from multiple different dimensions, the problem of semantic vector deviation caused by matching based on a single semantic vector and the low accuracy of the obtained target document vector index can be avoided; on the other hand, since the content retrieval results corresponding to the target document vector index and the content to be retrieved can be obtained, the accuracy of the obtained content retrieval results is improved.

[0077] Hereinafter, the data retrieval method described in the exemplary embodiments of the present disclosure will be explained and illustrated in detail with reference to the accompanying drawings.

[0078] First, the terms involved in the exemplary embodiments of the present disclosure are explained and illustrated.

[0079] RAG (Retrieval-Augmented Generation): RAG is an artificial intelligence technology framework that combines retrieval and generation, mainly used for natural language processing tasks. In the actual retrieval and generation process, the RAG system first retrieves the most relevant information for the current task or query from a large amount of data, and then uses this information for text generation or other tasks, thereby improving the accuracy and relevance of the generated content.

[0080] Multimodal fusion: Multimodal fusion refers to the integration and processing of information from different modalities (such as text, images, audio, and video) to more comprehensively understand and represent data. In the field of artificial intelligence, multimodal fusion allows models to simultaneously utilize multiple different types of input data for data fusion, thereby improving model performance and generalization capabilities.

[0081] Quantum attention: Quantum attention is a variant of the attention mechanism that borrows concepts from quantum computing. In the quantized attention mechanism, the calculation and allocation of attention weights introduce a quantized processing process, allowing the model to focus on different parts of the input data in a more flexible and efficient manner, thereby improving the model's ability to capture important information and processing efficiency.

[0082] Adversarial training: Adversarial training is a training method used to improve the robustness of machine learning models. Specifically, in practical applications, by introducing adversarial samples (i.e., specially designed input samples intended to cause the model to produce errors) during training, the model can maintain good performance even when faced with these difficult samples. Based on this, the generalization ability of the model can be effectively improved, making it more robust in practical applications.

[0083] Spatiotemporal decay: Spatiotemporal decay is a function or mechanism used to process data with temporal and spatial attributes. Specifically, in practical applications, spatiotemporal decay is often used to assign different weights to data in different temporal and spatial dimensions, so that closer or more relevant data points have a greater impact on the model, while the impact of farther or less relevant data points gradually decreases. Based on this, the timeliness and relevance of the data can be better reflected.

[0084] Secondly, the technical implementation principles of the exemplary embodiments of the present disclosure are explained and illustrated. Specifically, the data retrieval method described in the exemplary embodiments of the present disclosure can comprehensively optimize the retrieval performance of the RAG system through core technologies such as a multimodal feature fusion engine, quantized attention weight allocation, adversarial training enhancement module, and spatiotemporal correlation decay function, thereby achieving the purpose of improving the accuracy, context adaptability, robustness, and timeliness of retrieval.

[0085] Furthermore, the data retrieval system involved in the exemplary embodiment of the present disclosure is explained and illustrated. Figure 2 As shown, the data retrieval system may include a device side 210 where the user side is located and a server side 220 where the RAG system is located; wherein the device side and the server side may be communicatively connected via a wired network or a wireless network; the device side may be used to receive content to be retrieved input by the user and send the content to be retrieved to the server side; and, receive content retrieval results fed back by the server side and display the content retrieval results; the server side is used to implement the data retrieval method described in the example embodiment of the present disclosure.

[0086] The following will explain and illustrate the preset semantic embedding model involved in the exemplary embodiments of the present disclosure. Figure 3 As shown, the preset semantic embedding model may include a first input layer 301, multiple Transformer models 302, an average pooling layer 303, and a third output layer 304. The specific functions performed by each model layer in the semantic embedding process will be described in detail later and will not be further elaborated here.

[0087] Hereinafter, the specific generation process of the original document vector index involved in the exemplary embodiment of the present disclosure will be explained and illustrated. Specifically, the original document vector index in the three-dimensional feature vector container recorded herein may include the original semantic vector on the semantic dimension, the original time vector on the time dimension, and the original context vector on the context dimension; wherein, the original document vector index is obtained in the following manner: obtaining the original document, and determining the original semantic vector of the original document on the semantic dimension (the specific determination process of the original semantic vector is similar to the specific determination process of the semantic feature to be retrieved, and no further details are given here), the original time vector on the time dimension, and the original context vector on the context dimension (the specific determination process of the original context vector is similar to the specific determination process of the context feature to be retrieved, and no further details are given here); generating the original document vector index based on the original semantic vector, the original time vector, and the original context vector, and storing the original document vector index in a preset three-dimensional feature vector container. Among them, the original semantic vector recorded here may include a standard semantic vector that has not been enhanced adversarially and a semantic vector after enhancement; specifically, it can be achieved in the following way: embedding and mapping the original document based on a preset semantic embedding model to obtain a standard semantic vector of the original document in the semantic feature dimension; performing parameter perturbation on the standard semantic vector to obtain a semantic vector after enhancement adversarially, and generating an original semantic vector based on the standard semantic vector and the semantic vector after enhancement adversarially. That is, in the actual application process, a large number of original documents can first be loaded from the document database; then, the three-dimensional feature vector container (TriVector) is used to extract and fuse the features of the documents in the three dimensions of semantics, time and context, so as to achieve the purpose of breaking through the limitations of traditional single-vector retrieval and avoiding the problem of semantic drift.

[0088] In an example embodiment, the three-dimensional feature vector container TriVector described above can be a semantic vector (semantic Vec), a temporal vector (temporal Vec) and a contextual vector (contextual Vec). In the construction phase of the original document vector index, the original document is subjected to semantic feature extraction through the embedded semantic model to generate a standard semantic vector to reflect the semantic information of the document; the original time vector is generated using the original timestamp information of the original document to reflect the timeliness of the document; the original context vector is generated based on the context level of the original document to reflect the context level information of the document; wherein, the obtained original document vector index can refer to Figure 4As shown. On this basis, when performing query processing, multimodal feature extraction is also performed on the query content. Through multimodal feature fusion, the retrieval process can comprehensively consider multi-dimensional information such as semantics, time and context, overcoming the limitations of traditional single-vector retrieval. It should also be noted here that in the process of building the original document vector index, it can be implemented in the buildIndex method of the RAGProcessor class; specifically, the buildIndex function in the RAGProcessor class can be called to build the original document vector index, thereby realizing the extraction and integration of the document's multimodal features, laying the foundation for subsequent matching calculations. It should also be noted here that the three-dimensional vector space design can break through the traditional single-vector retrieval paradigm and is an industry first; at the same time, this innovative design provides a new idea and method for the retrieval algorithm of the RAG system, which can more comprehensively and accurately represent and match the features of documents and queries, and provide strong technical support for improving the hit rate of the RAG system.

[0089] In an example embodiment, in the process of adversarial enhancement of the standard semantic vector, adversarial samples can be generated by parameter perturbation and incorporated into the document index, so that the system can be exposed to and adapt to adversarial samples during training and retrieval, effectively improving the robustness and generalization ability of the system. Specifically, in the actual process of parameter perturbation, the AdversarialTrainer class can be added to the RAGProcessor class, and the standard semantic vector of the original document can be adversarially enhanced by the generateAdversarialExample method and the perturb Vector method to obtain the adversarially enhanced semantic vector. Specifically, the standard semantic vector is parameter perturbated to generate adversarial samples, while the time and context vectors remain unchanged; that is, a three-dimensional feature vector can be constructed based on the semantic vector after anti-enhancement and the original time vector and the original context vector; based on this, the robustness and generalization ability of the system can be enhanced, potential adversarial attacks can be effectively resisted, and the stability and accuracy of the system in complex environments can be improved. Moreover, by generating adversarial samples through parameter perturbation, the system can be exposed to and adapt to adversarial samples during training and retrieval, effectively improving the robustness and generalization ability of the system. This enables the system to maintain stability and accuracy when facing complex query environments and potential adversarial attacks, thus solving the defect of existing systems lacking adversarial sample defense mechanisms.

[0090] The following will be combined Figure 2-Figure 4 right Figure 1 The data retrieval method shown in is further explained and illustrated. Specifically:

[0091] In step S110 , the content to be retrieved is obtained, and multimodal features to be retrieved of the content to be retrieved in multiple different dimensions are determined.

[0092] In this example embodiment, first, the content to be retrieved is obtained; specifically, the content recorded here is input into the client by the user manually or by voice, or can be input into the client by uploading a file, and this example does not impose any special restrictions on this; the content to be retrieved recorded here can be text information, or image information, video information, or voice information, and this example does not impose any special restrictions on this; secondly, multimodal features to be retrieved of the content to be retrieved in multiple different dimensions are determined; specifically, this can be achieved in the following manner: embedding and mapping the content to be retrieved based on a preset semantic embedding model to obtain semantic features to be retrieved (semantic Vec) of the content to be retrieved in the semantic feature dimension; extracting the current timestamp of the content to be retrieved, and generating temporal features to be retrieved (temporal Vec) of the content to be retrieved in the time dimension based on the current timestamp of the content to be retrieved; extracting the context level of the content to be retrieved based on a preset context level extraction model, and generating contextual features to be retrieved (contextual Vec) of the content to be retrieved in the context dimension based on the context level of the content to be retrieved. Finally, based on the semantic features to be retrieved, the time features to be retrieved, and the context features to be retrieved, multimodal features to be retrieved are constructed for the content to be retrieved in multiple different dimensions; further, through multimodal feature fusion, the retrieval process can extract and fuse features of documents and queries from the three dimensions of semantics, time, and context, breaking through the limitations of traditional single-vector retrieval, enabling the retrieval process to comprehensively consider multi-dimensional information, effectively avoiding the semantic drift problem caused by vector matching, and significantly improving the accuracy of retrieval.

[0093] In an example embodiment, embedding mapping processing is performed on the content to be retrieved based on a preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension, which can be achieved as follows: word embedding is performed on the content to be retrieved to obtain the word embedding vector, word embedding matrix and position embedding matrix of the content to be retrieved; an embedding vector is generated based on the word embedding vector, word embedding matrix and position embedding matrix, and the embedding vector is input into the first Transformer model to generate a first text semantic vector corresponding to the first Transformer model; the first text semantic vector is input into the second Transformer model to generate a second text semantic vector corresponding to the second Transformer model, and the generation process of the second text semantic vector is repeated in sequence to obtain text semantic vectors corresponding to other Transformer models; a first current encoding vector of the content to be retrieved is generated based on the text semantic vectors and embedding vectors corresponding to each Transformer model, and the first current encoding vector is input into the average pooling layer to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension.

[0094] In an exemplary embodiment, referring to Figure 5 As shown, the preset context level extraction model recorded here includes a second input layer 501, a word embedding layer 502, a position encoding module 503, a feature encoding module 504, a feature decoding module 505, a linear transformation layer 506, a Softmax layer 507 and a second output layer 508; wherein, extracting the context level of the content to be retrieved based on the preset context level extraction model can be achieved in the following manner: word embedding processing is performed on the content to be retrieved based on the first word embedding layer to obtain a first word vector, and encoding processing is performed on the content to be retrieved based on the position encoding module to obtain a first position vector; the first word vector and the first position vector are superimposed to obtain a first input vector, and the first input vector is encoded based on the feature encoding module to obtain a first encoding result; the first encoding result is decoded based on the feature decoding module to obtain a first encoding matrix, and the first encoding matrix is ​​converted based on the linear transformation layer to obtain a first logical matrix; the first logical matrix is ​​mapped based on the Softmax layer to obtain the context level of the content to be retrieved. Furthermore, after the context level is obtained, embedding mapping processing can be performed on the context level based on the Embedding embedding mapping layer to obtain the context features to be retrieved.

[0095] In step S120, target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions are determined.

[0096] Specifically, the specific determination process of the target attention weight can be achieved in the following manner: determining the semantic similarity value between the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector, and determining the first original attention weight on the semantic dimension based on the semantic similarity value; determining the time attenuation coefficient based on the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector, and determining the second original attention weight on the time dimension based on the time attenuation coefficient; determining the context difference coefficient between the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector, and determining the third original attention weight on the context dimension based on the context difference coefficient; normalizing the first original attention weight, the second original attention weight and the third original attention weight to obtain the first target attention weight, the second target attention weight and the third target attention weight. Among them, the specific implementation process of normalization processing can be achieved in the following way: mapping the first original attention weight, the second original attention weight and the third original attention weight to the quantum bit space; performing quantum state amplitude encoding on the first original attention weight, the second original attention weight and the third original attention weight in the quantum bit space to obtain a quantum state; determining the first target attention weight, the second target attention weight and the third target attention weight based on the quantum state.

[0097] The following will further explain and illustrate the specific process of determining the target attention weight. Specifically, in the actual process of determining the attention weights in different dimensions, a quantized attention weight distribution method can be used to achieve this; for example, a quantized attention mechanism can be introduced to dynamically calculate the weights of queries and documents in the three dimensions of semantics, time, and context, and perform quantum normalization processing to achieve dynamic weight adjustment of different feature dimensions, thereby improving the accuracy of retrieval and context adaptability. In the actual application process, a quantized attention mechanism QuantumAttention can be introduced, and the calculateWeights method in the QuantumAttention class can be used to calculate the matching weights of queries and documents in the three dimensions of semantics, time, and context. Among them, the attention weight in the semantic dimension can be determined based on semantic similarity, the attention weight in the time dimension can be determined based on the time decay coefficient, and the attention weight in the context dimension can be determined based on the difference coefficient between context vectors.

[0098] Furthermore, in the actual calculation process, on the one hand, the semantic similarity between the semantic feature to be retrieved and the original semantic vector can be realized by using the cosine similarity algorithm; that is, the cosine value between the semantic feature to be retrieved and the original semantic vector can be calculated to determine the semantic similarity; on the other hand, the time decay coefficient can be calculated by the spatiotemporal correlation decay function (exponential decay function); that is, on the basis of the known time feature to be retrieved and the original time vector, the time decay coefficient can be obtained by calling the exponential decay function; on the other hand, the context difference coefficient can be determined by the length difference value between the context feature to be retrieved and the original context vector; that is, the length difference value between the context feature to be retrieved and the original context vector can be calculated. The length difference between the vectors is used to obtain the contextual difference coefficient; further, in the QuantumAttention class, the calculateWeights method can calculate the semantic similarity (using the cosineSimilarity method), the time decay weight (implemented by the temporalDecay method), and the contextual matching weight (calculated with the contextualMatch method) based on the three-dimensional feature vectors of the query content and the original document; then quantum normalization is used to map the weights of the three dimensions to the quantum space for normalization, so that each attention weight can meet certain specifications to ensure the rationality of the weight distribution.

[0099] Furthermore, it is necessary to further explain here that in the process of determining the attention weight of the time dimension, by introducing the spatiotemporal correlation decay function, the time dimension weight of the document can be dynamically adjusted, so that the time correlation of the document gradually decreases over time, reflecting the dynamic changes in the impact of the document's timeliness characteristics on the retrieval results, so that the retrieval results can better fit the time context of the query, and improve the fit between the retrieval results and the query. In the actual application process, the time characteristics of the content to be retrieved and the original time vector of the original document are used as input, and the time decay coefficient is substituted into the function to obtain the time correlation of the document. The time correlation of the document gradually decreases over time, reflecting the dynamic changes in the impact of the document's timeliness characteristics on the retrieval results; and the introduction of this function enables the retrieval results to better fit the time context of the query, further improving the fit between the retrieval results and the query, and enabling the system to more accurately handle time-related query requirements.

[0100] In step S130, based on the target attention weights on the multiple different dimensions, a comprehensive matching score between the multimodal features to be retrieved and the original document vector index is determined.

[0101] Specifically, the specific determination process of the comprehensive matching score can be achieved as follows: determining the original semantic matching score on the semantic dimension based on the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector in the original document vector index (specifically, the cosine value between the semantic feature to be retrieved and the original semantic vector can be determined, and then the semantic matching score is determined based on the cosine value); determining the original time matching score on the time dimension based on the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector in the original document vector index (specifically, the time difference between the time feature to be retrieved and the original time vector can be determined, and then the time matching score is determined based on the time difference); determining the original context matching score on the context dimension based on the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector in the original document vector index (specifically, the vector length difference between the context feature to be retrieved and the original context vector can be determined, and then the context matching score is determined based on the vector difference difference); performing weighted summation on the original semantic matching score and the first target attention weight, the original time matching score and the second target attention weight, and the original context matching score and the third target attention weight to obtain a comprehensive matching score between the multimodal feature to be retrieved and the original document vector index. Specifically, in practical applications, the hybridScore method can be combined with the quantized attention weight allocation results to calculate the comprehensive matching score between the query and the document. For example, the normalized target attention weight and the original score can be combined to calculate the weighted semantic matching score (implemented by the weightedCosine method), the time-related score, and the context-related score, and then add the three together to obtain the comprehensive matching score. This achieves dynamic attention weight allocation for different feature dimensions, allowing the retrieval process to automatically adjust the importance of each feature based on the query context.

[0102] In step S140, a target document vector index is determined from the original document vector index based on the comprehensive matching score, and a content retrieval result is generated according to the target document vector index and the content to be retrieved.

[0103] In this example embodiment, first, the target document vector index is determined; specifically, the retrieval results can be sorted according to the comprehensive matching score, the top-ranked original document vector index can be screened out as the target document vector index, and the original document corresponding to the target document vector index can be used as the final target document; it should be noted here that the comprehensive matching calculation method can make full use of technical advantages such as multimodal feature fusion, quantized attention weight allocation, and spatiotemporal correlation attenuation to achieve effective sorting of retrieval results and improve the hit rate of the RAG system.

[0104] Secondly, the content retrieval result corresponding to the target document vector index and the content to be retrieved can be obtained by: determining the target document corresponding to the target document vector index and loading the target document; inputting the loaded target document and the content to be retrieved into the preset content generation model to obtain the content retrieval result corresponding to the content to be retrieved. Figure 6 As shown, the preset content generation model described herein may include a third input layer 601, a first embedding mapping layer 602, a first encoding layer 603, a hybrid expert model 604, and a third output layer 605. Under this premise, the loaded target document and the content to be retrieved are input into the preset content generation model to obtain content retrieval results corresponding to the content to be retrieved. This can be achieved in the following manner: generating basic information to be predicted based on the target document and the content to be retrieved, and generating context information to be predicted based on preset parameter prompt information; performing embedding mapping processing on the basic information to be predicted based on the first embedding mapping layer to obtain document features, and performing embedding mapping processing on the context information to be predicted based on the first embedding mapping layer to obtain a context marker sequence; encoding the document features and the context marker sequence based on the first encoding layer to obtain a context overall representation; and performing content generation on the context marker sequence and the context overall representation based on the hybrid expert model to obtain content retrieval results corresponding to the content to be retrieved. Specifically, the preset first parameter prompt information recorded here can be, for example: Your task is to generate a content search result with no more than xx characters and no less than xx characters based on the input text; The generated content search result must include xxxxx content; ... and so on. In actual application, the corresponding parameter prompt information can be set according to actual needs, and this example does not specifically limit this. Finally, after obtaining the content search result, the content search result needs to be returned to the terminal device. After the terminal device receives the content search result, it can display the content search result for the user to view.

[0105] At this point, the data retrieval method described in the exemplary embodiments of the present disclosure has been fully implemented. Based on the aforementioned information, it can be seen that the data retrieval method described in the exemplary embodiments of the present disclosure has at least the following beneficial effects: First, it improves retrieval precision; that is, through multimodal feature fusion and quantized attention weight allocation, it can more accurately match documents and queries, improving retrieval accuracy and relevance. Second, it enhances contextual adaptability; that is, it can dynamically adjust the weights of each feature dimension, enabling the system to adaptively adjust the retrieval strategy based on the query context, improving the fit between retrieval results and queries. Third, it improves system robustness; that is, the adversarial training enhancement module effectively improves the system's robustness and generalization ability, enabling it to maintain stable performance in the face of adversarial examples. Furthermore, it reflects document timeliness; that is, the spatiotemporal correlation decay function can dynamically adjust the time weight of documents, so that retrieval results better reflect the timeliness characteristics of documents. Finally, it breaks through the traditional retrieval paradigm; that is, the three-dimensional vector space design breaks through the limitations of traditional single-vector retrieval, providing a new technical approach for optimizing RAG systems. Under this premise, the following objectives can also be achieved: 1. Improving user experience and satisfaction; that is, in various application fields, more accurate search results mean that users can obtain the information they need more quickly, reducing the problems of ineffective searches and information overload, and significantly improving user satisfaction and loyalty to products or services. 2. Improving work efficiency and productivity; that is, whether it is internal knowledge work, professional research, or daily office work, this technology can help users save a lot of information retrieval time, improve information acquisition efficiency, and thus improve overall work efficiency and productivity. 3. Enhancing enterprise competitiveness; that is, for enterprises providing intelligent question-and-answer, search engines, or knowledge-intensive services, adopting the data retrieval method described in the example embodiments of this disclosure can optimize service quality and performance, enabling them to stand out in the fierce market competition and gain greater market share and commercial value. 4. Promoting industry development and technological advancement; that is, the data retrieval method described in the example embodiments of this disclosure, with its innovative breakthroughs in RAG system algorithms, will promote the development of artificial intelligence and information retrieval technology, bring new technical ideas and solutions to related industries, and promote technological advancement and innovation across the industry. ⑤ Create new business opportunities; that is, the data retrieval method described in the disclosed example embodiments can serve as a core technical component to provide support for various emerging applications that require accurate information retrieval and semantic understanding, giving rise to new business models and business opportunities, such as industry solutions based on accurate knowledge retrieval, personalized content recommendation services, etc.

[0106] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0107] The exemplary embodiment of the present disclosure also provides a data retrieval device. Specifically, refer to Figure 7 As shown, the data retrieval device may include a multimodal feature to be retrieved determination module 710, an attention weight determination module 720, a comprehensive matching score determination module 730, and a content retrieval result generation module 740. Among them:

[0108] The multimodal features to be retrieved determination module 710 can be used to obtain the content to be retrieved and determine the multimodal features to be retrieved of the content to be retrieved in multiple different dimensions; the attention weight determination module 720 can be used to determine the target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions; the comprehensive matching score determination module 730 can be used to determine the comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in the multiple different dimensions; the content retrieval result generation module 740 can be used to determine the target document vector index from the original document vector index based on the comprehensive matching score, and generate the content retrieval result corresponding to the target document vector index and the content to be retrieved.

[0109] In an exemplary embodiment of the present disclosure, multimodal features to be retrieved of content to be retrieved in multiple different dimensions are determined, including: performing embedding mapping processing on the content to be retrieved based on a preset semantic embedding model to obtain semantic features to be retrieved of the content to be retrieved in the semantic feature dimension; extracting the current timestamp of the content to be retrieved, and generating time features to be retrieved of the content to be retrieved in the time dimension based on the current timestamp of the content to be retrieved; extracting the context level of the content to be retrieved based on a preset context level extraction model, and generating context features to be retrieved of the content to be retrieved in the context dimension based on the context level of the content to be retrieved.

[0110] In an exemplary embodiment of the present disclosure, the preset semantic embedding model includes multiple Transformer models and an average pooling layer; wherein, the content to be retrieved is embedded and mapped based on the preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension, including: word embedding of the content to be retrieved to obtain the word embedding vector, word embedding matrix and position embedding matrix of the content to be retrieved; an embedding vector is generated according to the word embedding vector, word embedding matrix and position embedding matrix, and the embedding vector is input into the first Transformer model to generate a first text semantic vector corresponding to the first Transformer model; the first text semantic vector is input into the second Transformer model to generate a second text semantic vector corresponding to the second Transformer model, and the generation process of the second text semantic vector is repeated in sequence to obtain text semantic vectors corresponding to other Transformer models; a first current encoding vector of the content to be retrieved is generated according to the text semantic vectors and embedding vectors corresponding to each Transformer model, and the first current encoding vector is input into the average pooling layer to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension.

[0111] In an exemplary embodiment of the present disclosure, the original document vector index in the preset three-dimensional feature vector container includes an original semantic vector on a semantic dimension, an original time vector on a time dimension, and an original context vector on a context dimension; wherein, determining the target attention weights of the multimodal feature to be retrieved and the original document vector index in the preset three-dimensional feature vector container on multiple different dimensions includes: determining a semantic similarity value between the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector, and determining a first original attention weight on the semantic dimension based on the semantic similarity value; determining a time decay coefficient based on the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector, and determining a second original attention weight on the time dimension based on the time decay coefficient; determining a context difference coefficient between the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector, and determining a third original attention weight on the context dimension based on the context difference coefficient; normalizing the first original attention weight, the second original attention weight, and the third original attention weight to obtain a first target attention weight, a second target attention weight, and a third target attention weight.

[0112] In an exemplary embodiment of the present disclosure, the first original attention weight, the second original attention weight and the third original attention weight are normalized to obtain a first target attention weight, a second target attention weight and a third target attention weight, including: mapping the first original attention weight, the second original attention weight and the third original attention weight to a quantum bit space; performing quantum state amplitude encoding on the first original attention weight, the second original attention weight and the third original attention weight in the quantum bit space to obtain a quantum state; and determining the first target attention weight, the second target attention weight and the third target attention weight based on the quantum state.

[0113] In an exemplary embodiment of the present disclosure, based on the target attention weights on the multiple different dimensions, the comprehensive matching score between the multimodal feature to be retrieved and the original document vector index is determined, including: determining the original semantic matching score on the semantic dimension according to the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector in the original document vector index; determining the original time matching score on the time dimension according to the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector in the original document vector index; determining the original context matching score on the context dimension according to the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector in the original document vector index; and performing weighted summation on the original semantic matching score and the first target attention weight, the original time matching score and the second target attention weight, and the original context matching score and the third target attention weight to obtain the comprehensive matching score between the multimodal feature to be retrieved and the original document vector index.

[0114] In an exemplary embodiment of the present disclosure, the content retrieval result corresponding to the target document vector index and the content to be retrieved includes: determining the target document corresponding to the target document vector index and loading the target document; inputting the loaded target document and the content to be retrieved into a preset content generation model to obtain a content retrieval result corresponding to the content to be retrieved.

[0115] In an exemplary embodiment of the present disclosure, the original document vector index is obtained in the following manner: obtaining the original document and determining the original semantic vector in the semantic dimension, the original time vector in the time dimension, and the original context vector in the context dimension of the original document; generating the original document vector index based on the original semantic vector, the original time vector, and the original context vector, and storing the original document vector index in a preset three-dimensional feature vector container.

[0116] In an exemplary embodiment of the present disclosure, the original semantic vector of the original document in the semantic dimension is determined, including: performing embedding mapping processing on the original document based on a preset semantic embedding model to obtain a standard semantic vector of the original document in the semantic feature dimension; performing parameter perturbation on the standard semantic vector to obtain an adversarially enhanced semantic vector, and generating an original semantic vector based on the standard semantic vector and the adversarially enhanced semantic vector.

[0117] The specific details of each module in the above data retrieval device have been described in detail in the corresponding data retrieval method, so they will not be repeated here.

[0118] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0119] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0120] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Those skilled in the art will appreciate that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to herein as a circuit, module, or system.

[0121] Refer to the following Figure 8 800 according to this embodiment of the present disclosure will be described. Figure 8 The electronic device 800 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 8As shown, electronic device 800 is implemented as a general-purpose computing device. Components of electronic device 800 may include, but are not limited to, the aforementioned at least one processing unit 810, the aforementioned at least one storage unit 820, a bus 830 connecting various system components (including storage unit 820 and processing unit 810), and a display unit 840.

[0123] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps described in the "Exemplary Method" section of the present disclosure according to various exemplary embodiments. For example, the processing unit 810 can perform the following steps: Figure 1 Step S110 shown in: obtaining the content to be retrieved, and determining the multimodal features to be retrieved of the content to be retrieved in multiple different dimensions; step S120: determining the target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions; step S130: determining the comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in the multiple different dimensions; step S140: determining the target document vector index from the original document vector index based on the comprehensive matching score, and performing content retrieval results according to the target document vector index and the content to be retrieved.

[0124] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0125] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0126] Bus 830 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0127] The electronic device 800 can also communicate with one or more external devices 900 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 800, and / or any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 850. Furthermore, the electronic device 800 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0129] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present disclosure.

[0130] A program product for implementing the above-described method according to an embodiment of the present disclosure is described. The program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0131] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0132] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0133] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0134] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0135] Furthermore, the figures above are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0136] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not invented herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A data retrieval method, characterized in that: include: Acquire content to be retrieved, and determine multimodal features to be retrieved of the content to be retrieved in multiple different dimensions; Determining target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions; Determining a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights on the multiple different dimensions; A target document vector index is determined from the original document vector index based on the comprehensive matching score, and a content retrieval result is generated according to the target document vector index and the content to be retrieved.

2. The data retrieval method according to claim 1, wherein: Determine the multimodal features of the content to be retrieved in multiple dimensions, including: Performing embedding mapping processing on the content to be retrieved based on a preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension; Extracting the current timestamp of the content to be retrieved, and generating a time feature of the content to be retrieved in a time dimension based on the current timestamp of the content to be retrieved; The context level of the content to be retrieved is extracted based on a preset context level extraction model, and based on the context level of the content to be retrieved, context features to be retrieved of the content to be retrieved in a context dimension are generated.

3. The data retrieval method according to claim 2, characterized in that: The preset semantic embedding model includes multiple Transformer models and average pooling layers; The embedding mapping process is performed on the content to be retrieved based on a preset semantic embedding model to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension, including: Performing word embedding on the content to be retrieved to obtain a word embedding vector, a word embedding matrix, and a position embedding matrix of the content to be retrieved; Generate an embedding vector according to the word embedding vector, the word embedding matrix, and the position embedding matrix, and input the embedding vector into a first Transformer model to generate a first text semantic vector corresponding to the first Transformer model; Inputting the first text semantic vector into a second Transformer model to generate a second text semantic vector corresponding to the second Transformer model, and repeating the generation process of the second text semantic vector in sequence to obtain text semantic vectors corresponding to other Transformer models; A first current encoding vector of the content to be retrieved is generated based on the text semantic vector and embedding vector corresponding to each Transformer model, and the first current encoding vector is input into the average pooling layer to obtain the semantic features to be retrieved of the content to be retrieved in the semantic feature dimension.

4. The data retrieval method according to claim 1, wherein: The original document vector index in the preset three-dimensional feature vector container includes an original semantic vector in the semantic dimension, an original time vector in the time dimension, and an original context vector in the context dimension; Determining target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions includes: Determining a semantic similarity value between the semantic feature to be retrieved in the multimodal feature to be retrieved and the original semantic vector, and determining a first original attention weight on the semantic dimension according to the semantic similarity value; Determining a time decay coefficient based on the time feature to be retrieved in the multimodal feature to be retrieved and the original time vector, and determining a second original attention weight in the time dimension based on the time decay coefficient; Determining a context difference coefficient between a context feature to be retrieved in the multimodal feature to be retrieved and the original context vector, and determining a third original attention weight on the context dimension according to the context difference coefficient; The first original attention weight, the second original attention weight and the third original attention weight are normalized to obtain a first target attention weight, a second target attention weight and a third target attention weight.

5. The data retrieval method according to claim 4, characterized in that: Normalizing the first original attention weight, the second original attention weight, and the third original attention weight to obtain a first target attention weight, a second target attention weight, and a third target attention weight, including: Mapping the first original attention weight, the second original attention weight, and the third original attention weight to a quantum bit space; Performing quantum state amplitude encoding on the first original attention weight, the second original attention weight, and the third original attention weight in the quantum bit space to obtain a quantum state; A first target attention weight, a second target attention weight, and a third target attention weight are determined based on the quantum state.

6. The data retrieval method according to claim 1, wherein: Determining a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights on the multiple different dimensions includes: Determining an original semantic matching score on a semantic dimension based on the semantic features to be retrieved in the multimodal features to be retrieved and the original semantic vector in the original document vector index; Determining an original time matching score in a time dimension according to a time feature to be retrieved in the multimodal feature to be retrieved and an original time vector in an original document vector index; Determining an original context matching score on a context dimension based on the context feature to be retrieved in the multimodal feature to be retrieved and the original context vector in the original document vector index; The original semantic matching score and the first target attention weight, the original time matching score and the second target attention weight, the original context matching score and the third target attention weight are weightedly summed to obtain a comprehensive matching score between the multimodal feature to be retrieved and the original document vector index.

7. The data retrieval method according to claim 1, characterized in that: The content retrieval result corresponding to the target document vector index and the content to be retrieved includes: Determining a target document corresponding to the target document vector index, and loading the target document; The loaded target document and the content to be retrieved are input into a preset content generation model to obtain a content retrieval result corresponding to the content to be retrieved.

8. The data retrieval method according to claim 1, wherein: The original document vector index is obtained in the following way: Acquire an original document, and determine an original semantic vector in a semantic dimension, an original time vector in a time dimension, and an original context vector in a context dimension of the original document; The original document vector index is generated according to the original semantic vector, the original time vector, and the original context vector, and the original document vector index is stored in a preset three-dimensional feature vector container.

9. The data retrieval method according to claim 8, characterized in that: Determining an original semantic vector of the original document in a semantic dimension includes: Performing embedding mapping processing on the original document based on a preset semantic embedding model to obtain a standard semantic vector of the original document in a semantic feature dimension; Parameter perturbation is performed on the standard semantic vector to obtain an adversarially enhanced semantic vector, and an original semantic vector is generated based on the standard semantic vector and the adversarially enhanced semantic vector.

10. A data retrieval device, characterized in that: include: A multimodal to-be-retrieved feature determination module, configured to obtain content to be retrieved and determine multimodal to-be-retrieved features of the content to be retrieved in multiple dimensions; An attention weight determination module is used to determine target attention weights of the multimodal features to be retrieved and the original document vector index in a preset three-dimensional feature vector container in multiple different dimensions; a comprehensive matching score determination module, configured to determine a comprehensive matching score between the multimodal features to be retrieved and the original document vector index based on the target attention weights in the multiple different dimensions; The content retrieval result generating module is used to determine a target document vector index from the original document vector index based on the comprehensive matching score, and generate a content retrieval result corresponding to the target document vector index and the content to be retrieved.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data retrieval method according to any one of claims 1 to 9 is implemented.

12. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the data retrieval method according to any one of claims 1 to 9 by executing the executable instructions.

Images