Data retrieval method, apparatus, processor, and electronic device

Abstract

The application discloses a data retrieval method and device, a processor and electronic equipment. It is related to the technical field of computers, and the method comprises the following steps: in response to a retrieval request for target retrieval data, performing the following retrieval steps: extracting initial data features of the target retrieval data; mapping the initial data features to obtain target data features; determining a target retrieval hash value according to the target data features; determining a similarity index of the target retrieval hash value and a plurality of data hash values in a predetermined database; and determining a target retrieval result corresponding to the target retrieval data according to the similarity index. The application solves the technical problem of low retrieval accuracy when similar data to the target data is retrieved in the related art.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data retrieval method, device, processor, and electronic device. Background Art

[0002] Currently, mainstream cross-modal hash retrieval methods primarily use projection matrices or deep learning network parameters to map multimodal data with different feature dimensions into a common space of the same dimension. The similarity between the multimodal data is measured by calculating the Hamming distance between the features, which then recommends ranked search results to the user. However, when using the methods provided in related technologies to retrieve data similar to the target data, the retrieval accuracy is low.

[0003] Currently, no effective solution has been proposed to the above-mentioned problems. Summary of the Invention

[0004] Embodiments of the present invention provide a data retrieval method, apparatus, processor, and electronic device to at least solve the technical problem of low retrieval accuracy when searching for data similar to target data in related technologies.

[0005] According to one aspect of an embodiment of the present invention, a data retrieval method is provided, comprising: responding to a retrieval request for target retrieval data, executing the following retrieval steps, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data: extracting initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; determining a target retrieval hash value based on the target data features; determining a similarity index between the target retrieval hash value and a plurality of data hash values ​​in a predetermined database; and determining a target retrieval result corresponding to the target retrieval data based on the similarity index.

[0006] Optionally, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, including: in response to the retrieval request, the target retrieval data is input into a target retrieval model, and the target retrieval model performs the retrieval steps, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and a sample retrieval result corresponding to the sample retrieval data.

[0007] Optionally, the method also includes: training the sample data in the following manner to obtain the target retrieval model: when the sample retrieval data includes image sample retrieval data and text sample retrieval data, obtaining an initial model, and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

[0008] Optionally, determining the joint data feature includes: determining a first feature item between the image sample retrieval data and the image sample retrieval data, determining a second feature item between the text sample retrieval data and the text sample retrieval data, determining a third feature item between the image sample retrieval data and the text sample retrieval data, and determining a fourth feature item between the text sample retrieval data and the image sample retrieval data; constructing a combined feature based on the first feature item, the second feature item, the third feature item and the fourth feature item; substituting the combined feature into a predetermined linear activation function to obtain the joint data feature.

[0009] Optionally, the determining of the first feature item between the image sample retrieval data and the image sample retrieval data, the determining of the second feature item between the text sample retrieval data and the text sample retrieval data, the determining of the third feature item between the image sample retrieval data and the text sample retrieval data, and the determining of the fourth feature item between the text sample retrieval data and the image sample retrieval data include: determining the first query vector corresponding to the image sample retrieval data, the product of the transpose vector of the second query vector corresponding to the image sample retrieval data, and the ratio of the target number of dimensions to obtain a first ratio, and determining the first query vector corresponding to the text sample retrieval data, the product of the transpose vector of the second query vector corresponding to the text sample retrieval data, and the ratio of the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data, the product of the transpose vector of the second query vector corresponding to the text sample retrieval data, and the ratio of the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data, the product of the transpose vector of the second query vector corresponding to the text sample retrieval data The product of the transpose vector of the query vector and the ratio of the target dimension number is obtained to obtain a third ratio, and the product of the first query vector corresponding to the text sample retrieval data and the transpose vector of the second query vector corresponding to the image sample retrieval data and the ratio of the target dimension number is determined to obtain a fourth ratio; the first ratio, the second ratio, the third ratio and the fourth ratio are respectively input into a predetermined probability distribution conversion function to obtain the corresponding first result, second result, third result and fourth result respectively; it is determined that the product of the first result and the third query vector corresponding to the image sample retrieval data is the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is the fourth feature item.

[0010] Optionally, the method further includes: obtaining the sample data in the following manner: determining search results corresponding to the data in the predetermined database; and using the data in the predetermined database and the search results corresponding to the data in the predetermined database as the sample data.

[0011] Optionally, determining the target retrieval hash value based on the target data feature includes: determining the size relationship between each feature value in the target data feature and a predetermined threshold to obtain a predetermined size relationship; updating the feature value whose corresponding feature value is greater than the predetermined threshold in the predetermined size relationship to a first predetermined value, and updating the feature value whose corresponding feature value is less than or equal to the predetermined threshold to a second predetermined value to obtain the target retrieval hash value, wherein the first predetermined value is higher than the second predetermined value.

[0012] According to one aspect of an embodiment of the present invention, a data retrieval device is provided, comprising: an execution module for responding to a retrieval request for target retrieval data and executing the following retrieval steps, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data; an extraction module for extracting initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; a mapping module for mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; a first determination module for determining a target retrieval hash value based on the target data features; a second determination module for determining a similarity index between the target retrieval hash value and a plurality of data hash values ​​in a predetermined database; and a third determination module for determining a target retrieval result corresponding to the target retrieval data based on the similarity index.

[0013] According to one aspect of an embodiment of the present invention, a processor is provided, wherein the processor is configured to run a program, wherein the program executes any one of the above methods when running.

[0014] According to one aspect of an embodiment of the present invention, an electronic device is provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the methods described above.

[0015] In an embodiment of the present invention, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data: extracting the initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions. Mapping the initial data features to obtain the target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions. Determining the target retrieval hash value based on the target data features; determining the similarity index of the target retrieval hash value and the multiple data hash values ​​in the predetermined database, so as to achieve the purpose of determining the target retrieval result corresponding to the target retrieval data based on the similarity index. Since the target data features are obtained by mapping the initial data features, the consistency of the target data feature representation and the consistency of the target number of dimensions are achieved, which can improve the accuracy of the target data feature matching and the target retrieval data matching to a certain extent. This solves the technical problem of low retrieval accuracy in the related art when retrieving data similar to the target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0017] Figure 1 is a flow chart of a data retrieval method provided according to an embodiment of the present application;

[0018] Figure 2 is a structural block diagram of a data retrieval device provided according to an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0021] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0022] The present invention will be described below in conjunction with preferred implementation steps. Figure 1 is a flow chart of a data retrieval method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0023] Step S101: In response to a search request for target search data, the following search steps are performed, wherein the search request is used to request to retrieve data similar to the target search data:

[0024] In this embodiment, the above-mentioned target retrieval data can be image data or text data, that is, the user's retrieval request can be image data, text data or other forms of feature data, which is not limited here and can be customized according to specific scenarios and applications.

[0025] It should be noted that responding to the retrieval request for the above-mentioned target retrieval data and performing the following related retrieval steps include: sorting the target retrieval data according to similarity, obtaining the data most similar to the retrieval request data, that is, the above-mentioned retrieval of data similar to the target retrieval data.

[0026] Step S102, extracting the initial data features of the target search data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions;

[0027] In this embodiment, the initial data features extracted above can be different according to the type of target retrieval data. For example, when the target retrieval data is image data, a feature extraction method can be used to extract some initial data features of the image data, such as color features, texture features, shape features, and deep learning features. It can also be a feature representation after extraction by a neural network. Different types of images have different features, and based on these data features, the initial number of dimensions corresponding to the initial data features of the image data is determined. When the target retrieval data is text data, a bag-of-words model, text statistical features, and other methods can be used to represent the text in a vector form and extract some initial data features of the text data, such as word-level features, syntactic features, document-level features, and other features. It can also be a feature representation after extraction by a bag-of-words model. Different types of text have different features, and based on these data features, the initial number of dimensions corresponding to the initial data features of the text data is determined. The above data features can be determined according to different data features without limitation.

[0028] It should be noted that extracting the initial data features and determining the corresponding target dimension number can effectively improve the accuracy of target retrieval data. The appropriate number of dimensions can retain the characteristic information of the target retrieval data and can more accurately retrieve and match data similar to the target retrieval data.

[0029] Step S103, mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions;

[0030] In this embodiment, based on the initial data features extracted in the above steps, according to the data type and application requirements of the target retrieval data, a suitable mapping method is selected. For example, a convolutional neural network can be used for feature mapping in image data, while a word embedding model can be used for feature mapping in text data. That is, the initial data features are mapped to the target data feature space. The mapping process converts the original features into new feature representations, which usually changes the dimension of the features. In order to ensure that the number of dimensions of the target data features meets the requirements of the target number of dimensions, the dimensions can be expanded or compressed so that the dimensions of the target data features are consistent with the expected number of target dimensions. In actual applications, the selection of a suitable mapping method and target number of dimensions needs to be selected and adjusted according to the characteristics of the data and the task requirements.

[0031] It should be noted that by mapping the initial data features to obtain the target data features and determining the number of dimensions of the target data features, not only can the consistency of the target data feature representation and the consistency of the target dimension number be improved, but also the accuracy of target data feature matching and target retrieval data matching can be improved.

[0032] Step S104, determining a target search hash value based on target data characteristics;

[0033] In this embodiment, the target retrieval hash value refers to a hash value representing the data to be retrieved. The data to be retrieved is converted into a hash value for comparison with hash values ​​in a predetermined database. The target retrieval hash value is typically calculated using a hash function to map the data into a fixed-length binary code. Specifically, the hash function can map target data features to a fixed-size hash code space and has a certain degree of collision avoidance, meaning that different target data features are mapped to different hash codes as much as possible. Specifically, the target data features are input into the selected hash function, and the corresponding target retrieval hash value is obtained.

[0034] It's important to note that selecting an appropriate hash function is crucial to the quality and performance of the target retrieval hash value. Different hash functions may be suitable for different data types and application scenarios, so they need to be selected and evaluated based on the specific situation. Using a given target search hash value can, to a certain extent, improve the storage, processing, and retrieval efficiency of the target retrieval data. At the same time, it can quickly retrieve target data with similar data characteristics, achieving efficient similarity retrieval of the target retrieval data.

[0035] Step S105, determining similarity indexes between the target search hash value and multiple data hash values ​​in a predetermined database;

[0036] In this embodiment, the similarity index of the above-mentioned multiple data hash values ​​may refer to the degree of similarity between the target retrieval hash value and each data hash value in the database. The similarity index may use different measurement methods to represent the degree of similarity between the target retrieval hash value and the data hash value in the database. The method for determining the similarity index is not limited here and may be customized according to specific application scenarios and requirements.

[0037] According to the above steps, assuming that the target retrieval data is image data, a database will be established in advance, which contains the hash values ​​of multiple images. For each image, it is converted into a hash value through a hash function and stored in the database; based on the data features of the extracted image data, the target retrieval hash value is determined, and then the target retrieval hash value is compared with the hash value of each image in the database, and the similarity index is calculated. By comparing the similarity index, images similar to the target image can be found and returned to the user for retrieval.

[0038] It should be noted that the similarity index determined above can evaluate the similarity and matching degree between the target retrieval hash value corresponding to the target retrieval data and multiple hash values ​​in the predetermined database, which helps to provide more accurate and targeted data retrieval results, and improve user experience and data analysis accuracy.

[0039] Step S106: determining the target search result corresponding to the target search data according to the similarity index.

[0040] In this embodiment, according to the similarity index determined in the above steps, that is, the hash value corresponding to the target retrieval data is compared with each hash value in the predetermined database to obtain the similarity index, and then the target retrieval results are sorted according to this similarity index. According to the sorting results, the data with the highest similarity to the target retrieval data can be determined as the target retrieval result.

[0041] It should be noted that determining the target search result corresponding to the target search data based on the similarity index can not only provide an accurate matching result with the target search data, but also improve the retrieval efficiency of the target search data.

[0042] Through the above steps S101-S106, in response to the retrieval request for the target retrieval data, the following retrieval steps are performed, wherein the retrieval request is used to request the retrieval of data similar to the target retrieval data: extract the initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions. Map the initial data features to obtain the target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; determine the target retrieval hash value based on the target data features. Determine the similarity index between the target retrieval hash value and the multiple data hash values ​​in the predetermined database, so as to achieve the purpose of determining the target retrieval result corresponding to the target retrieval data based on the similarity index. Since the target data features are obtained by mapping the initial data features, the consistency of the target data feature representation and the consistency of the target number of dimensions are achieved, which can improve the accuracy of the target data feature matching and the target retrieval data matching to a certain extent. This solves the technical problem of low retrieval accuracy in the related art when retrieving data similar to the target data.

[0043] As an optional embodiment, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, including: in response to the retrieval request, the target retrieval data is input into a target retrieval model, and the target retrieval model performs the retrieval steps, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and a sample retrieval result corresponding to the sample retrieval data.

[0044] In this embodiment, the above-mentioned target retrieval model is obtained by training the initial model through sample data. It has a certain learning ability and can retrieve corresponding retrieval results based on the input target retrieval data. Its sample data includes: sample retrieval data and sample retrieval results. The sample retrieval data is the data used to train the target retrieval model, and the sample retrieval result can be the expected output result corresponding to the sample data.

[0045] In response to a search request for target retrieval data, the target retrieval data, which may be user-provided data to be searched, is input into the target retrieval model. The target retrieval model then performs retrieval steps based on the input target retrieval data and generates retrieval results. The target retrieval results generated by the target retrieval model are then compared with the sample retrieval results to evaluate the model's performance on the retrieval task. The results are then verified and returned to the user.

[0046] It should be noted that the performance and accuracy of the target retrieval model are closely related to the sample data. Therefore, when training the initial model, it should be ensured that the sample data has sufficient diversity and richness to cover the expected retrieval scenarios and needs well.

[0047] As an optional embodiment, when the sample retrieval data includes image sample retrieval data and text sample retrieval data, an initial model is obtained, and a loss function for model training is constructed, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

[0048] In this embodiment, the above-mentioned first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature. A measurement indicator (such as cosine similarity) can be used to calculate the similarity between them and use it as part of the loss function.

[0049] The above-mentioned second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature. Similarly, a measurement indicator can be used to calculate the similarity between them and use it as part of the loss function.

[0050] The third loss function term is used to maintain the similarity between the joint data features and the true similarity matrix. The true similarity matrix reflects the true similarity relationship between the sample data. A metric (such as cross entropy loss) can be used to compare the joint data features with the true similarity matrix and use it as part of the loss function.

[0051] It should be noted that an initial model is first obtained. The initial model can be a previously trained model or a model built from scratch. Then, a loss function is constructed for model training based on the image sample retrieval data and text sample retrieval data in the sample retrieval data. The loss function consists of three parts: the first loss function term, the second loss function term, and the third loss function term. Based on the constructed loss function, the initial model is trained using the sample data. By optimizing the loss function, the model weights and parameters are adjusted so that the target retrieval model can better capture the similarities between the sample data features. After the model training is completed, the target retrieval model is evaluated. A test dataset can be used to evaluate the model's performance on the retrieval task, and the model can be adjusted and optimized based on the evaluation results. Through this training process, a target retrieval model for image sample retrieval data and text sample retrieval data can be obtained. This model can capture the similarity relationship between sample data by combining data features.

[0052] As an optional embodiment, determining the joint data feature includes: determining a first feature item between image sample retrieval data and image sample retrieval data, determining a second feature item between text sample retrieval data and text sample retrieval data, determining a third feature item between image sample retrieval data and text sample retrieval data, and determining a fourth feature item between text sample retrieval data and image sample retrieval data; constructing a combined feature based on the first feature item, the second feature item, the third feature item and the fourth feature item; substituting the combined feature into a predetermined linear activation function to obtain the joint data feature.

[0053] In this embodiment, the first feature item mentioned above can be an image-based feature representation method, such as an image feature vector extracted using a convolutional neural network (CNN); the second feature item mentioned above can be a text-based feature representation method, such as using natural language processing technology to convert text into a vector representation; the third feature item mentioned above can be a feature obtained by combining image and text data and using a cross-modal expression method, such as using a shared space encoding method for images and texts; the fourth feature item mentioned above can be a feature obtained by combining image and text data, and a method similar to the third feature item can be used. The method for determining the first feature item, the second feature item, the third feature item, and the fourth feature item in this step is not limited here and can be customized according to specific scenarios and applications.

[0054] According to the first, second, third and fourth feature items mentioned above, they are combined together to construct a comprehensive joint data feature representation, which can be achieved through simple splicing, weighted summation or other combination methods. The constructed combined features are then substituted into a predetermined linear activation function for processing. The linear activation function can be a linear transformation or other transformation. The combined features are further mapped in the feature space. Finally, the final joint data feature representation is obtained by applying a predetermined linear activation function to the combined features. This feature representation fuses the information of the image sample retrieval data and the text sample retrieval data.

[0055] It should be noted that through the above steps, a joint data feature describing the relationship between image sample retrieval data and text sample retrieval data can be obtained. This joint data feature can play an important role in the subsequent training and retrieval process.

[0056] As an optional embodiment, determining the first feature item between image sample retrieval data and image sample retrieval data, determining the second feature item between text sample retrieval data and text sample retrieval data, determining the third feature item between image sample retrieval data and text sample retrieval data, and determining the fourth feature item between text sample retrieval data and image sample retrieval data includes: determining the first query vector corresponding to the image sample retrieval data, multiplying the product of the transpose vector of the second query vector corresponding to the image sample retrieval data and the ratio of the target dimension number to obtain a first ratio, and determining the first query vector corresponding to the text sample retrieval data, multiplying the product of the transpose vector of the second query vector corresponding to the text sample retrieval data and the ratio of the target dimension number to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data, multiplying the product of the transpose vector of the second query vector corresponding to the text sample retrieval data and the ratio of the target dimension number to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data, multiplying the product of the transpose vector of the second query vector corresponding to the text sample retrieval data and the ratio of the target dimension number to obtain a second ratio, and determining the first query vector corresponding to the text ... The product of the transpose vectors of the two query vectors and the ratio of the target number of dimensions obtain a third ratio, and the product of the first query vector corresponding to the text sample retrieval data and the transpose vector of the second query vector corresponding to the image sample retrieval data and the ratio of the target number of dimensions obtains a fourth ratio; the first ratio, the second ratio, the third ratio and the fourth ratio are respectively input into a predetermined probability distribution conversion function to obtain the corresponding first result, second result, third result and fourth result respectively; the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is determined as the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is determined as the fourth feature item.

[0057] In this embodiment, in the process of determining the first feature item, the second feature item, the third feature item and the fourth feature item in the above steps, it is also necessary to determine the first ratio, the second ratio, the third ratio and the fourth ratio. The above first ratio is obtained by calculating the product of the first query vector corresponding to the image sample retrieval data and the transposed vector of the second query vector corresponding to the image sample retrieval data, and performing proportional calculation with the target number of dimensions; the above second ratio is obtained by calculating the product of the first query vector corresponding to the text sample retrieval data and the transposed vector of the second query vector corresponding to the text sample retrieval data, and performing proportional calculation with the target number of dimensions; the above third ratio is obtained by calculating the product of the first query vector corresponding to the image sample retrieval data and the transposed vector of the second query vector corresponding to the text sample retrieval data, and performing proportional calculation with the target number of dimensions. The above fourth ratio is obtained by calculating the product of the first query vector corresponding to the text sample retrieval data and the transposed vector of the second query vector corresponding to the image sample retrieval data, and performing proportional calculation with the target number of dimensions.

[0058] The first ratio, second ratio, third ratio, and fourth ratio determined by the above steps are respectively input into a predetermined probability distribution conversion function for processing to obtain the corresponding first result, second result, third result, and fourth result, respectively. Then, the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is determined as the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is determined as the fourth feature item. The feature items determined by the above steps can better reflect their corresponding features, making the subsequently determined retrieval data more accurate.

[0059] As an optional embodiment, the method further includes: obtaining sample data in the following manner: determining search results corresponding to the data in the predetermined database; and using the data in the predetermined database and the search results corresponding to the data in the predetermined database as sample data.

[0060] In this embodiment, in response to a retrieval request for target retrieval data, a retrieval operation is performed using a predetermined target retrieval model and data in a database to obtain retrieval results corresponding to each data. The target retrieval model is obtained by training an initial model using sample data. The sample data can be data in a predetermined database, as well as retrieval results corresponding to the data in the predetermined database. The sample data can be data in a predetermined database, which can include image sample retrieval data and text sample retrieval data. A suitable data type is then selected based on specific applications and requirements. The sample data can also be retrieval results corresponding to the data in the database as sample retrieval results. These results can be predetermined correlation or similarity scores, or information of specific categories or labels, determined based on the requirements of the specific task. Finally, the constructed sample data is used to train the initial model to obtain the target retrieval model.

[0061] It should be noted that, through the above method, sample data containing retrieval results corresponding to the data in the predetermined database can be obtained. During the model training process, such sample data can help the initial model learn the correlation between the data and the retrieval results, and can further judge the performance and effect of the initial model, and provide feedback and reference for the subsequent optimization of the target retrieval model, so that the target retrieval model can better understand and process the retrieval task.

[0062] As an optional embodiment, determining a target retrieval hash value based on target data characteristics includes: determining the size relationship between each characteristic value in the target data characteristics and a predetermined threshold to obtain a predetermined size relationship; updating the characteristic value whose corresponding characteristic value is greater than the predetermined threshold in the predetermined size relationship to a first predetermined value, and updating the characteristic value whose corresponding characteristic value is less than or equal to the predetermined threshold to a second predetermined value to obtain a target retrieval hash value, wherein the first predetermined value is higher than the second predetermined value.

[0063] In this embodiment, the size relationship between each feature value and a predetermined threshold is determined. That is, for each feature value in the target data feature, its size relationship with the predetermined threshold is determined by comparing the size of the feature value with the threshold. Based on the size relationship between each feature value and the predetermined threshold, a predetermined size relationship is obtained. For example, a feature value greater than the threshold can be considered "greater than", and a feature value less than or equal to the threshold can be considered "less than or equal to". Based on the predetermined size relationship, the feature value corresponding to the target data feature is updated. For example, if a feature value is greater than the predetermined threshold, the feature value can be updated to a first predetermined value. If a feature value is less than or equal to the predetermined threshold, the feature value can be updated to a second predetermined value. By combining the updated first and second predetermined values ​​together and performing a hash function to obtain a target retrieval hash value, ensuring that the first predetermined value is higher than the second predetermined value, different levels of hash codes can be established for the target data to reflect the importance of different features.

[0064] It should be noted that the target retrieval hash value determination process can be adjusted and optimized according to specific needs and tasks. Through such steps, the target data features can be converted into target retrieval hash values ​​for fast and efficient matching and screening in retrieval tasks.

[0065] Based on the embodiment and optional embodiments, an optional implementation is provided, which is described in detail below.

[0066] In the related art, when searching for data similar to the target data, there is a technical problem of low retrieval accuracy. For example, in the related art, when using a related method to search for data similar to the target data, there are the following problems: ignoring the modality of the target data, which may cause the following problems:

[0067] 1) The extracted data features are incomplete and inaccurate;

[0068] 2) The similarity and correlation between the data features of target data of different modalities cannot be determined; the two problems mentioned above that exist in ignoring the modality of target data will lead to low retrieval accuracy.

[0069] In view of this, a data retrieval method is provided in an optional implementation manner of the present invention, and the optional implementation manner of the present application is described in detail below.

[0070] S1. Responding to a search request for image retrieval data and text retrieval data, and executing the following search steps, wherein the search request is used to retrieve data similar to the image retrieval data and the text retrieval data, respectively;

[0071] It should be noted that, in response to a retrieval request, the image retrieval data and the text retrieval data are input into the target retrieval model, and the target retrieval model performs a retrieval step, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and sample retrieval results corresponding to the sample retrieval data.

[0072] S2. Extracting initial data features corresponding to the image retrieval data and the text retrieval data, respectively, and mapping the initial data features to obtain target data features corresponding to the image retrieval data and the text retrieval data, respectively, and target dimensions corresponding to the target data features;

[0073] It should be noted that the initial data features corresponding to the image retrieval data and text retrieval data are different. The following is an explanation of each case:

[0074] 1) Extraction of initial data features corresponding to image retrieval data: Convolutional neural network (CNN) can be used to extract image features. CNN is currently the most representative image feature extraction network in the field of deep learning. The working mechanism of CNN is to directly input the original image into the network. After operations such as convolution, pooling, full connection, and normalized exponential function Softmax processing, the extracted image features are the initial data features. CNN has relatively few network parameters, which greatly improves the training speed of the model.

[0075] 2) Extraction of initial data features corresponding to text retrieval data: The bag-of-words model (BOW) can be used to extract text features. The bag-of-words model (BOW) is the most commonly used text representation in natural language processing. Given two text data, all the words that have appeared are counted to construct a data dictionary. The initial text data is transformed through the BOW model to generate two one-dimensional vectors, where each value of the vector represents the number of times a word appears in the text retrieval data.

[0076] After extracting the initial data features, it is also necessary to determine the number of dimensions corresponding to the initial data features, that is, the initial number of dimensions, which can be determined in the following way: the features extracted by the CNN and BOW models in the above example, that is, the initial data features corresponding to the image retrieval data are expressed as U, and the initial data features corresponding to the text retrieval data are expressed as V, that is, the initial number of dimensions corresponding to the initial data features corresponding to the image retrieval data can be 512, and the initial number of dimensions corresponding to the initial data features corresponding to the text retrieval data can be 20, which can be expressed as: the image dimension is d U =512, the number of text dimensions is d V =20.

[0077] Furthermore, the matrix ω corresponding to the image features isx , the matrix corresponding to the text features ω y Mapped to a public space with a dimension of 64, the target dimension corresponding to the target data feature is obtained, that is, the target dimension d hash =64.

[0078] S3. Determine the magnitude relationship between each characteristic value in the target data characteristic and a predetermined threshold, and determine a target retrieval hash value based on the target data characteristic;

[0079] It should be noted that the determined size relationship in the above steps is used as the predetermined size relationship, and then the target retrieval hash value is determined in combination with the target data characteristics. The determination process is as follows: Function, converting real number features into target retrieval hash values, namely including: image hash code X and text hash code Y, the symbolic function sgn(x) in this step can update the eigenvalue whose corresponding eigenvalue is greater than the predetermined threshold in the predetermined size relationship to a first predetermined value, and update the eigenvalue whose corresponding eigenvalue is less than or equal to the predetermined threshold to a second predetermined value, to obtain the target retrieval hash value, the first predetermined value is higher than the second predetermined value, the first predetermined value can be 1, and the second predetermined value can be 0. Furthermore, the determined target hash value can be input into the attention mechanism neural network to extract deeper data features.

[0080] S4. Determine similarity indexes between the target search hash value and multiple data hash values ​​in the predetermined database, and determine a target search result corresponding to the target search data based on the similarity indexes.

[0081] It should be noted that the target retrieval model in the above steps is obtained by training the initial model using sample data. The sample data includes: sample retrieval data, that is, image sample retrieval data and text sample retrieval data, and sample retrieval results corresponding to the sample retrieval data. Based on the above sample data, the initial model is obtained, and the loss function for model training is constructed. Before constructing the loss function, it is necessary to first determine the joint data features. The attention mechanism neural network is used in this step, and the steps are as follows:

[0082] According to the above steps, X can represent the image hash code, Y can represent the text hash code, the subscript x in the following formula represents the parameter matrix of the image feature, y represents the parameter matrix of the text feature, and T in the upper right corner represents the transpose of the vector or matrix. are query vectors of image features, is the vector of the query image (Query vector), is the vector of the key points of the image (Key vector), is the brightness vector (Value vector) of the image, Qx is the vector of the query image corresponding to the image hash code transformation, K x is the key point vector of the image corresponding to the image hash code transformation, V x is the brightness vector of the image corresponding to the image hash code transformation, through The inner product of the matrix is ​​used to calculate the correlation between the image features X, and then the correlation matrix, V x Perform matrix multiplication to obtain image-image Attention (Q x , K x , V x ), and so on text-text Attention(Q y , K y , V y ), image-text feature Attention(Q x , K y , V x ), text-image feature Attention (Q y , K x , V y ), where Q y is the vector of the query text corresponding to the hash code transformation, K y is the key point vector of the text corresponding to the text hash code transformation, V y is the brightness vector of the text after the text hash code is transformed, Attention() is a linear activation function, Right now:

[0083] According to the image-image features, we can get:

[0084]

[0085] In the above formula, The first feature item between the image sample retrieval data and the image sample retrieval data is obtained.

[0086] According to the text-text features, we can get:

[0087]

[0088] In the above formula, It is a second feature item between the text sample retrieval data and the text sample retrieval data.

[0089] According to the image-text features, we can get:

[0090]

[0091] In the above formula, It is the third feature item between the image sample retrieval data and the text sample retrieval data.

[0092] According to the text-image features, we can get:

[0093]

[0094] In the above formula, It is a fourth feature item between the text sample retrieval data and the image sample retrieval data.

[0095] Based on the above content, taking image features as an example, where Q = XW Q , K=XW K 、V=XW V , based on the first feature item, the second feature item, the third feature item and the fourth feature item, construct the combined feature Z as follows:

[0096] Z=Attention(Q x , K x , V x )+Attention(Q y , K y , V y )+Attention(Q x , K y , V x )+Attention(Q y , K x , V y );

[0097] Input Z into the linear activation function f(Z)=ZW+b to obtain the extracted feature M, which is the joint data feature M.

[0098] After determining the joint data feature M, the loss function can be constructed. The constructed loss function formula is as follows:

[0099]

[0100] In the above formula, minLoss is the minimum loss function, X is the image hash code, Y is the text hash code, M is the joint data feature, ||XM||2 is the first loss function term, ||YM||2 is the second loss function term, is the third loss function term.

[0101] In the above implementation steps, based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

[0102] By using the optional implementation, at least the following beneficial effects can be achieved:

[0103] (1) Since the target retrieval hash value is determined by the predetermined magnitude relationship between each feature value in the target data feature and the predetermined threshold, that is, the target retrieval hash value determination process can be adjusted and optimized according to specific needs and tasks. Through such steps, the target data feature can be converted into the target retrieval hash value, so as to perform fast and efficient matching and screening in the retrieval task;

[0104] (2) Since the target joint data feature can describe the relationship data feature between the image sample retrieval data and the text sample retrieval data, this feature represents the fusion of the information of the image sample retrieval data and the text sample retrieval data, which can improve the accuracy of target data feature matching and target retrieval data matching to a certain extent;

[0105] (3) Since the loss function is established based on the joint data features, it is used to continuously update the model parameters when training the model until the model converges and finally determine the optimal target retrieval model. That is, by constructing a minimized loss function, the training model can be made more accurate and optimized for a given task.

[0106] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0107] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the embodiment can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0108] According to an embodiment of the present invention, a device for implementing the above data retrieval method is also provided. Figure 2 is a structural block diagram of a data retrieval device provided according to an embodiment of the present application, such as Figure 2As shown, the device includes: an execution module 201, an extraction module 202, a mapping module 203, a first determination module 204, a second determination module 205 and a third determination module 206. The device will be described in detail below.

[0109] The execution module 201 is configured to respond to a search request for target search data and execute the following search steps, wherein the search request is for searching for data similar to the target search data:

[0110] The extraction module 202 is connected to the execution module 201 and is used to extract the initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions;

[0111] The mapping module 203 is connected to the extraction module 202 and is used to map the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions;

[0112] A first determination module 204, connected to the mapping module 203, is used to determine a target search hash value based on target data characteristics;

[0113] The second determining module 205 is connected to the first determining module 204 and is used to determine similarity indexes between the target search hash value and the hash values ​​of multiple data in the predetermined database;

[0114] The third determining module 206 is connected to the second determining module 205 and is configured to determine a target search result corresponding to the target search data based on the similarity index.

[0115] It should be noted here that the execution module 201, extraction module 202, mapping module 203, first determination module 204, second determination module 205 and third determination module 206 correspond to steps S101 to S106 in implementing the data retrieval method, and the instances and application scenarios implemented by the multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the embodiments.

[0116] The data retrieval device provided by the embodiment of the present application can respond to a retrieval request for target retrieval data and perform the following retrieval steps, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data: extracting the initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; mapping the initial data features to obtain the target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; determining the target retrieval hash value based on the target data features; determining the similarity index between the target retrieval hash value and multiple data hash values ​​in a predetermined database; achieving the purpose of determining the target retrieval result corresponding to the target retrieval data based on the similarity index, since the target data features are obtained by mapping the initial data features, thereby achieving the technical effect of ensuring the consistency of the target data feature representation and the consistency of the target number of dimensions, and to a certain extent, improving the accuracy of the target data feature matching and the target retrieval data matching. This solves the technical problem of low retrieval accuracy in the related art when retrieving data similar to the target data.

[0117] The data retrieval device includes a processor and a memory. The multiple modules are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

[0118] The processor includes a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and adjusting kernel parameters solves the technical problem of low retrieval accuracy when searching for data similar to the target data.

[0119] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0120] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a data retrieval method is implemented.

[0121] An embodiment of the present invention provides a processor for running a program, wherein a data retrieval method is provided when the program is running.

[0122] Figure 3 is a schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 3As shown, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and runnable on the processor. When the processor executes the program, the following steps are implemented: in response to a retrieval request for target retrieval data, the following retrieval steps are performed, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data: extracting initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; determining a target retrieval hash value based on the target data features; determining a similarity index between the target retrieval hash value and multiple data hash values ​​in a predetermined database; and determining a target retrieval result corresponding to the target retrieval data based on the similarity index.

[0123] Optionally, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, including: in response to the retrieval request, the target retrieval data is input into a target retrieval model, and the target retrieval model performs the retrieval steps, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and a sample retrieval result corresponding to the sample retrieval data.

[0124] Optionally, the method also includes: training the sample data in the following manner to obtain a target retrieval model: when the sample retrieval data includes image sample retrieval data and text sample retrieval data, obtaining an initial model, and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

[0125] Optionally, determining the joint data feature includes: determining a first feature item between image sample retrieval data and image sample retrieval data, determining a second feature item between text sample retrieval data and text sample retrieval data, determining a third feature item between image sample retrieval data and text sample retrieval data, and determining a fourth feature item between text sample retrieval data and image sample retrieval data; constructing a combined feature based on the first feature item, the second feature item, the third feature item and the fourth feature item; substituting the combined feature into a predetermined linear activation function to obtain the joint data feature.

[0126] Optionally, determining the first feature item between the image sample retrieval data and the image sample retrieval data, determining the second feature item between the text sample retrieval data and the text sample retrieval data, determining the third feature item between the image sample retrieval data and the text sample retrieval data, and determining the fourth feature item between the text sample retrieval data and the image sample retrieval data includes: determining the ratio of the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the image sample retrieval data and the target number of dimensions to obtain a first ratio, and determining the first query vector corresponding to the text sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data and the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data and the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data and the second query vector corresponding to the text sample retrieval data The product of the transpose vector of the quantity and the ratio of the target dimension number are obtained to obtain a third ratio, and the first query vector corresponding to the text sample retrieval data and the product of the transpose vector of the second query vector corresponding to the image sample retrieval data and the ratio of the target dimension number are determined to obtain a fourth ratio; the first ratio, the second ratio, the third ratio and the fourth ratio are respectively input into the predetermined probability distribution conversion function to obtain the corresponding first result, second result, third result and fourth result respectively; the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is determined as the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is determined as the fourth feature item.

[0127] Optionally, the method further includes: obtaining sample data in the following manner: determining search results corresponding to the data in the predetermined database; and using the data in the predetermined database and the search results corresponding to the data in the predetermined database as sample data.

[0128] Optionally, determining a target retrieval hash value based on target data characteristics includes: determining a size relationship between each characteristic value in the target data characteristics and a predetermined threshold value to obtain a predetermined size relationship; updating a characteristic value whose corresponding characteristic value is greater than the predetermined threshold value in the predetermined size relationship to a first predetermined value, and updating a characteristic value whose corresponding characteristic value is less than or equal to the predetermined threshold value to a second predetermined value to obtain a target retrieval hash value, wherein the first predetermined value is higher than the second predetermined value.

[0129] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0130] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: responding to a retrieval request for target retrieval data, executing the following retrieval steps, wherein the retrieval request is used to request retrieval of data similar to the target retrieval data: extracting initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions; determining a target retrieval hash value based on the target data features; determining a similarity index between the target retrieval hash value and a plurality of data hash values ​​in a predetermined database; and determining a target retrieval result corresponding to the target retrieval data based on the similarity index.

[0131] Optionally, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, including: in response to the retrieval request, the target retrieval data is input into a target retrieval model, and the target retrieval model performs the retrieval steps, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and a sample retrieval result corresponding to the sample retrieval data.

[0132] Optionally, the method also includes: training the sample data in the following manner to obtain a target retrieval model: when the sample retrieval data includes image sample retrieval data and text sample retrieval data, obtaining an initial model, and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

[0133] Optionally, determining the joint data feature includes: determining a first feature item between image sample retrieval data and image sample retrieval data, determining a second feature item between text sample retrieval data and text sample retrieval data, determining a third feature item between image sample retrieval data and text sample retrieval data, and determining a fourth feature item between text sample retrieval data and image sample retrieval data; constructing a combined feature based on the first feature item, the second feature item, the third feature item and the fourth feature item; substituting the combined feature into a predetermined linear activation function to obtain the joint data feature.

[0134] Optionally, determining the first feature item between the image sample retrieval data and the image sample retrieval data, determining the second feature item between the text sample retrieval data and the text sample retrieval data, determining the third feature item between the image sample retrieval data and the text sample retrieval data, and determining the fourth feature item between the text sample retrieval data and the image sample retrieval data includes: determining the ratio of the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the image sample retrieval data and the target number of dimensions to obtain a first ratio, and determining the first query vector corresponding to the text sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data and the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data and the target number of dimensions to obtain a second ratio, and determining the first query vector corresponding to the image sample retrieval data and the second query vector corresponding to the text sample retrieval data The product of the transpose vector of the quantity and the ratio of the target dimension number are obtained to obtain a third ratio, and the first query vector corresponding to the text sample retrieval data and the product of the transpose vector of the second query vector corresponding to the image sample retrieval data and the ratio of the target dimension number are determined to obtain a fourth ratio; the first ratio, the second ratio, the third ratio and the fourth ratio are respectively input into the predetermined probability distribution conversion function to obtain the corresponding first result, second result, third result and fourth result respectively; the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is determined as the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is determined as the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is determined as the fourth feature item.

[0135] Optionally, the method further includes: obtaining sample data in the following manner: determining search results corresponding to the data in the predetermined database; and using the data in the predetermined database and the search results corresponding to the data in the predetermined database as sample data.

[0136] Optionally, determining a target retrieval hash value based on target data characteristics includes: determining a size relationship between each characteristic value in the target data characteristics and a predetermined threshold value to obtain a predetermined size relationship; updating a characteristic value whose corresponding characteristic value is greater than the predetermined threshold value in the predetermined size relationship to a first predetermined value, and updating a characteristic value whose corresponding characteristic value is less than or equal to the predetermined threshold value to a second predetermined value to obtain a target retrieval hash value, wherein the first predetermined value is higher than the second predetermined value.

[0137] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0141] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0142] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (lashRAM). The memory is an example of a computer-readable medium.

[0143] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0144] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0145] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data retrieval method, characterized in that: include: In response to a search request for target search data, the following search steps are performed, wherein the search request is used to request the retrieval of data similar to the target search data: Extracting initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; Mapping the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions, and the target data features are in the same public space; Determining a target retrieval hash value based on the target data characteristics; Determining similarity indexes between the target search hash value and a plurality of data hash values ​​in a predetermined database; Determining a target search result corresponding to the target search data based on the similarity index; Wherein, in response to a retrieval request for target retrieval data, the following retrieval steps are performed, including: in response to the retrieval request, inputting the target retrieval data into a target retrieval model, the target retrieval model performing the retrieval steps, wherein the target retrieval model is obtained by training an initial model with sample data, the sample data including sample retrieval data and a sample retrieval result corresponding to the sample retrieval data; The method also includes: training the sample data in the following manner to obtain the target retrieval model: when the sample retrieval data includes image sample retrieval data and text sample retrieval data, obtaining an initial model, and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

2. The method according to claim 1, characterized in that Determining the joint data characteristics includes: Determining a first feature item between the image sample retrieval data and the image sample retrieval data, determining a second feature item between the text sample retrieval data and the text sample retrieval data, determining a third feature item between the image sample retrieval data and the text sample retrieval data, and determining a fourth feature item between the text sample retrieval data and the image sample retrieval data; Constructing a combined feature based on the first feature item, the second feature item, the third feature item and the fourth feature item; Substituting the combined feature into a predetermined linear activation function to obtain the joint data feature.

3. The method according to claim 2, characterized in that The determining of the first feature item between the image sample retrieval data and the image sample retrieval data, determining the second feature item between the text sample retrieval data and the text sample retrieval data, determining the third feature item between the image sample retrieval data and the text sample retrieval data, and determining the fourth feature item between the text sample retrieval data and the image sample retrieval data includes: Determine the ratio of the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the image sample retrieval data to the target number of dimensions to obtain a first ratio, determine the ratio of the first query vector corresponding to the text sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data to the target number of dimensions to obtain a second ratio, determine the ratio of the first query vector corresponding to the image sample retrieval data and the product of the transposed vector of the second query vector corresponding to the text sample retrieval data to the target number of dimensions to obtain a third ratio, and determine the ratio of the first query vector corresponding to the text sample retrieval data and the product of the transposed vector of the second query vector corresponding to the image sample retrieval data to the target number of dimensions to obtain a fourth ratio; Inputting the first ratio, the second ratio, the third ratio and the fourth ratio into a predetermined probability distribution conversion function respectively to obtain a corresponding first result, a second result, a third result and a fourth result respectively; Determine that the product of the first result and the third query vector corresponding to the image sample retrieval data is the first feature item, the product of the second result and the third query vector corresponding to the text sample retrieval data is the second feature item, the product of the first result and the third query vector corresponding to the image sample retrieval data is the third feature item, and the product of the first result and the third query vector corresponding to the text sample retrieval data is the fourth feature item.

4. The method according to claim 1, wherein The method further includes: obtaining the sample data in the following manner: Determining search results corresponding to the data in the predetermined database; The data in the predetermined database and the search results corresponding to the data in the predetermined database are used as the sample data.

5. The method according to any one of claims 1 to 4, characterized in that Determining a target search hash value based on the target data characteristics includes: Determine a magnitude relationship between each characteristic value in the target data characteristic and a predetermined threshold value to obtain a predetermined magnitude relationship; In the predetermined size relationship, the corresponding eigenvalue whose eigenvalue is greater than the predetermined threshold is updated to a first predetermined value, and the corresponding eigenvalue whose eigenvalue is less than or equal to the predetermined threshold is updated to a second predetermined value to obtain the target retrieval hash value, wherein the first predetermined value is higher than the second predetermined value.

6. A data retrieval device, characterized in that: include: An execution module is configured to respond to a search request for target search data and execute the following search steps, wherein the search request is for requesting to retrieve data similar to the target search data: An extraction module, configured to extract initial data features of the target retrieval data, wherein the number of dimensions corresponding to the initial data features is the initial number of dimensions; a mapping module, configured to map the initial data features to obtain target data features, wherein the number of dimensions corresponding to the target data features is the target number of dimensions, and the target data features are in the same public space; A first determining module, configured to determine a target retrieval hash value based on the target data characteristics; A second determination module is used to determine similarity indexes between the target search hash value and a plurality of data hash values ​​in a predetermined database; A third determining module is used to determine a target search result corresponding to the target search data according to the similarity index; The execution module is further configured to input the target retrieval data into a target retrieval model in response to the retrieval request, and the target retrieval model executes the retrieval step, wherein the target retrieval model is obtained by training an initial model with sample data, and the sample data includes sample retrieval data and a sample retrieval result corresponding to the sample retrieval data; The execution module is also used to train the sample data in the following manner to obtain the target retrieval model: when the sample retrieval data includes image sample retrieval data and text sample retrieval data, an initial model is obtained, and a loss function for model training is constructed, wherein the loss function includes a first loss function term, a second loss function term and a third loss function term, the first loss function term is used to maintain the similarity between the first sample data feature corresponding to the image sample retrieval data and the joint data feature, the second loss function term is used to maintain the similarity between the second sample data feature corresponding to the text sample retrieval data and the joint data feature, and the third loss function term is used to maintain the similarity between the joint data feature and the true similarity matrix; based on the loss function, the sample data is used to train the initial model to obtain the target retrieval model.

7. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the method according to any one of claims 1 to 5 when running.

8. An electronic device, characterized in that: The method comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

Images