Training method, retrieval method, system and equipment of data retrieval model

Abstract

The invention discloses a training method, a retrieval method, a system and equipment of a data retrieval model. The method comprises the following steps: acquiring image data and corresponding text data; mapping the image data and the text data into a common subspace by using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data; establishing a corresponding relationship between the first feature data and the second feature data based on the corresponding relationship between the image data and the text data; calculating the feature similarity between the first feature data and the corresponding second feature data; and determining a target loss value according to the feature similarity, and performing parameter updating on the to-be-optimized data retrieval model based on the target loss value to obtain a trained data retrieval model. The training method is beneficial to improving the accuracy and efficiency of data retrieval. The method can be widely applied to the technical field of retrieval.

Description

Technical Field

[0001] The present application relates to the field of retrieval technology, and in particular to a training method, a retrieval method, a system and a device for a data retrieval model. Background Art

[0002] Currently, data in all walks of life are in a stage of explosive growth. Taking the financial field as an example, financial data covers a wide range of data types, from traditional numerical data to complex multimedia data (such as images and text). These data play a vital role in analysis, prediction and decision-making in the financial field. Financial practitioners need to quickly and accurately extract useful information from large amounts of data to support decision-making and analysis.

[0003] In the related art, a keyword-based search method is generally used to search for data. This search method has limitations in application. For example, different words may have similar meanings, but simple keyword matching cannot capture this semantic relationship, that is, it is difficult to fully utilize the inherent correlation between data, resulting in insufficient accuracy and completeness of search results.

[0004] In summary, the problems existing in related technologies need to be solved urgently. Summary of the invention

[0005] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, one purpose of an embodiment of the present application is to provide a training method for a data retrieval model, which is based on the semantic alignment between image data and text data, can better capture the intrinsic correlation between different data, and is conducive to improving the accuracy and efficiency of data retrieval.

[0007] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present application include:

[0008] On the one hand, the present application embodiment provides a method for training a data retrieval model, comprising:

[0009] Acquire image data and text data corresponding to the image data;

[0010] Based on the data retrieval model to be optimized, the image data and the text data are mapped to a common subspace using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data;

[0011] Based on the correspondence between the image data and the text data, establishing a correspondence between the first feature data and the second feature data;

[0012] Calculating feature similarity between the first feature data and the corresponding second feature data;

[0013] According to the feature similarity, a target loss value is determined, and based on the target loss value, parameters of the data retrieval model to be optimized are updated to obtain a trained data retrieval model.

[0014] In addition, the training method of the data retrieval model according to the above embodiment of the present application may also have the following additional technical features:

[0015] Furthermore, in one embodiment of the present application, the acquiring of the image data and the text data corresponding to the image data includes:

[0016] Get image data;

[0017] Performing text area detection on the image data to determine a target area;

[0018] Optical character recognition is performed on the target area to obtain text data corresponding to the image data.

[0019] Further, in one embodiment of the present application, the using of a manifold learning algorithm to map the image data and the text data into a common subspace to obtain first feature data corresponding to the image data and second feature data corresponding to the text data includes:

[0020] Extracting third feature data corresponding to the image data, and extracting fourth feature data corresponding to the text data;

[0021] Performing local linear embedding on the third feature data in the common subspace to obtain first feature data after dimensionality reduction;

[0022] The fourth feature data is locally linearly embedded in the common subspace to obtain the second feature data after dimensionality reduction.

[0023] Further, in one embodiment of the present application, the first feature data and the second feature data are vector data; the calculating the feature similarity between the first feature data and the corresponding second feature data includes:

[0024] Calculating a first length of the first feature data and a second length of the second feature data;

[0025] A first value is obtained according to the product of the first length and the second length, and a second value is obtained according to the inner product of the first characteristic data and the second characteristic data;

[0026] The feature similarity is obtained according to the quotient of the second value and the first value.

[0027] Furthermore, in one embodiment of the present application, determining the target loss value according to the feature similarity includes:

[0028] Calculate the mean data of the feature similarities corresponding to each of the first feature data, and determine the first loss value according to the mean data; wherein the magnitude of the first loss value is negatively correlated with the magnitude of the mean data;

[0029] Determining a sparse norm according to the parameters of the data retrieval model;

[0030] A target loss value is determined according to the first loss value and the sparse norm.

[0031] Further, in one embodiment of the present application, the step of updating the parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model includes:

[0032] Get the preset training iteration rounds;

[0033] Based on the target loss value, the parameters of the data retrieval model to be optimized are updated, and the number of times the parameters are currently updated is queried;

[0034] If the number of parameter updates currently performed is equal to the number of training iteration rounds, the parameter update of the data retrieval model is stopped, and the current data retrieval model is determined as the trained data retrieval model.

[0035] On the other hand, an embodiment of the present application further provides a data retrieval method, the method comprising:

[0036] Get the search text data entered by the target object;

[0037] Inputting the search text data into the data search model trained by the aforementioned data search model training method, and searching and determining the target data through the data search model; wherein the target data includes at least one of image data or text data;

[0038] The target data is fed back to the target object.

[0039] On the other hand, an embodiment of the present application also provides a data retrieval model training system, including:

[0040] An acquisition unit, used for acquiring image data and text data corresponding to the image data;

[0041] A mapping unit, used to map the image data and the text data into a common subspace using a manifold learning algorithm based on the data retrieval model to be optimized, so as to obtain first feature data corresponding to the image data and second feature data corresponding to the text data;

[0042] an establishing unit, configured to establish a corresponding relationship between the first feature data and the second feature data based on the corresponding relationship between the image data and the text data;

[0043] A calculation unit, used for calculating the feature similarity between the first feature data and the corresponding second feature data;

[0044] An updating unit is used to determine a target loss value according to the feature similarity, and to update parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model.

[0045] On the other hand, an embodiment of the present application provides a computer device, including:

[0046] at least one processor;

[0047] at least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned data retrieval model training method or data retrieval method.

[0049] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned data retrieval model training method or data retrieval method.

[0050] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0051] The training method of the data retrieval model disclosed in the embodiment of the present application, the training method obtains image data and text data corresponding to the image data; based on the data retrieval model to be optimized, the image data and the text data are mapped to a common subspace using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data; based on the correspondence between the image data and the text data, a correspondence between the first feature data and the second feature data is established; the feature similarity between the first feature data and the corresponding second feature data is calculated; according to the feature similarity, a target loss value is determined, and based on the target loss value, the parameters of the data retrieval model to be optimized are updated to obtain a trained data retrieval model. The training method can train a model that can be used to implement data retrieval, and based on the semantic alignment between image data and text data, it can better capture the intrinsic correlation between different data, which is conducive to improving the accuracy and efficiency of data retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of a training method for a data retrieval model and an implementation environment of a data retrieval method provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a flow chart of a method for training a data retrieval model provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a process for obtaining image data and text data provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a process for extracting first feature data and second feature data provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of a process for determining feature similarity provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a process for determining a target loss value provided in an embodiment of the present application;

[0059] Figure 7A schematic diagram of the structure of a training system for a data retrieval model provided in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0062] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0064] First, some nouns involved in this application are analyzed:

[0065] 1) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0066] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0067] 2) Machine Learning (ML) is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning. The pre-trained model is the latest development in deep learning, which integrates the above technologies.

[0068] 3) Manifold Learning is a class of data mining algorithms and techniques for nonlinear dimensionality reduction. It assumes that high-dimensional data actually lies on or close to the manifold of low-dimensional space. By finding this manifold, high-dimensional data can be mapped to a lower-dimensional space while maintaining the important properties and structure of the original data. This technique is particularly suitable for processing high-dimensional data sets with intrinsic low-dimensional structures.

[0069] 4) Locally Linear Embedding (LLE) is a nonlinear dimensionality reduction method, mainly used for data visualization and feature extraction of high-dimensional data. The basic idea of ​​LLE is to reconstruct the local relationship between data points in a low-dimensional space so that the reconstruction error is minimized.

[0070] The data retrieval model training method, data retrieval method, system, device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the data retrieval model training method in the embodiments of the present application is described.

[0071] The data involved in the data retrieval model training method and data retrieval method provided in the embodiments of the present application may be data, which can be executed in various application scenarios:

[0072] For example, in the application scenario of enterprise analysis, by retrieving enterprise-related data, such as profit data, asset-liability data, and cash flow data, etc., the latest developments of the enterprise can be tracked, making it convenient to adjust investment strategies in a timely manner.

[0073] For example, in the application scenario of market situation monitoring, through the data retrieval method provided in the embodiment of the present application, it is possible to search and query the transaction data related to stocks, or track foreign exchange rates, thereby realizing data collection and processing for the financial market.

[0074] For example, in the application scenario of risk management, for certain financial products (such as bonds, funds, etc.), relevant business data can be collected through the data retrieval method provided in the embodiments of the present application, so as to perform credit rating on them and assess possible financial risks.

[0075] Of course, it should be noted that the above application scenarios are only exemplary and do not limit the actual application of the method in the embodiment of the present application. Those skilled in the art can understand that in different application scenarios, the method provided in the embodiment of the present application can be used to perform the specified task.

[0076] Currently, data in all walks of life are in a stage of explosive growth. Taking the financial field as an example, financial data covers a wide range of data types, from traditional numerical data to complex multimedia data (such as images and text). These data play a vital role in analysis, prediction and decision-making in the financial field. Financial practitioners need to quickly and accurately extract useful information from large amounts of data to support decision-making and analysis.

[0077] In the related art, a keyword-based search method is generally used to search for data. This search method has limitations in application. For example, different words may have similar meanings, but simple keyword matching cannot capture this semantic relationship, that is, it is difficult to fully utilize the inherent correlation between data, resulting in insufficient accuracy and completeness of search results.

[0078] In order to solve the problems existing in the related art, the embodiment of the present application provides a training method, a retrieval method, a system and a device for a data retrieval model. The training method obtains image data and text data corresponding to the image data; based on the data retrieval model to be optimized, the image data and the text data are mapped to a common subspace using a manifold learning algorithm to obtain the first feature data corresponding to the image data and the second feature data corresponding to the text data; based on the correspondence between the image data and the text data, the correspondence between the first feature data and the second feature data is established; the feature similarity between the first feature data and the corresponding second feature data is calculated; according to the feature similarity, a target loss value is determined, and the parameters of the data retrieval model to be optimized are updated based on the target loss value to obtain a trained data retrieval model. The training method can train a model that can be used to implement data retrieval. Based on the semantic alignment between the image data and the text data, it can better capture the intrinsic correlation between different data, which is conducive to improving the accuracy and efficiency of data retrieval.

[0079] Of course, it should be noted that the data involved in the retrieval of the methods in the embodiments of the present application may also be data in other fields besides data. For example, in some embodiments, the methods in the present application can be applied in the medical field to realize the search for medical images and diagnostic texts; in some embodiments, the methods in the present application can be applied in the media and entertainment field to realize the search for video images and reporting information. The present application does not impose any restrictions on this.

[0080] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of the implementation environment of the data retrieval model training method and the data retrieval method provided in the embodiment of the present application. The hardware and software components of the implementation environment mainly include a user terminal 110 and a server 120, and the user terminal 110 is in communication connection with the server 120. The data retrieval model training method and the data retrieval method can be configured on the user terminal 110, can also be configured on the server 120 side for execution, or can be executed according to the interaction between the two.

[0081] Specifically, the user terminal 110 in the present application may include but is not limited to any one or more of a smart watch, a smart phone, a computer, a personal digital assistant (PDA), an intelligent voice interaction device, a smart home appliance or a vehicle-mounted terminal. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The user terminal 110 and the server 120 can establish a communication connection through a wireless network or a wired network, which uses standard communication technology and / or protocols. The network can be set to the Internet or any other network, such as but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network.

[0082] Of course, it is understandable that Figure 1 The implementation environment in the embodiment of the present application is only an optional application scenario of the training method of the data retrieval model provided in the embodiment of the present application, and the actual application is not fixed to Figure 1 The hardware and software environment shown.

[0083] Combine the following Figure 1 The implementation environment shown is used to explain in detail the training method of the data retrieval model and the data retrieval method provided in the embodiments of the present application.

[0084] First, please refer to Figure 2 , Figure 2 is a flow chart of a training method for a data retrieval model provided in an embodiment of the present application, Figure 2 The training method of the data retrieval model shown can be applied to the user terminal 110 or the relevant computer device in the server 120, but is not limited to the above form. Figure 2 The method includes but is not limited to steps 210 to 250.

[0085] Step 210: Acquire image data and text data corresponding to the image data;

[0086] Step 220: Based on the data retrieval model to be optimized, the image data and the text data are mapped to a common subspace using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data;

[0087] Step 230: establishing a corresponding relationship between the first feature data and the second feature data based on the corresponding relationship between the image data and the text data;

[0088] Step 240: Calculate the feature similarity between the first feature data and the corresponding second feature data;

[0089] Step 250: Determine a target loss value according to the feature similarity, and update the parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model.

[0090] In an embodiment of the present application, a method for training a data retrieval model is provided. The data retrieval model can search and retrieve target data that best matches the text data in relevant data sources based on the retrieval text data input by the user, and then feed it back to the user. Specifically, its basic principle is: extract relevant feature data based on the retrieval text data input by the user, and then search the data source for the corresponding feature data and the data that is most relevant to the feature data, so as to determine the target data. In an embodiment of the present application, before the data retrieval model is put into use, it can be trained to improve its performance, which is conducive to improving the accuracy of the retrieval.

[0091] Specifically, when training the data retrieval model, the image data and the text data corresponding to the image data can be obtained in batches. In the embodiment of the present application, the specific contents contained in the image data and the text data can be diverse. For example, taking the financial scenario as an example, for the image data, in some embodiments, the image data can include data of chart type, such as the trend chart of stock price, K-line chart, technical indicator chart, etc.; in some embodiments, the image data can include photos of physical objects, such as images of banknotes and coins issued by different countries and regions, images of physical vouchers in financial transactions, etc.; in some embodiments, the image data can include the cover and screenshots of relevant reports. For the text data, in some embodiments, the text data can include news reports and comments, such as news reports on macroeconomics, financial markets, listed companies, etc. In some embodiments, the text data can include research reports, such as research reports on the development status, competitive landscape, future trends, etc. of a specific industry; in some embodiments, the text data can include financial statements and descriptions. Of course, it can be understood that the specific contents contained in the image data and text data can be various, and the present application does not limit this.

[0092] It should be noted that in the embodiments of the present application, there is a corresponding relationship between some image data and text data. For example, a certain image data shows a trend chart of stock prices, and a certain text data is an analysis of the trend chart of stock prices. The two have a strong correlation. For this case, a corresponding relationship between the two can be established. Of course, it is understandable that the corresponding relationship between image data and text data can be one image data corresponding to multiple text data, or multiple image data corresponding to one text data, and the present application does not limit this.

[0093] In the embodiment of the present application, after acquiring the image data and the text data, the data retrieval model to be optimized can be used to map the image data and the text data to a common subspace through a manifold learning algorithm to obtain feature data corresponding to the image data and the text data. Here, the feature data corresponding to the image data is recorded as the first feature data, and the feature data corresponding to the text data is recorded as the second feature data. In the embodiment of the present application,

[0094] After obtaining the first feature data and the second feature data, a corresponding relationship between the first feature data and the second feature data can be established based on the corresponding relationship between the image data and the text data. That is, if there is a corresponding relationship between a certain image data and a certain text data, then a corresponding relationship can also be established between the first feature data extracted based on the image data and the second feature data extracted based on the text data. It can be understood that the corresponding relationship between the image data and the text data indicates that the contents of the two are relatively close or have a strong correlation. Therefore, in the feature space, it is hoped that the first feature data and the second feature data are also relatively close, which indicates that the data retrieval model has good feature extraction and recognition capabilities.

[0095] In an embodiment of the present application, the feature similarity between the first feature data and the corresponding second feature data can be calculated. Based on the feature similarity, it can be measured whether the feature extraction capability of the data retrieval model meets the requirements. If the feature similarity between the first feature data and the corresponding second feature data is low, it means that the performance of the data retrieval model is weak. At this time, a larger loss value can be determined. On the contrary, if the feature similarity between the first feature data and the corresponding second feature data is high, it means that the performance of the data retrieval model is strong. At this time, a smaller loss value can be determined. In an embodiment of the present application, the loss value corresponding to the data retrieval model is recorded as the target loss value. Through the target loss value, the parameters of the data retrieval model to be optimized can be updated to obtain a trained data retrieval model.

[0096] It can be understood that the data retrieval model training method provided in the embodiments of the present application can train a model that can be used to implement data retrieval. Based on the semantic alignment between image data and text data, it can better capture the intrinsic correlation between different data, which is conducive to improving the accuracy and efficiency of data retrieval.

[0097] Specifically, refer to Figure 3 , Figure 3 A schematic diagram of a process for acquiring image data and text data provided in an embodiment of the present application is shown. In some embodiments, the acquisition of image data and text data corresponding to the image data includes:

[0098] Get image data;

[0099] Performing text area detection on the image data to determine a target area;

[0100] Optical character recognition is performed on the target area to obtain text data corresponding to the image data.

[0101] In the embodiment of the present application, when acquiring image data and text data, in some cases, the image data itself may contain some text data. Therefore, it is possible to directly obtain the text data contained in the image data through recognition as the text data corresponding to the image data.

[0102] Specifically, in the embodiment of the present application, after the image data is obtained, it can be pre-processed, for example, graying, binarization, noise reduction and other operations can be performed to improve the accuracy of subsequent text recognition and extraction. After pre-processing, the image data can be detected for text area, and the area where the content of the text part is located is determined as the target area. Here, when performing text area detection, there are many methods that can be used. For example, in some embodiments, a method based on edge and contour can be used. This method first uses an edge detection algorithm (such as Canny edge detection) to find the edge in the image, and then forms a contour by finding continuous edges, and screens out contours that may contain text. In some embodiments, a method based on a connected domain can be used, such as converting the image into a black and white image by binarization, and then finding all connected white pixels (i.e., the text part), and judging whether it is a text area according to the size, shape and other features of the connected domain. In some embodiments, a method based on deep learning can also be used, such as using some text detection models to determine the target area, and this application does not limit this.

[0103] In the embodiment of the present application, after determining the target area, optical character recognition can be performed on the target area to obtain text data corresponding to the image data. Specifically, the detected text area can be input into an optical character recognition engine for character recognition and converted into a text format. It should be noted that the recognized text may contain some errors, which can be corrected in the embodiment of the present application. This can be achieved through spell checking, dictionary-based methods or machine learning models, which will not be elaborated here.

[0104] Specifically, refer to Figure 4 , Figure 4 A schematic diagram of a process for extracting first feature data and second feature data provided in an embodiment of the present application is shown. In some embodiments, the use of a manifold learning algorithm to map the image data and the text data to a common subspace to obtain the first feature data corresponding to the image data and the second feature data corresponding to the text data includes:

[0105] Extracting third feature data corresponding to the image data, and extracting fourth feature data corresponding to the text data;

[0106] Performing local linear embedding on the third feature data in the common subspace to obtain first feature data after dimensionality reduction;

[0107] The fourth feature data is locally linearly embedded in the common subspace to obtain the second feature data after dimensionality reduction.

[0108] In an embodiment of the present application, when extracting the first feature data and the second feature data, first, a basic method can be used to extract the feature data corresponding to the image data, which is recorded as the third feature data. For example, a related image encoder can be used to extract the third feature data. In addition, the feature data corresponding to the text data is extracted using the same basic method, which is recorded as the fourth feature data. Similarly, this process can be implemented by a related text encoder, which is not described in detail here. After obtaining the third feature data and the fourth feature data, the third feature data can be reduced in dimension using a local linear embedding method in a common subspace to obtain the first feature data, and the fourth feature data can be reduced in dimension to obtain the second feature data.

[0109] The Locally Linear Embedding (LLE) used in the embodiments of the present application is a nonlinear dimensionality reduction method, which is mainly used for data visualization and feature extraction of high-dimensional data. The basic idea of ​​local linear embedding is to reconstruct the local relationship between data points in a low-dimensional space so that the reconstruction error is minimized. Dimensionality reduction based on local linear embedding can maintain the adjacent reconstruction relationship of feature data in the original space, which can reduce the dimensionality reduction loss of features, and is conducive to improving the performance of data retrieval models while reducing the amount of data processing.

[0110] Specifically, refer to Figure 5 , Figure 5 A schematic diagram of a process for determining feature similarity provided in an embodiment of the present application is shown. In some embodiments, the first feature data and the second feature data are vector data; the calculation of the feature similarity between the first feature data and the corresponding second feature data includes:

[0111] Calculating a first length of the first feature data and a second length of the second feature data;

[0112] A first value is obtained according to the product of the first length and the second length, and a second value is obtained according to the inner product of the first characteristic data and the second characteristic data;

[0113] The feature similarity is obtained according to the quotient of the second value and the first value.

[0114] In the embodiment of the present application, the first feature data and the second feature data can be represented in vector form. For the feature data in vector form, the feature similarity between the first feature data and the second feature data can be determined by a cosine feature similarity algorithm.

[0115] Specifically, in the embodiment of the present application, the length of the first feature data, recorded as the first length, and the length of the second feature data, recorded as the second length, can be first determined. Then, the product of the first length and the second length is calculated as the first value, and the inner product of the first feature data and the second feature data is calculated as the second value, and then the quotient of the first value and the second value is calculated as the feature similarity between the first feature data and the second feature data.

[0116] Specifically, refer to Figure 6 , Figure 6 A schematic diagram of a process for determining a target loss value provided in an embodiment of the present application is shown. In some embodiments, determining the target loss value based on the feature similarity includes:

[0117] Calculate the mean data of the feature similarities corresponding to each of the first feature data, and determine the first loss value according to the mean data; wherein the magnitude of the first loss value is negatively correlated with the magnitude of the mean data;

[0118] Determining a sparse norm according to the parameters of the data retrieval model;

[0119] A target loss value is determined according to the first loss value and the sparse norm.

[0120] In the embodiment of the present application, for the artificial intelligence model, its prediction accuracy can be measured by the loss function (Loss Function), which is defined on a single (group) of training data and is used to measure the prediction error of a training data. In actual training, a training data set has a lot of training data, such as the embodiment of the present application includes multiple groups of image data and text data corresponding to the image data. Therefore, the cost function (CostFunction) is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction error of all training data, which can better measure the prediction effect of the model. In the embodiment of the present application, when determining the target loss value, the feature similarities corresponding to all the first feature data can be calculated first, and then the average value is calculated for them, and a loss value is determined based on the mean data, which is recorded as the first loss value. The size of the first loss value is negatively correlated with the size of the mean data. This application does not limit the specific functional relationship between the two. In addition, in the embodiment of the present application, a sparse norm can be determined according to the parameters of the data retrieval model. The sparse norm refers to a mathematical specification for promoting the sparsity of vectors or matrices, which is widely used in many fields, especially in machine learning and signal processing. The main purpose of the sparse norm is to make the solution of the optimization problem as sparse as possible, that is, as many elements as possible are zero. Specifically, in the embodiment of the present application, the L1 norm or the L2 norm can be used to sum the first loss value and the sparse norm to obtain the target loss value, thereby updating the parameters of the data retrieval model.

[0121] In the embodiment of the present application, the parameter update of the data retrieval model can be performed in a cyclic iteration manner. That is, after updating the parameters of a round of data retrieval model, the data retrieval model with updated parameters is continued to be processed to determine a new target loss value, and then the parameters of the data retrieval model are updated again. This cycle is repeated until the pre-set training end condition is met, and the training can be considered completed, and a trained data retrieval model is obtained.

[0122] In the embodiments of the present application, the conditions for the end of training can be flexibly set according to the needs. For example, in some embodiments, the target round of training cycle iteration can be set as the condition for the end of training. When the update round of the parameters of the data retrieval model reaches the target round, the training can be considered to be completed; in some embodiments, the difference threshold of the target loss value obtained in two adjacent training processes can be set as the condition for the end of training. After updating the parameters of a round of data retrieval model, the absolute value of the difference between the comprehensive loss value obtained in this round of training and the comprehensive loss value obtained in the previous round of training is calculated. If the absolute value of the difference is greater than the set difference threshold, the iterative training continues; if the absolute value of the difference is less than or equal to the set difference threshold, the training can be considered to be completed. Of course, the above is only an exemplary introduction to some optional training end condition setting methods in the embodiments of the present application, and does not mean to limit the situation during actual implementation.

[0123] In an embodiment of the present application, a data retrieval method is also provided, which specifically includes:

[0124] Get the search text data entered by the target object;

[0125] Inputting the search text data into the data search model trained by the aforementioned data search model training method, and searching and determining the target data through the data search model; wherein the target data includes at least one of image data or text data;

[0126] The target data is fed back to the target object.

[0127] In the embodiment of the present application, after the data retrieval model is trained, it can be formally put into application. In the application stage, according to the retrieval text data input by the user (i.e., the target object), the target data that best matches the text data can be searched and retrieved in the relevant data source, and then fed back to the user. Its basic implementation principle has been introduced in the above embodiment and will not be repeated here.

[0128] It can be understood that the contents of the above-mentioned data retrieval model training method embodiment are all applicable to the present data retrieval method embodiment, and the beneficial effects achieved by the present data retrieval method embodiment are the same as the beneficial effects achieved by the above-mentioned data retrieval model training embodiment.

[0129] Reference Figure 7 In an embodiment of the present application, a training system for a data retrieval model is also proposed, comprising:

[0130] An acquisition unit 710 is used to acquire image data and text data corresponding to the image data;

[0131] A mapping unit 720, configured to map the image data and the text data to a common subspace using a manifold learning algorithm based on the data retrieval model to be optimized, so as to obtain first feature data corresponding to the image data and second feature data corresponding to the text data;

[0132] An establishing unit 730, configured to establish a corresponding relationship between the first feature data and the second feature data based on the corresponding relationship between the image data and the text data;

[0133] A calculation unit 740, configured to calculate a feature similarity between the first feature data and the corresponding second feature data;

[0134] The updating unit 750 is used to determine a target loss value according to the feature similarity, and update the parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model.

[0135] It can be understood that the contents of the above-mentioned data retrieval model training method embodiment are applicable to the present processing system embodiment, the functions specifically implemented by the present processing system embodiment are the same as those in the above-mentioned data retrieval model training method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned data retrieval model training method embodiment.

[0136] Reference Figure 8 , the embodiment of the present application further discloses a computer device, including:

[0137] at least one processor 810;

[0138] At least one memory 820, used to store at least one program;

[0139] When at least one program is executed by at least one processor 820, at least one processor 820 implements the above-mentioned data retrieval model training method embodiment or data retrieval method embodiment or.

[0140] It can be understood that the contents of the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment are applicable to the present computer device embodiment, the functions specifically implemented by the present computer device embodiment are the same as those of the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment.

[0141] The embodiment of the present application also discloses a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the training method embodiment of the above-mentioned data retrieval model or the data retrieval method embodiment.

[0142] It can be understood that the contents of the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment are applicable to the present computer-readable storage medium embodiment, and the functions specifically implemented by the present computer-readable storage medium embodiment are the same as those of the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned data retrieval model training method embodiment or the data retrieval method embodiment.

[0143] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0144] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0145] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0147] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0148] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0149] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0150] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0151] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

[0152] In the description of this specification, the description with reference to the terms "one embodiment", "another embodiment" or "certain embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0153] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A training method for a data retrieval model, It is characterized in that include: Acquire image data and text data corresponding to the image data; Based on the data retrieval model to be optimized, the image data and the text data are mapped to a common subspace using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data; Based on the correspondence between the image data and the text data, establishing a correspondence between the first feature data and the second feature data; Calculating feature similarity between the first feature data and the corresponding second feature data; According to the feature similarity, a target loss value is determined, and based on the target loss value, parameters of the data retrieval model to be optimized are updated to obtain a trained data retrieval model.

2. A method for training a data retrieval model according to claim 1, It is characterized in that The acquiring of image data and text data corresponding to the image data comprises: Get image data; Performing text area detection on the image data to determine a target area; Optical character recognition is performed on the target area to obtain text data corresponding to the image data.

3. A method for training a data retrieval model according to claim 1, It is characterized in that The method of mapping the image data and the text data to a common subspace using a manifold learning algorithm to obtain first feature data corresponding to the image data and second feature data corresponding to the text data includes: Extracting third feature data corresponding to the image data, and extracting fourth feature data corresponding to the text data; Performing local linear embedding on the third feature data in the common subspace to obtain first feature data after dimensionality reduction; The fourth feature data is locally linearly embedded in the common subspace to obtain the second feature data after dimensionality reduction.

4. A method for training a data retrieval model according to any one of claims 1 to 3, It is characterized in that The first feature data and the second feature data are vector data; and the calculating the feature similarity between the first feature data and the corresponding second feature data includes: Calculating a first length of the first feature data and a second length of the second feature data; A first value is obtained according to the product of the first length and the second length, and a second value is obtained according to the inner product of the first characteristic data and the second characteristic data; The feature similarity is obtained according to the quotient of the second value and the first value.

5. A method for training a data retrieval model according to claim 1, It is characterized in that Determining a target loss value according to the feature similarity includes: Calculate the mean data of the feature similarities corresponding to each of the first feature data, and determine the first loss value according to the mean data; wherein the magnitude of the first loss value is negatively correlated with the magnitude of the mean data; Determining a sparse norm according to the parameters of the data retrieval model; A target loss value is determined according to the first loss value and the sparse norm.

6. A method for training a data retrieval model according to claim 5, It is characterized in that The updating of the parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model comprises: Get the preset training iteration rounds; Based on the target loss value, the parameters of the data retrieval model to be optimized are updated, and the number of times the parameters are currently updated is queried; If the number of parameter updates currently performed is equal to the number of training iteration rounds, the parameter update of the data retrieval model is stopped, and the current data retrieval model is determined as the trained data retrieval model.

7. A data retrieval method, It is characterized in that The method comprises: Get the search text data entered by the target object; Inputting the retrieval text data into a data retrieval model trained by the data retrieval model training method according to any one of claims 1 to 6, and searching and determining target data through the data retrieval model; wherein the target data includes at least one of image data or text data; The target data is fed back to the target object.

8. A training system for a data retrieval model, It is characterized in that include: An acquisition unit, used for acquiring image data and text data corresponding to the image data; A mapping unit, used to map the image data and the text data into a common subspace using a manifold learning algorithm based on the data retrieval model to be optimized, so as to obtain first feature data corresponding to the image data and second feature data corresponding to the text data; an establishing unit, configured to establish a corresponding relationship between the first feature data and the second feature data based on the corresponding relationship between the image data and the text data; A calculation unit, used for calculating the feature similarity between the first feature data and the corresponding second feature data; An updating unit is used to determine a target loss value according to the feature similarity, and to update parameters of the data retrieval model to be optimized based on the target loss value to obtain a trained data retrieval model.

9. A computer device, It is characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program executable by a processor stored therein, Features: The processor-executable program is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images