Method, apparatus and computer program product for data query

Abstract

An example embodiment according to the present disclosure provides a method, apparatus, and computer program product for data query. The method comprises the following steps: selecting a target pre-training model from a deep network pool comprising a plurality of pre-training models according to the type of input data; performing feature extraction on the input data by utilizing the selected target pre-training model so as to determine a text descriptor for the input data; and generating a query table for the query based on the text descriptor. According to the method disclosed by the invention, different target pre-training models can be selected from the deep network pool comprising a plurality of pre-training models to process (such as compression) the input data according to different input data. According to the method disclosed by the invention, a plurality of deep networks are assembled into one pool, so that the data are automatically processed to obtain the text descriptor for data retrieval, and the total memory cost of the deep network pool and a small amount of overhead of the text descriptor are obviously smaller than that of the original data, so that efficient data compression and retrieval are realized.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of data management, and more particularly, to methods, devices, and computer program products for data query. Background Art

[0002] Large Language Models (LLMs) have been booming recently. With their invention came the development of AI Generated Content (AIGC). Many new applications have been announced to generate AI data. All achievements are attributed to the progress of deep learning, especially large deep learning models. But the more advanced AI models generate artificial data (exponentially), the more data storage is needed, and the better data management is needed. Therefore, a new data storage and query technology is needed to cope with the large amount of AI data, so as to be able to effectively manage large amounts of real and synthetic data. Summary of the invention

[0003] According to an exemplary embodiment of the present disclosure, a content lossless data storage and query based on a deep network pool (DNL) is provided. Specifically, a neural network pool composed of multimodalities is proposed, which can be used for text, audio, image / video and 3D data storage and management. According to the method of the present disclosure, multiple deep networks are assembled into a pool, which can automatically process data to obtain text descriptors or identifiers for data retrieval, and the overall memory cost of DNL and the small overhead of text descriptors are significantly less than the original data. It can also achieve efficient data compression and data retrieval. All synthetic data generated by artificial intelligence models can be stored as the model itself (for example, only text descriptors are stored, etc.) instead of storing the original synthetic data. It can significantly reduce data storage and can effectively retrieve query data with less computational overhead. In the first aspect of the embodiment of the present disclosure, a data query method is provided, including: selecting a target pre-trained model from a deep network pool including multiple pre-trained models according to the type of input data; using the selected target pre-trained model, feature extraction is performed on the input data to determine the text descriptor for the input data; and based on the text descriptor, a query table for query is generated.

[0004] In a second aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes at least one processor; and instructions coupled to the at least one processor and stored thereon, the instructions causing the electronic device to perform the following actions when executed by the at least one processor: selecting a target pre-trained model from a deep network pool including a plurality of pre-trained models according to the type of input data; performing feature extraction on the input data using the selected target pre-trained model to determine a text descriptor for the input data; and generating a query table for querying based on the text descriptor.

[0005] In a third aspect of an embodiment of the present disclosure, a computer program product is provided, which is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions, which when executed cause a machine to perform the method according to the first aspect of the present disclosure.

[0006] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0008] Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present disclosure can be implemented;

[0009] Figure 2 A flowchart of a method for data processing according to some embodiments of the present disclosure is shown;

[0010] Figure 3 A schematic diagram showing a detailed architecture for data management according to some embodiments of the present disclosure;

[0011] Figure 4 shows a flow chart for model selection according to some embodiments of the present disclosure;

[0012] Figure 5A and Figure 5B A flowchart and schematic diagram showing a process for data deduplication according to some embodiments of the present disclosure are shown;

[0013] Figure 6 A schematic diagram showing a process for data query according to some embodiments of the present disclosure is shown;

[0014] Fig. 7A and 7B A schematic diagram illustrating a use case for data compression and retrieval according to some embodiments of the present disclosure; and

[0015] Figure 8 A block diagram of a computing device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] As mentioned above, Large Language Models (LLMs) have recently flourished. Along with their invention came the development of Artificial Intelligence Generated Content (AIGC). The performance of LLMs and AIGC is very impressive and has become an important part of the new era of big data. Many new applications have been announced to generate AI data. All achievements are attributed to the progress of deep learning, especially large deep learning models. But the more advanced AI models generate artificial data (exponentially), the more data storage is needed, and the better data management is needed. A new data storage and query technology is needed to cope with the large amount of AI data, so that large amounts of real and synthetic data can be managed effectively.

[0019] Traditionally, data storage is based on bit rate compression, deduplication, and other data-based processing. However, AIGC demonstrated that large deep learning models can store all data for manipulation and retrieval. For example, a large model was trained with at least 570GB of text, books, and other source data, and produced a deep learning model with 175B parameters, about 230GB model size, and a data compression ratio of about 2.48. This number can be further reduced if we consider using this large model to generate (if possible unlimited) synthetic text data. This information gain is very significant and cannot be ignored.

[0020] Figure 1A schematic diagram of an example environment 100 in which multiple embodiments of the present disclosure can be implemented is shown. The computing device 102 in the example environment 100 can be any device with computing capabilities. As a non-limiting example, the computing device 102 can be any type of fixed computing device, mobile computing device, or portable computing device, including but not limited to a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a multimedia computer, a mobile phone, etc.; in addition, all or part of the components of the computing device 102 can also be distributed in the cloud.

[0021] In the example environment 100, various data (including pictures, audio, video, text language, 3D data) and the like can be input into the computing device 102 as input data. The computing device 102 may include various software, and the management method for data storage and query according to the present disclosure may be embedded in a software layer on the physical layer of the server and the memory. The computing device 102 processes the input data (e.g., some pictures) to generate a text descriptor or identifier (ID) corresponding to the corresponding data to form a lookup table for querying. The input data may be input into a deep network pool (DNL) through an application program interface (APIs). Multiple deep networks for text, audio, image / video, and 3D data constitute the deep network pool DNL. For different input data, one or more suitable deep networks may be selected from multiple deep networks to process the input data to obtain a text descriptor or identifier for the data, and optionally, the text descriptor or identifier may be deduplicated, thereby deduplicating the input data to obtain a query table that does not contain duplicate text descriptors. When performing data retrieval, a user may input a query request through an application program interface (APIs) for data retrieval. Then, based on the comparison between the query request and the lookup table, the computing device 102 may restore the data to output uncompressed data. The entire data access and management process (including data storage and query methods) may be provided to customers as a cloud service. Many related applications may be added as new features.

[0022] In order to more clearly understand the method for obtaining data query provided by the embodiment of the present disclosure, reference will be made to Figure 2-7B To further describe the embodiments of the present disclosure.

[0023] Figure 2 A flowchart of a method for data processing according to some embodiments of the present disclosure is shown. Figure 3 A schematic diagram showing a detailed architecture for data management according to some embodiments of the present disclosure is shown.

[0024] like Figure 3As shown, the detailed architecture for data management consists of three parts: 1) model selection, 2) deduplication, and 3) candidate selection. Before selecting a model, a model library is formed, which includes many pre-trained deep learning models (PTMs). They are pre-configured and installed for use. However, based on the input data, a model selection module will be used to determine the best model combination to process the data X and obtain the corresponding data features F. The model selection module samples the input data to evaluate the performance of the candidate models and selects the best model to process the data. Then, the detailed architecture for data management also includes a deduplication module, which is used to further process the data features F and remove duplicate features through feature similarity measurement results. The compressed and deduplicated features form a lookup table, which will be used for data retrieval. For the final candidate entry selection module, its design purpose is to propose N candidates that match the user query. That is, if the user wants to retrieve pictures of traveling in Japan, for example, the model can search all possible matches and return all relevant images. The following will refer to Figure 2 and Figure 3 To describe each step and module in detail.

[0025] like Figure 2 As shown, in box 201, a target pre-trained model is selected from a deep network pool including multiple pre-trained models according to the type of input data. Figure 3 As shown, the input data may include any one of pictures, audio, video, text language, and 3D data. Or the input data may be at least a part of any one of the data, for example, a subset of multiple pictures. Figure 3 As shown, the computing device 102 may include a deep network pool 301 (also referred to as a model storage module), which may include various pre-trained models, among which a first type of multiple pre-trained models A may be used to process images, a second type of multiple pre-trained models B may be used to process audio, a third type of multiple pre-trained models C may be used to process videos, a fourth type of multiple pre-trained models D may be used to process textual languages, and a fifth type of multiple pre-trained models E may be used to process 3D data. The above-mentioned types of pre-trained models are merely illustrative, and other types of pre-trained models may be included when other types of data are included. Figure 3 As shown, the computing device 102 also includes a model selection module 302, which selects a target pre-trained model from a plurality of pre-trained models based on the type of input data.

[0026] The model library is a collection of multiple deep learning models. For given user data, it is necessary to select the best model for data compression and extract the corresponding features for storage. It is also necessary to select the corresponding model for the data for data retrieval (or data reconstruction). Mathematically speaking, the data compression model can be defined as F=G(X), where X is the user data and F is the extracted features. Data retrieval can be defined as X=H(F), where H is the inverse process of G.

[0027] The following will refer to Figure 4 Detailed description of the flow chart for a model according to some embodiments of the present disclosure.

[0028] In the model selection procedure, first, in block 401, the input data is randomly sampled to obtain a subset of the input data. The input data here can be any one of pictures, audio, video, text language, and 3D data, and the data is randomly sampled to obtain a subset of the data, such as a picture subset of multiple pictures.

[0029] Then, in box 402, based on the subset, an evaluation score for each of the multiple pre-trained models is determined. The evaluation score is defined by three criteria: 1) data fidelity, 2) compression rate, and 3) computational complexity. Mathematically, data fidelity can be defined as D=Q(Y|X), where X is the original data, Y is the compressed data, and Q is an assessment of information loss, i.e., the l2 norm. For data fidelity, there is an inverse relationship between it and the compression rate (R) and computational complexity (C). When a user prioritizes data fidelity over complexity and compression rate, more time and computing resources will be spent to maximize the encoding of the original data for compression.

[0030] The data fidelity D can be set, that is, the data fidelity can be divided into several intervals, and each model can be scored based on the data fidelity, that is, the cost estimate P(D) of each model can be obtained according to the following formula:

[0031]

[0032] Among them, α, β, γ, and λ are hyperparameters. When the data fidelity D is less than the threshold τ1, it is low-quality data compression and retrieval, and the data compression rate R(D) is prioritized. When the data fidelity D is between τ1 and τ2, it is medium-quality data compression and retrieval, considering both the data compression rate R(D) and the computational complexity C(D), which are inversely proportional and proportional to the cost estimate P(D), respectively. When the data fidelity D is greater than τ2, it is high-quality data compression and retrieval, and the computational complexity C(D) is considered, because the data compression rate has little effect on nearly lossless data compression.

[0033] That is, the process of scoring the model may include: determining the data fidelity of the subset; calculating the data compression rate and computational complexity of each module based on the data fidelity; and then determining the cost estimate of each module (i.e., scoring it) based on the calculated data compression rate and computational complexity. If the determined data fidelity is greater than τ2, then the models are scored according to the third formula in formula (1), i.e., their cost estimates are calculated.

[0034] In box 403, based on the determined evaluation score, a target pre-trained model is selected from multiple pre-trained models. When the score of each model is determined, the most suitable target pre-trained model can be selected from multiple pre-trained models for this type of input data. For the same input data (for example, the same audio or video or picture data), if the data fidelity set by the user is different, the selected most suitable pre-trained model is also different. The suitable target pre-trained model can be a model or a combination of multiple models.

[0035] Return to reference Figure 2 and Figure 3 ,like Figure 2 As shown, in box 202, the selected target pre-trained model is used to perform feature extraction on the input data to determine a text descriptor for the input data. Figure 3 As shown, after the model selection module, various types of pre-trained models suitable for various data can be selected. Figure 3 As shown, the computing device also includes a data compression module 303, which processes (e.g., compresses) the input data using the selected target pre-trained model to obtain a corresponding feature F. The feature F can be a vector, which can be a text descriptor or identifier. The storage space required for the text descriptor is very small relative to the original data.

[0036] Considering that there may be some duplicate data in the input data, for example, the same image is input multiple times, you can consider deleting the duplicate data. Return to reference Figure 2 and Figure 3 In block 203, there is an optional step of removing duplicate data from the multiple input data based on the text descriptors for the multiple input data. The text descriptors here are text descriptors for a group of related data, that is, there may be duplicate content between them.

[0037] To eliminate duplicate data, such as Figure 3As shown, the computing device also includes a data deduplication module 304, which is used to identify the latent space representation of the data, cluster similar data together, and disperse different data. Specifically, the low-dimensional projection of the data is learned to extract the eigenvectors for manifold learning.

[0038] The following will refer to Figure 5A and 5B The process used to deduplicate data is described in detail. Figure 5A As shown, in block 501 , a cosine similarity matrix of a plurality of text descriptors is determined.

[0039] For a given input data feature vector F∈R 1*512 , the feature vector F∈R 1*512 It can correspond to 512 text descriptors in the query table (each text descriptor can be a vector). Figure 5B As shown, only 1 to 7 text descriptors 510 are shown as examples. First, the cosine similarity matrix CS of the feature vector F of the input data is calculated = F × F T That is, the cosine similarity matrix CS is calculated between the 512 entries or text descriptors in the lookup table. Each element of the cosine similarity matrix CS can measure the similarity between two vectors in the feature vector. T Multiply them together to get a 512-row 512-column matrix, which is the cosine similarity matrix CS of F. Each element CS[i][j] of the cosine similarity matrix CS represents the feature vector F∈R 1*512 The cosine similarity between the i-th vector and the j-th vector in , that is, the cosine similarity between the i-th text descriptor and the j-th text descriptor in 512 text descriptors.

[0040] Then, if Figure 5A As shown, in block 502, the cosine similarity matrix is ​​converted to a Laplacian graph L = D + A, where L is a graph, and D and A are the degree matrix and the adjacency matrix, respectively. Figure 5B As shown, the Laplace graph converted from the cosine similarity matrix is ​​shown in 520. D and A are obtained by comparing the cosine score with a predefined threshold τ, that is, the cosine similarity matrix is ​​converted into a Laplace graph according to the predefined threshold τ. The threshold τ here is a predefined value, which can be used to control the connection relationship of the nodes in the Laplace graph. A smaller threshold can increase the number of connections of the node, while a larger threshold can reduce the number of connections of the node. The specific threshold to be selected needs to be judged and adjusted according to the actual application scenario.

[0041] In order to convert this matrix into a Laplacian graph, we need to first construct a graph where the nodes represent the original data (i.e., the vector or text descriptor of the input data) and the edges represent the similarity between the nodes. In the process of constructing the graph, we need to determine two key matrices: the degree matrix D and the adjacency matrix A. Degree matrix D: This is a diagonal matrix where each diagonal element represents the degree of the corresponding node, i.e., the number of edges directly connected to the node. Adjacency matrix A: This matrix represents the connection relationship between the nodes in the graph.

[0042] In the degree matrix D, each row and column represents a node, and the diagonal elements D[i][i] represent the degree of node i (or the weight of the edge between node i and itself). The off-diagonal elements D[i][j] represent the weight of the edge between node i and node j. This degree matrix D can be calculated as follows: If the input data vector F∈R 1*512 If the i-th vector and the j-th vector in F are not exactly the same (i.e., the i-th text descriptor and the j-th text descriptor in the 512 text descriptors are not exactly the same, and the cosine similarity between them is not equal to 1), then D[i][j] is equal to the inner product of the i-th vector and the j-th vector of F divided by the module length product of the two vectors (i.e., D[i][j] = F[i]·F[j] / (||F[i]||||F[j]||)). If the i-th vector and the j-th vector in F are exactly the same (i.e., the i-th text descriptor and the j-th text descriptor in the 512 text descriptors are exactly the same, and the cosine similarity between them is equal to 1), then D[i][j] is equal to 1 (because the inner product between two identical vectors is 1).

[0043] Calculate the adjacency matrix A. Each element A[i][j] of the adjacency matrix A indicates whether there is an edge connecting nodes i and j. If there is an edge connecting, then A[i][j] = 1, otherwise A[i][j] = 0. The adjacency matrix A can be calculated in the following way: If the eigenvector F∈R 1*512If the i-th vector and the j-th vector in F are not exactly the same (i.e., the i-th text descriptor and the j-th text descriptor in the 512 text descriptors are not exactly the same, and the cosine similarity between them is not equal to 1), then A[i][j] is equal to 1 (because there is an edge connecting two non-identical nodes). If the i-th vector and the j-th vector in F are exactly the same (i.e., the i-th text descriptor and the j-th text descriptor in the 512 text descriptors are exactly the same, and the cosine similarity between them is equal to 1), then A[i][j] is equal to 0 (because there is no edge connecting identical nodes). Adding the degree matrix D and the adjacency matrix A yields the Laplacian graph L=D+A. In general, by converting the cosine similarity matrix into a Laplacian graph, graph neural networks can be better used to process and analyze data and mine complex relationships between data.

[0044] Then, if Figure 5A As shown, in box 503, the Laplacian graph is processed using a graph neural network to obtain a compressed Laplacian graph. Specifically, the Laplacian graph 520 is sent to a graph neural network (GNN) 530 for learning to learn automatic data compression at a specific bit rate to obtain a compressed Laplacian graph 540. The graph neural network 530 is a neural network that specializes in processing graph structured data. It can perform complex feature learning on the nodes and edges in the graph. In the process of deduplication, the graph neural network can learn the features of each node in the graph and aggregate similar nodes together. The graph neural network learns how to encode similar nodes into similar outputs to achieve deduplication processing.

[0045] The loss function of processing the Laplacian graph using a graph neural network (GNN) can be calculated using the following formula (2):

[0046] L = -log2q([f(x) · λ] + μ) + β · d(x, g([f(x) · λ] / λ)) (2)

[0047] Where x represents the Laplacian graph of the input, “·” is point-by-point multiplication, and division “ / ” is also performed point-by-point. λ is a parameter that controls the ratio of the number of bits used for compression. f and g are the encoder and decoder parameters of the GNN. The encoder f is responsible for converting the original data into a compressed representation, and the decoder g restores the compressed representation to the original data or an approximation of the original data.

[0048] In this way, by processing the Laplacian graph 520 through the graph neural network 530, the Laplacian graph can be compressed to reduce redundant data connections while retaining data information to the maximum extent. In other words, a compressed Laplacian graph 540 can be obtained. Figure 5B As shown, the compressed Laplacian graph 540 contains all the data in the main diagonal direction, so the Laplacian graph 530 can be compressed while retaining the data in the main diagonal direction as much as possible. This process is user-controllable deduplication, and the user can define the compression rate for deduplication.

[0049] like Figure 5A As shown, in block 504, repeated text descriptors in the plurality of text descriptors are removed based on the compressed Laplacian graph to remove repeated data. Figure 5B It can be seen from the figure that by taking the 1st to 7th text descriptors 510 in the lookup table as input, through deduplication processing, it can be found that the 4th text descriptor is repeated with the 1st text descriptor, and the 3rd text descriptor is repeated with the 6th text descriptor, so the 4th and 6th text descriptors can be removed. Therefore, the repeated text descriptors in the lookup table can be removed based on the compressed or dimensionally reduced Laplacian graph to obtain a deduplicated lookup table 550, so that the data corresponding to the text descriptor can be removed.

[0050] Return to reference Figure 2 In block 204, a query table for querying is generated based on the text descriptor. Figure 3 As shown, each entry in the query table 305 corresponds to a compression feature, text descriptor or identifier of an input data. After the deduplication process of the frame 203, the deduplication query table 305 can be generated. Each entry is obtained by processing the input data through one or more most suitable trained models. Multiple entries can be obtained through the same trained model.

[0051] According to the data management method disclosed in the present invention, a process of querying data may also be included. Figure 2 and Figure 6 A diagram to illustrate the operations performed when querying data.

[0052] In block 205, at least one candidate text descriptor corresponding to at least one text descriptor in the query table is determined based on a comparison between the query request label of the user and the text descriptors of the query table. If the text descriptors in the query table are divided into different storage areas according to different data categories, if the user wants to query, for example, photos of a trip to Japan, the query request is compared with the text descriptors corresponding to the pictures, but not with the text descriptors for video or audio, etc. In one example, to achieve the goal, the cosine similarity between the query and the dictionary is calculated, where the query is the user's input label and the dictionary is the entire data set of the query table (i.e., all text descriptors of the query table for a certain type of data. For example, Figure 6As shown, a query request 610 can be converted into a query request tag. A dictionary 620 includes the entire data set of the query table. Then a cosine similarity matrix 630 between the query request tag and the dictionary is calculated. The calculated cosine similarity matrix 630 is then input into a multi-layer perceptron (MLP) network 640 to sort the cosine scores. The top k cosine scores 650 can be used to retrieve K data candidates 660. Accordingly, as Figure 3 As shown, the computing device 102 further includes a candidate selection module 306 , which selects at least one candidate text descriptor 307 corresponding to the query request from the plurality of text descriptors. The candidate selection module 306 may perform the steps described in block 205 .

[0053] like Figure 2 As shown, in block 206, the original data corresponding to the at least one candidate text descriptor is restored using a pre-trained data recovery module corresponding to the at least one candidate text descriptor. Figure 6 As shown, for K candidates 660, the pre-trained model in the data retrieval network 670 is used to restore the original data according to the feature vector. Note that the data retrieval network 670 is the same as the one obtained from Figure 3 The suitable models selected in the model pool 301 are paired one-to-one to ensure that the information gain is maximized. In other words, the data compression and data recovery operations are a pair of inverse operations that can be implemented using the same pre-trained model in the model pool.

[0054] In the data management method according to the present disclosure, a deep network pool (DNL) is used for content lossless data storage. Specifically, a neural network pool composed of multimodalities is proposed, which can be used for text, audio, image / video and 3D data storage and management. According to the method of the present disclosure, multiple deep networks are assembled into a pool, which can automatically process data to obtain text descriptors or IDs for data retrieval, and the overall memory cost of DNL and the small overhead of text descriptors are significantly smaller than the original data. It can also achieve efficient data compression as well as data retrieval. In addition, according to the method of the present disclosure, a data purification scheme embedded in DNL is also proposed to deduplicate customer data to achieve efficient data management. It detects and deletes data with duplicate content to retain information as much as possible.

[0055] Therefore, the entire data access and management can be provided to users as a cloud service. Many related applications can be added as new functions. In short, the main contributions of the data management method according to the present disclosure include: providing a complete deep learning-based framework that can automatically handle all data types for data management, especially efficient data storage and retrieval. Then, a deduplicator that can process data features (text descriptors) is also proposed, so that redundant data can be removed for information abstraction. Finally, based on the feature sorting scheme, a novel data retrieval method is proposed that can generate multiple candidates that match the text descriptors of practical applications.

[0056] The following will refer to Fig. 7A and 7B An example of a practical use scenario of the data management method according to the present disclosure is described below.

[0057] In the present disclosure, a new framework for data compression through deep network pooling (DNL) is proposed, which can be used for data management and retrieval, and can effectively remove redundant data and achieve high compression rates. Given different types of data, including text, images, audio, and video, the input data is projected into the latent space learned by DNL for data compression. The DNL can 1) remove redundant data based on feature similarity (redundant images), and 2) retrieve relevant data based on user queries. For example, a user can query for "Hart". Then, through the processing of the deep network pool, videos, audio, and pictures related to Hart can be output.

[0058] To illustrate the data compression ratio using DNL compared to other compression techniques, take image compression as an example. Figure 7B In the figure, it can be seen that using DNL for data compression is particularly beneficial when using more and higher resolution images. In traditional methods, the memory usage of 2K or 4K images is linearly proportional to the number of images. However, when using DNL for image compression, the slope of the line is small because the main data storage is the DNL model file (about 200MB), and the rest of the data is the feature vector for each input data (about 2KB per data sample).

[0059] The data management method according to the present disclosure is a simple and efficient data management technology. This may be a huge advantage for artificial intelligence synthetic data processing. All synthetic data generated by the artificial intelligence model can be stored as the model itself instead, without storing the original synthetic data. It can significantly reduce data storage and can effectively implement retrieval query data with less computational overhead. In the long run, the proposed deep network pool (DNL) can be deployed in the cloud, so cloud data storage services as well as hardware deployment at the edge can be provided, which uses GNN for data deduplication and MLP networks for data candidate recommendations. The method and architecture according to the present disclosure can be used in data management related applications.

[0060] Example Device

[0061] Figure 8 800 is a schematic block diagram of an example device 800 that can be used to implement an embodiment of the present disclosure. Figure 1 The computing device 102 of the present invention. As shown in the figure, the device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0062] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0063] The processing unit 801 performs the various methods and processes described above, such as any one of the processes 200, 400, 500, and 600. For example, in some embodiments, any one of the processes 200, 400, 500, and 600 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more steps of any one of the processes 200, 400, 500, and 600 described above may be performed. Alternatively, in other embodiments, the CPU 801 may be configured to perform any one of the processes 200, 400, 500, and 600 by any other suitable means (e.g., by means of firmware).

[0064] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0065] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the workflow diagram and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0066] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0067] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0068] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A data query method, comprising: Selecting a target pre-trained model from a deep network pool including multiple pre-trained models according to the type of input data; Using the selected target pre-trained model, perform feature extraction on the input data to determine a text descriptor for the input data; as well as Based on the text descriptor, a lookup table for querying is generated.

2. The data query method according to claim 1, further comprising: Based on a plurality of text descriptors for a plurality of input data, duplicate data in the plurality of input data are removed to obtain the deduplicated query table.

3. The data query method according to claim 1, further comprising: Determining at least one candidate text descriptor corresponding to at least one text descriptor in the query table based on a comparison between the query request tag of the user and the text descriptors in the query table; as well as restoring original data corresponding to the at least one candidate text descriptor using a pre-trained data recovery module corresponding to the at least one candidate text descriptor, The pre-training data recovery module corresponds to the target pre-training model for feature extraction and for the at least one candidate text descriptor. 4 . The data query method according to claim 1 , wherein the input data comprises at least one of the following categories: image data, video data, voice data, natural language text, and three-dimensional image data.

5. The data query method according to claim 1, wherein selecting a target trained model from a deep network pool including a plurality of pre-trained models according to the type of input data comprises: Randomly sampling the input data to obtain a subset of the input data; Based on the subset, determining an evaluation score for each module in the plurality of pre-trained models; as well as Based on the evaluation scores, the target pre-trained model is selected from the multiple pre-trained models.

6. The data query method according to claim 5, wherein determining the evaluation score of each module in the plurality of pre-trained models based on the subset comprises: determining data fidelity of the subset; Based on the data fidelity, determining a data compression rate and a computational complexity for each module in the plurality of pre-trained models; as well as Determine an evaluation score for each module based on at least one of the determined data compression rate and computational complexity, The evaluation score determined for the same pre-trained model changes as the data fidelity changes.

7. The data query method according to claim 2, wherein based on a plurality of text descriptors for a plurality of input data, removing duplicate data from the plurality of input data to obtain the deduplicated query table comprises: determining a cosine similarity matrix of the plurality of text descriptors; Converting the cosine similarity matrix into a Laplacian graph; Processing the Laplacian graph using a graph neural network to obtain a compressed Laplacian graph; and Based on the compressed Laplacian graph, duplicate text descriptors among the plurality of text descriptors are removed to remove duplicate data.

8. The data query method according to claim 3, wherein based on the comparison between the query request tag of the user and the text descriptors in the query table, determining at least one candidate text descriptor corresponding to at least one text descriptor in the query table comprises: determining a plurality of similarities between the query request tag and the text descriptors in the query table; as well as The multiple similarities are sorted to determine at least one corresponding text descriptor with a higher sorting level as at least one candidate text descriptor.

9. The data query method according to claim 8, wherein sorting the plurality of similarities to determine at least one corresponding text descriptor with a higher sorting as at least one candidate text descriptor comprises: The pre-similarity matrix composed of the multiple similarities is input into a multi-layer perceptron network to sort the multiple similarities.

10. An electronic device, comprising: processor; as well as a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform actions, the actions comprising: Selecting a target pre-trained model from a deep network pool including multiple pre-trained models according to the type of input data; Using the selected target pre-trained model, performing feature extraction on the input data to determine a text descriptor for the input data; and Based on the text descriptor, a lookup table for querying is generated.

11. The electronic device according to claim 10, wherein the actions further comprise: Based on a plurality of text descriptors for a plurality of input data, duplicate data in the plurality of input data are removed to obtain the deduplicated query table.

12. The electronic device according to claim 10, wherein the actions further comprise: Determining at least one candidate text descriptor corresponding to at least one text descriptor in the query table based on a comparison between the query request tag of the user and the text descriptors in the query table; as well as restoring original data corresponding to the at least one candidate text descriptor using a pre-trained data recovery module corresponding to the at least one candidate text descriptor, The pre-training data recovery module corresponds to the target pre-training model for feature extraction and for the at least one candidate text descriptor. 13 . The electronic device of claim 10 , wherein the input data comprises at least one of the following categories: image data, video data, voice data, natural language text, and three-dimensional image data.

14. The electronic device of claim 10, wherein selecting a target trained model from a deep network pool including a plurality of pre-trained models according to a type of input data comprises: Randomly sampling the input data to obtain a subset of the input data; Based on the subset, determining an evaluation score for each module in the plurality of pre-trained models; as well as Based on the evaluation scores, the target pre-trained model is selected from the multiple pre-trained models.

15. The electronic device according to claim 14, wherein determining the evaluation score of each module in the plurality of pre-trained models based on the subset comprises: determining data fidelity of the subset; Based on the data fidelity, determining a data compression rate and a computational complexity for each module in the plurality of pre-trained models; as well as Determine an evaluation score for each module based on at least one of the determined data compression rate and computational complexity, The evaluation score determined for the same pre-trained model changes as the data fidelity changes.

16. The electronic device according to claim 11, wherein based on a plurality of text descriptors for a plurality of input data, removing duplicate data from the plurality of input data to obtain the deduplicated lookup table comprises: determining a cosine similarity matrix of the plurality of text descriptors; Converting the cosine similarity matrix into a Laplacian graph; Processing the Laplacian graph using a graph neural network to obtain a compressed Laplacian graph; and Based on the compressed Laplacian graph, duplicate text descriptors among the plurality of text descriptors are removed to remove duplicate data.

17. The electronic device of claim 12, wherein based on a comparison between the query request tag of the user and the text descriptors in the query table, determining at least one candidate text descriptor corresponding to at least one text descriptor in the query table comprises: determining a plurality of similarities between the query request tag and the text descriptors in the query table; as well as The multiple similarities are sorted to determine at least one corresponding text descriptor with a higher sorting level as at least one candidate text descriptor.

18. The electronic device according to claim 17, wherein sorting the plurality of similarities to determine at least one corresponding text descriptor with a higher sorting level as at least one candidate text descriptor comprises: The pre-similarity matrix composed of the multiple similarities is input into a multi-layer perceptron network to sort the multiple similarities.

19. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions that when executed cause a machine to perform actions comprising: Selecting a target pre-trained model from a deep network pool including multiple pre-trained models according to the type of input data; Using the selected target pre-trained model, perform feature extraction on the input data to determine a text descriptor for the input data; as well as Based on the text descriptor, a lookup table for querying is generated.

20. The computer program product of claim 19, wherein the actions further comprise: Based on a plurality of text descriptors for a plurality of input data, duplicate data in the plurality of input data are removed to obtain the deduplicated query table.

Images