Data processing method and device

Abstract

The invention provides a data processing method and device which are used for improving retrieval efficiency. The computing device generates an initial codebook; wherein the structure of the initial codebook is a d * m matrix, the initial codebook comprises a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d * p matrix, m is greater than p, p is greater than 1, and d is greater than 1; according to each initial sub-codebook, generating a codebook used for carrying out compressed encoding on object data of the candidate objects stored in the original base library, and code words after compressed encoding; wherein the structure of the code word is a matrix comprising a plurality of code word blocks, each code word block comprises a first element, other positions except the plurality of code word blocks in the matrix are second elements, and each column in the matrix corresponds to one code word block. The codebook and the codeword are obtained by compressing and coding the object data in the original base library, and the codeword blocks in the codeword are dense matrixes, so that the operation and memory access speed of the computing equipment can be greatly improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202310910436.3, and the original application date is July 21, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data processing method and device. Background Art

[0003] With the continuous development of computer technology, a huge amount of information data has been generated. Large database retrieval is to retrieve the required data from the massive amount of information data. The process of large database retrieval is to obtain the object data of the query object requested for query, search for information data similar to the object data of the query object from the information database, and return the found information data as the retrieval result.

[0004] A typical problem of large database retrieval is that the amount of data stored in the database is very large. Taking the face feature database as an example, the face feature database can store the face features of a large number of sample objects, and a sample object corresponds to a face feature with a higher dimension. For example, the face feature of a sample object can be 256-dimensional, and the amount of data stored in the face feature database can reach billions.

[0005] Large database retrieval is a data-intensive task that generally runs on the accelerator card of the computing device. When executing the retrieval task, the accelerator card needs to load all the information data in the database into the video memory. Due to the large amount of data in the database, the retrieval efficiency is low. Summary of the invention

[0006] The present application provides a data processing method and device to improve retrieval efficiency.

[0007] In the first aspect, the embodiment of the present application provides a data processing method, which can be specifically executed by a computing device or by a chip inside the computing device, or by a processor in the computing device (exemplarily, the processor can be a central processing unit, an accelerator card, etc.). Taking the execution of a computing device as an example, the method includes: the computing device generates an initial codebook; wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes multiple initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, and m is greater than p, p is greater than 1, and d is greater than 1; the computing device generates a codebook for compressing and encoding the object data of the candidate object stored in the original base library according to each initial sub-codebook in the initial codebook, and a compressed and encoded codeword; wherein the structure of the codeword is a matrix including multiple codeword blocks, the codeword block includes a first element, and other positions in the matrix except for multiple codeword blocks are second elements, and each column in the matrix corresponds to a codeword block.

[0008] Through the above design, the computing device generates a codebook and codeword corresponding to the object data corresponding to multiple candidates stored in the original base library according to the initial codebook. Since the codeword is obtained by compressing and encoding the object data of the candidate object stored in the original base library, the data volume of the codeword is much smaller than the data volume of the object data corresponding to multiple candidates stored in the original base library. When searching, the codeword and codebook obtained by compression coding can be searched to improve the search efficiency. In addition, the structure of the codeword is a matrix including multiple codeword blocks. For example, non-0 elements can be concentrated in the codeword blocks, and the codeword block structure is a dense matrix. In this way, when the computing device reads the codeword and performs calculation processing, the calculation speed of the dense matrix is ​​much higher than other calculation forms; and each column in the codeword matrix corresponds to a codeword block, that is, each column in the codeword matrix only includes an area with non-0 elements. Therefore, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device.

[0009] In one possible design, when generating the initial codebook, the computing device selects object data of m sample objects from the object data of candidate objects stored in the original base library to form multiple sample sets, where each sample set includes object data of p sample objects; and generates each initial sub-codebook included in the initial codebook according to the object data of the sample objects included in each sample set.

[0010] Through the above design, the computing device generates an initial codebook based on the object data of the sample object selected from the original base library. This initial codebook generation method makes the obtained initial codebook more accurate.

[0011] In one possible design, when determining the initial sub-codebook corresponding to each sample set, the computing device initializes, for any sample set, a matrix composed of object data of sample objects included in the sample set by an orthogonal basis to generate the initial sub-codebook corresponding to the sample set.

[0012] Through the above design, the initial sub-codebook obtained by performing an orthogonal basis initialization on the matrix composed of the object data of the sample object can better meet the requirements of the codebook and improve the accuracy of the generated codebook.

[0013] In one possible design, the computing device determines, according to the subspace distance between the object data of each sample object and each initial subcodebook, a sample object data set corresponding to each initial subcodebook; for any initial subcodebook, generates, according to the sample object data set corresponding to the initial subcodebook, an updated subcodebook and a corresponding codeword block corresponding to the initial subcodebook; generates a codebook according to the updated subcodebook corresponding to each initial subcodebook; and generates a codeword according to the codeword block corresponding to each initial subcodebook.

[0014] Through the above design, in the process of determining the codebook and the codeword, the computing device first determines the sample object data set corresponding to the initial sub-codebook according to the subspace distance between the object data included in the sample set and the initial sub-codebook, so that the object data with high similarity to the initial sub-codebook can be used as a sample object data set, and a new sub-codebook and corresponding codeword blocks are generated by processing the sample object set, so that the object data of the candidate objects corresponding to such a sub-codebook and the codeword block are data with high similarity, so that the codebook and codeword are generated based on this method, and the retrieval efficiency can be improved in the subsequent retrieval process.

[0015] In one possible design, the computing device determines, for object data of any sample object, an initial sub-codebook corresponding to the object data of the sample object from multiple initial sub-codebooks according to a subspace distance between the object data of the sample object and each initial sub-codebook; and determines, according to a corresponding relationship between the object data of the sample object and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook, wherein the sample object data set includes the object data of the sample object corresponding to the initial sub-codebook.

[0016] Through the above design, the sample object data set corresponding to each initial sub-codebook determined by the computing device includes object data of relatively similar candidate objects. Codebooks and codewords are generated in this way, which can improve retrieval efficiency in subsequent retrieval processes.

[0017] In a possible design, when generating a codeword according to the codeword blocks corresponding to each initial sub-codebook, the computing device divides the codeword blocks corresponding to each initial sub-codebook into reference codewords; quantizes the reference codewords, and adjusts the elements in the quantized reference codewords to generate the codewords. Exemplarily, the computing device adjusts the elements in the quantized reference codewords so that the structure of the adjusted codewords is a matrix including a plurality of codeword blocks, and each column in the matrix corresponds to a codeword block.

[0018] Through the above design, the computing device quantizes the reference codewords composed of each codeword block to obtain a higher compression rate; in addition, by adjusting the elements in the reference codeword, each column in the adjusted codeword corresponds to a codeword block, and the adjusted codeword is a dense matrix, which can improve the calculation speed when the computing device calls the codeword.

[0019] In one possible design, the structure of the codeword is a block diagonal matrix.

[0020] In one possible design, after receiving a query request, the computing device determines a query vector corresponding to the query request; and retrieves a target object matching the query request from multiple candidate objects stored in an original database based on the query vector, codeword, and codebook.

[0021] Through the above design, after obtaining the query vector, the computing device searches according to the codeword obtained after compressing and encoding the original object data, and the codeword structure is specially designed so that the codeword structure includes a matrix of multiple blocks, and each column in the matrix corresponds to a block. The codeword structure is a dense matrix. Since the operation speed of the dense matrix is ​​much higher than other operation forms, the computing device can significantly improve the operation speed based on the dense codeword matrix during the retrieval process, thereby improving the retrieval efficiency; in addition, since each column in the codeword matrix corresponds to a block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device, thereby further improving the retrieval efficiency.

[0022] In a possible design, each codeword in the codeword is divided into blocks and stored in different storage areas.

[0023] Through the above design, each codeword block in the codeword structure represents data with high similarity. By storing each codeword block in a different storage area, it is possible to centrally store data with high similarity, which is convenient for data retrieval in subsequent processes.

[0024] In one possible design, the computing device determines a target storage area according to a query vector and a codebook; and the computing device retrieves a target object matching the query request from multiple candidate objects according to codeword blocks stored in the target storage area.

[0025] Through the above design, when executing the retrieval process, the computing device first determines the target storage area for storing the codeword blocks according to the query vector and the codebook, and then further searches for the target object matching the query request in the target storage area. Compared with the method of searching in all codeword blocks, this query method can significantly improve the retrieval efficiency.

[0026] In a possible design, the codebook includes multiple sub-codebooks, each sub-codebook corresponds to a codeword block; when determining the target storage area, the computing device determines, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, where the first similarity information is used to characterize the degree of similarity between the query vector and the sub-codebook; the computing device determines a target sub-codebook from the multiple sub-codebooks included in the codebook according to the first similarity information between the query vector and each sub-codebook in the codebook; the computing device determines a target codeword block corresponding to the target sub-codebook according to a correspondence between the sub-codebook and the codeword block, and uses a storage area storing the target codeword block as the target storage area.

[0027] Through the above design, when the computing device determines the target storage area from multiple storage areas, it calculates the first similarity information between the query vector and each sub-codebook in the codebook to determine the target sub-codebook with a higher similarity to the query vector, and then based on the correspondence between the sub-codebook and the codeword block, it can determine the storage area storing the codeword block corresponding to the target sub-codebook as the target storage area. Based on this design, it can be determined from multiple storage areas that the codeword block stored in the target storage area is more similar to the query vector, thereby simplifying the retrieval process.

[0028] In one possible design, a computing device determines a product of a query vector and a codebook to obtain an intermediate vector; divides the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook; and determines first similarity information between the query vector and each sub-codebook in the codebook according to the sub-intermediate vectors corresponding to each sub-codebook.

[0029] Through the above design, a solution is provided for accurately determining the first similarity information between the query vector and each sub-codebook in the codebook. Based on this design, the first similarity information between the query vector and each sub-codebook in the codebook can be accurately determined.

[0030] In one possible design, the computing device determines, based on the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, where the second similarity information is used to characterize the degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area; and then the computing device determines the target object from the candidate objects corresponding to the target sub-codebook based on the second similarity information.

[0031] Through the above design, after determining the target storage area, the computing device calculates the second similarity information between the query vector and the codeword blocks stored in the target storage area, and retrieves the target object matching the query vector from the candidate objects corresponding to the target storage area; based on the second similarity information, the target object can be accurately retrieved.

[0032] In a second aspect, an embodiment of the present application provides a data processing device, which has the function of implementing the method in the first aspect or any possible implementation of the first aspect, and the device can be a computing device or a processor included in the computing device. The functions of the above-mentioned data processing device can be implemented by hardware, or by hardware executing corresponding software, and the hardware or software includes one or more modules or units or means corresponding to the above-mentioned functions.

[0033] In a possible implementation, in a possible design, the structure of the device includes an initialization unit and a processing unit. These units can perform the corresponding functions of the computing device in the method example of the first aspect above, and refer to the detailed description in the method example for details, which will not be repeated here.

[0034] In another possible implementation, the structure of the device includes a processor and may also include a memory. The processor is coupled to the memory and may be used to execute computer program instructions stored in the memory so that the device performs the method in the first aspect or any possible implementation of the first aspect. Optionally, the device further includes a communication interface, and the processor is coupled to the communication interface. When the device is a computing device, the communication interface may be a transceiver or an input / output interface.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0036] In a fourth aspect, an embodiment of the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0037] The technical effects that can be achieved in any of the second to fourth aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of an image retrieval scenario provided in an embodiment of the present application;

[0039] Figure 2A schematic diagram of a bottom storage structure provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a commodity recommendation scenario provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0042] Figure 5 A schematic diagram of a process for establishing a compressed database provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a sample set division method provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of determining a subspace distance provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of performing PCA dimensionality reduction processing on a matrix provided in an embodiment of the present application;

[0046] Fig. 9 A codeword structure provided in an embodiment of the present application;

[0047] Fig.10 Another codeword structure provided in an embodiment of the present application;

[0048] Fig.11 A schematic diagram of the relationship between an original base library, a codebook, and codewords provided in an embodiment of the present application;

[0049] Fig.12 A schematic diagram of a partition storage provided in an embodiment of the present application;

[0050] Fig.13 A search flow chart provided for an embodiment of the present application;

[0051] Fig.14 A schematic diagram of determining a target storage area provided in an embodiment of the present application;

[0052] Fig.15 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0053] Fig.16 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0054] Fig.17 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] First, some terms in the present application are explained so as to be understood by those skilled in the art.

[0056] 1. Database compression: Database compression refers to compressing the data stored in the database to reduce the storage space occupied by the data and improve the performance and efficiency of the database. The main purpose of database compression is to optimize the utilization of storage resources, improve query performance and reduce storage costs. By compressing the database, the storage space occupied can be reduced, thereby reducing storage costs. In addition, when reading and writing operations on the compressed database, the efficiency of data reading and writing operations can be improved, and the speed of query and transaction processing can be accelerated due to the reduction in the amount of data.

[0057] 2. Vector retrieval technology: In a given vector data set, vectors similar to the query vector are retrieved according to a certain measurement method.

[0058] 3. K-means clustering algorithm is an iterative clustering analysis algorithm. Specifically, given the number of categories k, the entire data set is clustered. The objective function is to minimize the sum of the distances from all samples to the class center. The objective function is iteratively calculated and optimized to obtain k class centers and the category to which each sample belongs.

[0059] 4. Principal component analysis (PCA) dimensionality reduction is a common data analysis method. It uses orthogonal transformation to convert data represented by possibly correlated variables into data represented by a few linearly unrelated variables. Linearly unrelated variables are called principal components. The number of principal components is usually smaller than the number of original variables, so principal component analysis is often used to reduce the dimensionality of high-dimensional data and extract the main characteristic components of the data. Common methods of PCA dimensionality reduction include covariance matrix method and eigenvalue decomposition method; among them, the covariance matrix method uses the covariance matrix to solve the eigenvalue eigenvector and then obtain the reduced dimensionality data; the eigenvalue decomposition method can be solved using the singular value decomposition method to obtain the reduced dimensionality data.

[0060] 5. The base database is a database or data structure used to store extracted feature vectors. It is usually used in tasks such as face recognition, image retrieval, and speech recognition for similarity matching or comparison. When building the base database, it is necessary to extract features for each sample object and store the extracted features in the base database. Exemplarily, the base database can use a matrix structure to store data, and each column in the matrix represents a feature vector of a sample object.

[0061] Figure 1A schematic diagram of an image retrieval scenario is shown. First, a picture set including a large number of sample pictures is obtained, and the sample pictures are pictures including sample objects; object recognition is performed on each sample picture in the picture set, and the object data (also called object features) of the sample object in each sample picture can be extracted through a feature extraction model, and the extracted object data of each sample object is stored to form a base library. Figure 2 The schematic diagram of the base database structure is shown, and the base database structure can be a d*n matrix, where n represents the object data of n sample objects stored in the base database, and d represents the dimension of the object data of each sample object. After receiving the query request, the query request may include the image to be queried, and the object features of the query object in the query image to be queried are extracted to obtain the query vector; according to the query vector, the base database is searched to retrieve the object data that meets the similarity requirement with the query vector; it is determined from which target images these object data that meet the similarity requirement are extracted, and the determined target images are returned as the search results. For example, in an Internet application, according to the product image to be queried input by the user, the image containing the product with a similar appearance to the product image input by the user is retrieved from the base database.

[0062] Figure 3 A schematic diagram of a product recommendation scenario is shown. First, a large number of product samples are obtained, and the product features of each product are extracted through a feature extraction model (for example, feature extraction can be performed on pictures including product appearance, product description information, etc.), and the extracted product features of each product are stored to form a base library. When recommending products to users, the query request obtained includes a user identifier, and the user features of the user are extracted through a feature extraction model (in implementation, feature extraction can be performed on the user's historical behavior data, user portraits, etc. to extract user features), and the user features are used as query vectors; according to the query vector, a search is performed in the base library to retrieve product features that meet the similarity requirements with the query vector; it is determined from which target products these product features that meet the similarity requirements are extracted, and the determined target products are returned as search results.

[0063] In implementation, a query request can be received by a computing device, and then the computing device retrieves object data that meets the requirements of similarity with the query vector from the base database according to the query vector corresponding to the query request, and uses the candidate objects of the object data that meet the requirements of similarity with the query vector as the retrieval results. Since retrieval from the base database is a data-intensive task, it can be run on the accelerator card of the computing device. When executing the retrieval task, the computing device loads the object data in the base database into the video memory, so that the object data that matches the query vector can be found from the large amount of object data in the base database, thereby obtaining the retrieval results. When the amount of object data stored in the base database is large, retrieving from all the object data stored in the database will result in low retrieval efficiency.

[0064] Based on the above problems, an embodiment of the present application provides a data processing method, which compresses and encodes the object data of multiple candidate objects through a code book to obtain codewords. The compressed and encoded codewords can be stored in a base database. When performing a retrieval task, the retrieval results can be quickly obtained based on the query vector, the code book and the codewords.

[0065] The following is an introduction to the system architecture applicable to the embodiments of the present application. Figure 4 As shown, the system includes a terminal device 40, a computing device 41 and a storage device 42. The computing device 41 can receive a query request sent by the terminal device 40, and the computing device 41 determines a query vector corresponding to the query request; the storage device 42 stores codewords obtained by compressing and encoding object data of multiple candidate objects, and a codebook corresponding to the codewords.

[0066] Exemplarily, the query request may include a picture to be queried, and the computing device 41 may extract object data of the query object included in the picture to be queried, and use the extracted object data of the query object as a query vector.

[0067] The computing device 41 obtains the codebook and the codeword from the storage device 42, and retrieves the target object matching the query request from multiple candidate objects according to the query vector, the codeword and the codebook. In implementation, when the computing device 41 performs a retrieval task, the retrieval task may be performed by a processor in the computing device 41 or by an accelerator card; if the retrieval task is performed by the processor, the processor loads the codebook and the codeword from the storage device 42 into the memory; if the retrieval task is performed by the accelerator card, the processor loads the codebook and the codeword from the storage device 42 into the video memory.

[0068] After obtaining the query vector, the computing device 41 searches from the compressed and encoded base library. The base library stores codewords obtained after compressing and encoding the original object data, and the codeword structure is specially designed so that the codeword structure includes a matrix of multiple blocks, and each column in the matrix corresponds to a codeword block. Each codeword block obtained in the codeword structure is a dense matrix. Since the operation speed of the dense matrix is ​​much higher than other operation forms, the processor or acceleration card of the computing device 41 can significantly improve the operation speed during the retrieval process based on the dense codeword matrix, thereby improving the retrieval efficiency. In addition, since each column in the codeword matrix corresponds to a block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed, thereby further improving the retrieval efficiency.

[0069] It should be noted that Figure 4 The system architecture shown is only an example. The embodiment of the present application does not limit the specific form of the system architecture. The system architecture applicable to the embodiment of the present application may have more Figure 4 More or fewer devices than those shown.

[0070] Based on the above Figure 4 The system architecture shown in the figure, the embodiment of the present application provides a data processing solution, the implementation process of this solution can be divided into two stages, the first stage is the database construction stage, and the second stage is the retrieval stage. In the database construction stage, the object data based on multiple candidate objects are compressed and encoded to obtain a compressed base database; in the retrieval stage, according to the query vector corresponding to the query request, the target object matching the query request is retrieved from the compressed base database. The following introduces these two stages respectively.

[0071] 1. Database construction stage.

[0072] In the database building stage, the database building process can be executed by a computing device; wherein the computing device executing the database building process and Figure 4 The computing devices executing the retrieval process may be different computing devices.

[0073] In the database building stage of the present application embodiment, a large number of samples can be collected; the computing device can extract object data of multiple candidate objects from the samples. For example, the sample can be a picture, and the object data of the candidate object can be feature data of the object in the picture, where the object in the picture includes but is not limited to people, goods, buildings, animals, and plants.

[0074] The computing device can construct an original base database based on the object data of multiple candidate objects extracted; the structure of the original base database can be in the form of a matrix, and each column in the matrix represents the object data of a candidate object. Since the number of candidate objects collected in the database construction stage is large, and the object data of each candidate object is a high-dimensional feature, the amount of data in the original base database will be large.

[0075] In implementation, the computing device may extract object data of the candidate object from each collected sample based on the trained feature extraction model. The embodiment of the present application does not limit the type of feature extraction model used.

[0076] The following is combined with Figure 5 The following describes the database building process performed by the computing device in the embodiment of the present application.

[0077] like Figure 5 The schematic diagram of the process of establishing a compressed database shown in the figure may specifically include the following steps:

[0078] Step 501: The computing device generates an initial codebook;

[0079] The initial codebook includes multiple initial sub-codebooks; optionally, the structure of the initial sub-codebook is a d*p matrix, and the structure of the initial codebook composed of the multiple initial sub-codebooks is a d*m matrix, and m is greater than p, p is greater than 1, and d is greater than 1. Optionally, the value of m can be an integer multiple of p, and the value of p can be a preset subspace size.

[0080] When a computing device generates an initial codebook, it can be implemented in a variety of different ways:

[0081] One possible implementation is to select object data of m sample objects from the object data of candidate objects stored in the original base library to form multiple sample sets; and generate an initial sub-codebook according to the object data of the sample objects included in each sample set.

[0082] In implementation, the computing device may select object data of m sample objects from the original base library, and group the object data of the m sample objects into multiple sample sets according to the size of p, and each sample set includes object data of p sample objects. The value of m is the number of columns corresponding to the codebook in the embodiment of the present application, and the value of m is much smaller than the number of candidate objects stored in the original base library.

[0083] Assuming that the value of m is 9 and the preset subspace size is 3, the object data of 9 sample objects are selected from the original base library and the object data of the 9 sample objects are divided into three groups. The grouping method can be as follows: Figure 6As shown, the object data of sample objects m1, m2, and m3 are grouped into sample set 1, the object data of sample objects m4, m5, and m6 are grouped into sample set 2, and the object data of sample objects m7, m8, and m9 are grouped into sample set 3.

[0084] In implementation, the computing device may randomly select object data of m sample objects from the original base library. Exemplarily, the original base library stores uncompressed object data of multiple candidate objects, such as the data structure stored in the original base library is as follows: Figure 2 As shown, the computing device can randomly select m sample objects from n candidate objects, where the value of m is much smaller than the value of n.

[0085] When the computing device generates the initial sub-codebook, for any sample set, the computing device may perform orthogonal basis initialization on a matrix composed of object data of sample objects included in the sample set to obtain an initial sub-codebook; wherein the number of columns of the obtained initial sub-codebook is p.

[0086] In implementation, for each sample set, the object data of the sample objects included in the sample set are formed into a matrix, and the formed matrix is ​​initialized with an orthogonal basis, so as to obtain an initial sub-codebook corresponding to the sample set. Figure 6 As shown, the object data of the sample objects m1, m2, and m3 included in the sample set 1 are initialized with an orthogonal basis to obtain the initial subcodebook corresponding to the sample set 1; the object data of the sample objects m4, m5, and m6 included in the sample set 2 are initialized with an orthogonal basis to obtain the initial subcodebook corresponding to the sample set 2; the object data of the sample objects m7, m8, and m9 included in the sample set 3 are initialized with an orthogonal basis to obtain the initial subcodebook corresponding to the sample set 3.

[0087] When the computing device of the embodiment of the present application performs orthogonal basis initialization on the matrix composed of the object data included in the sample set, a plurality of different methods may be used to obtain the orthogonal basis of the matrix, and the obtained orthogonal basis may be used as the initialization matrix of the matrix, and the obtained initialization matrix may be used as the initial sub-codebook corresponding to the sample set.

[0088] The orthogonal basis can be initialized by QR decomposition; the principle of QR decomposition is that for a given matrix A, it can be decomposed into the product of an orthogonal matrix Q and an upper triangular matrix R, that is, A = QR; the orthogonal matrix Q can be selected as an orthogonal basis to initialize the matrix. Alternatively, the orthogonal basis can be initialized by diagonalization; the principle of diagonalization is that for some special types of matrices (such as symmetric matrices or positive definite matrices), an orthogonal basis can be obtained by diagonalization, and the eigenvectors of the symmetric matrix form an orthogonal basis to initialize the matrix.

[0089] When the computing device generates the initial codebook, another possible implementation method is to pre-set multiple sample data (the sample data may not be the object data in the original base library), divide the multiple sample data into sample sets, and randomly initialize the matrix composed of the sample data in the sample set to obtain each initial sub-codebook; each obtained initial sub-codebook is composed of the initial codebook.

[0090] Step 502: The computing device generates a codebook for compressing and encoding the object data of the candidate object stored in the original base library and a codeword after the compression encoding according to each initial sub-codebook in the initial codebook.

[0091] In a possible implementation manner, the computing device determines, according to the subspace distance between the object data of each sample object and each initial sub-codebook, a sample object data set corresponding to each initial sub-codebook; for any initial sub-codebook, according to the sample object data set corresponding to the initial sub-codebook, an updated sub-codebook and a corresponding codeword block corresponding to the initial sub-codebook are generated; according to the updated sub-codebook corresponding to each initial sub-codebook, a codebook is generated; and, according to the codeword block corresponding to each initial sub-codebook, a codeword is generated.

[0092] The following describes a process in which the computing device determines, according to the subspace distance between the object data included in each sample set and each initial subcodebook, a sample object data set corresponding to each initial subcodebook.

[0093] Step 1a: The computing device determines the subspace distance between the object data of each sample object in each sample set and each initial sub-codebook;

[0094] Here, we take the example of selecting object data of m sample objects from the original base database and dividing the sample sets; Figure 7 The object data of m candidate samples and m / p initial subcodebooks are shown, where the dimension of the initial subcodebook is d*p. The subspace distance between the object data of each sample object and each initial subcodebook is determined respectively. An optional way to determine the subspace distance is to determine the subspace distance according to the product between the vector (with the dimension of 1*d) composed of the object data of a sample object and an initial subcodebook, for the object data of a sample object and an initial subcodebook; Figure 7 As shown, the product of a 1*d vector composed of object data of a sample object and the d*p initial sub-codebook is a 1*p vector. The computing device can calculate the L2 norm of the 1*p vector to obtain the subspace distance.

[0095] Step 2a: The computing device determines, from a plurality of initial sub-codebooks, an initial sub-codebook corresponding to the object data of the sample object according to a subspace distance between the object data of the sample object and each initial sub-codebook;

[0096] For object data of any sample object, a subspace distance between the object data of the sample object and each initial subcodebook is determined in step 1; in a possible implementation manner, the initial subcodebook with the smallest subspace distance is bound to the sample object, thereby obtaining an initial subcodebook corresponding to the object data of the sample object.

[0097] Step 3a: The computing device determines a sample object data set corresponding to each initial sub-codebook according to the correspondence between the object data of the sample object and the initial sub-codebook, wherein the sample object data set includes the object data of the sample object corresponding to the initial sub-codebook.

[0098] Based on the initial sub-codebooks corresponding to the object data of each sample object obtained in step 2a, a correspondence between the object data of the sample object and the initial sub-codebook can be generated. Based on the generated correspondence between the object data of the sample object and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook can be obtained.

[0099] Exemplarily, assuming that the value of m is 9, the number of initial sub-codebooks obtained based on the object data of 9 sample objects is 3; if the correspondence between the object data of the sample objects generated based on step 2a and the initial sub-codebook is as shown in Table 1, the sample object data set corresponding to the initial sub-codebook 1 includes the object data of sample object 1, the object data of sample object 3, and the object data of sample object 7; the sample object data set corresponding to the initial sub-codebook 2 includes the object data of sample object 2, the object data of sample object 5, and the object data of sample object 9; the sample object data set corresponding to the initial sub-codebook 3 includes the object data of sample object 4, the object data of sample object 6, and the object data of sample object 8.

[0100] Object data for sample object 1 Initial subcodebook 1 Object data for sample object 2 Initial subcodebook 2 Object data for sample object 3 Initial subcodebook 1 Object data for sample object 4 Initial subcodebook 3 Object data for sample object 5 Initial subcodebook 2 Object data for sample object 6 Initial subcodebook 3 Object data for sample object 7 Initial subcodebook 1 Object data for sample object 8 Initial subcodebook 3 Object data of sample object 9 Initial subcodebook 2

[0101] Table 1

[0102] After determining the sample object data set corresponding to each initial sub-codebook, the computing device generates, for any initial sub-codebook, an updated sub-codebook corresponding to the initial sub-codebook and a corresponding codeword block according to the sample object data set corresponding to the initial sub-codebook.

[0103] Optionally, the computing device may perform PCA dimensionality reduction processing on the sample object data set corresponding to the initial sub-codebook to generate an updated sub-codebook corresponding to the initial sub-codebook and corresponding codeword blocks.

[0104] In the implementation, when the computing device performs PCA dimensionality reduction processing on the sample object data set corresponding to the initial sub-codebook, the sample object data set corresponding to the initial sub-codebook is formed into a matrix, and the formed matrix is ​​subjected to PCA dimensionality reduction processing to obtain an updated sub-codebook and codeword blocks corresponding to the initial sub-codebook. The updated sub-codebook corresponds to the preset subspace size, the number of rows of the updated sub-codebook is the dimension d of the object data, and the number of columns of the updated sub-codebook is the preset subspace size p; and the number of rows of the obtained codeword blocks is the preset subspace size p, and the number of columns of the codeword blocks is the number of sample objects included in the sample object data set.

[0105] Exemplarily, assuming that the sample object data set corresponding to the initial sub-codebook includes object data of x sample objects, the sample object data set corresponding to the initial sub-codebook forms a matrix of d*x. Figure 8 The process of performing PCA dimensionality reduction processing on the matrix composed of the sample object data set corresponding to the initial sub-codebook is shown, and the d*x matrix is ​​reduced in dimension into a d*p sub-codebook and a p*x codeword block.

[0106] It should be noted that after obtaining the updated sub-codebook and codeword block, the computing device of the embodiment of the present application can repeatedly execute the above-mentioned process of updating the sub-codebook and the corresponding codeword block according to the sample object data set based on the updated sub-codebook, and continuously update the sub-codebook and codeword block until the determined sub-codebook and codeword block meet the conditions.

[0107] The computing device of the embodiment of the present application may be based on minimizing the least squares reconstruction loss function, and based on the loss function, determine whether each updated sub-codebook and codeword block meets the conditions;

[0108] The minimized least squares reconstruction loss function of the embodiment of the present application can be defined as follows:

[0109]

[0110] Wherein, C is a dense codebook matrix, whose dimension can be d*m; B is a structured block sparse codeword matrix, whose dimension can be m*n; A is a matrix composed of object data of multiple candidate objects in the original base library; Represents the Frobenius norm operation. B includes multiple codebook blocks, and the codebook blocks may include non-zero elements, and the positions in B other than the codebook blocks are 0, and each column of B corresponds to a codebook block (that is, each column in B has only one continuous region with non-zero elements); optionally, the sets of positions of non-zero elements in different columns of B are either completely the same or completely different, and the union of the positions of non-zero elements in different columns covers all dimensions.

[0111] After the computing device generates the updated sub-codebook and the corresponding codeword block corresponding to the initial sub-codebook, a codebook is generated according to the updated sub-codebook corresponding to each initial sub-codebook; and a codeword is generated according to the codeword block corresponding to each initial sub-codebook.

[0112] When the computing device generates a codeword, a possible implementation method is that the computing device divides each codeword into blocks to form a reference codeword, quantizes the reference codeword, and adjusts elements in the quantized reference codeword to generate the codeword.

[0113] In implementation, the computing device may use standard int 8 quantization to quantize the reference codeword; or may use lower bit quantization to quantize the reference codeword, thereby achieving a higher compression rate.

[0114] Optionally, the structure of the codeword generated by the computing device in the embodiment of the present application is a matrix including multiple codebook blocks, the codebook blocks include first elements, other positions in the matrix except the multiple blocks are second elements, and each column in the matrix corresponds to a block.

[0115] Exemplarily, the first element is a non-zero element, and the second element is a 0 element. Since the codeword structure only has non-zero elements in the codebook blocks, and other positions in the codeword are 0 elements, the non-zero elements in the codeword are concentrated in each codebook block, and each codebook block in the codebook is a dense matrix.

[0116] When adjusting the elements in the reference codeword after quantization processing, the computing device may rearrange each column of the reference codeword according to a non-zero element pattern, thereby generating a codeword.

[0117] like Fig. 9 The codeword structure obtained in the embodiment of the present application shown in the figure is a matrix including multiple codeword blocks, and each column in the matrix corresponds to a codeword block, and the positions of different codeword blocks in the column can be exactly the same. Fig.10 The codeword structure obtained in the embodiment of the present application shown in FIG. 1 is a matrix including multiple codeword blocks, and each column in the matrix corresponds to a codeword block, and the positions of different codeword blocks in the column are completely different; Fig.10 The codeword structure shown is a block diagonal matrix.

[0118] Correspondingly, based on the rearrangement of the reference codeword, the elements in each column of the original base library storing the object data of multiple candidate objects can also be rearranged in the same way, so that the matrix corresponding to the original base library is approximately equal to the product of the generated codebook and the codeword. Fig.11As shown, the matrix A (whose dimension is d*n) corresponding to the original base library is approximately equal to the product of the codebook C (whose dimension is d*m) and the codeword B, wherein the codeword B is a block diagonal matrix, and the codebook C includes multiple sub-codebooks, the number of columns of each sub-codebook is a preset subspace size p, the number of rows of each codeword block of the codeword B is the preset subspace size p, and each codeword block of the codeword B corresponds one-to-one to each sub-codebook in the codebook C.

[0119] After the computing device of the embodiment of the present application obtains the codebook and codeword corresponding to the object data of multiple candidate objects in the original base based on the above process, the codeword can be stored in the compressed base, and the base stores the data in the form of a matrix. Since the object data of multiple candidate objects can be approximately equal to the product of the codebook and the codeword, when other devices read the codewords stored in the target database, the object data of multiple candidate objects can be restored based on the codebook. As can be seen from the above introduction, the codeword structure of the embodiment of the present application is a matrix including multiple codeword blocks, and only the first element (exemplarily, the first element is a non-zero element) is included in the codeword block, and the second element (exemplarily, the second element is a zero element) is included in other positions of the matrix, thereby greatly reducing the amount of data stored in the target database.

[0120] When storing codewords in a base library, the base library may be divided into multiple storage areas. Optionally, the storage area in the embodiment of the present application may be a bucket, and a bucket stores data with high similarity.

[0121] In implementation, after the computing device obtains the codebooks and codewords corresponding to the object data of multiple candidate objects, when storing the codewords in the base database, one possible implementation method is that each storage area in the multiple storage areas is used to store a codeword block in the codeword. Fig.12 As shown, each codeword in the codeword structure is divided into blocks and stored in different storage areas in sequence.

[0122] In the database building process provided by the embodiment of the present application, the original base database storing the object data is subjected to database compression processing, and the object data of multiple candidate objects are compressed and encoded into codewords, and the structure of the codeword is a matrix including multiple codeword blocks. Since the non-zero elements in the codewords of the embodiment of the present application are concentrated in the codeword blocks, each codeword block in the codeword is a dense matrix, so that when the computing device reads the codeword and performs calculations, the operation speed of the dense matrix is ​​much higher than other operation forms; in addition, since each column in the codeword matrix corresponds to a codeword block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device. In summary, the codeword structure of the embodiment of the present application can greatly improve the computing speed and memory access speed of the computing device.

[0123] 2. Retrieval stage.

[0124] In the retrieval phase, the retrieval process may be executed by a computing device; wherein the computing device executing the retrieval process and the computing device executing the database building process in the database building phase may be different computing devices.

[0125] The following is combined with Fig.13 The following describes the retrieval process performed by the computing device in the embodiment of the present application.

[0126] Step 1301: The computing device receives a query request and determines a query vector corresponding to the query request.

[0127] After receiving the query request, the computing device determines the query vector according to the query content. Optionally, the computing device may extract features from the query content based on a feature extraction model to generate a query vector. Exemplarily, if the query request is to query for products similar to the products in the query image, and the query content obtained by the computing device is an image, the computing device extracts features of the products from the image to generate a query vector.

[0128] Step 1302: The computing device retrieves a target object matching the query request from a plurality of candidate objects stored in the original base library according to the query vector, the codeword, and the codebook.

[0129] The codeword used by the computing device in step 1302 is obtained by compressing and encoding the object data of multiple candidate objects through a codebook, and the codebook and codeword may be the codebook and codeword generated in the database building stage.

[0130] Since the embodiment of the present application stores each codeword block of the codeword in multiple storage areas of the base library, when the computing device retrieves the target object matching the query request, it first determines the target storage area from the multiple storage areas, and then retrieves the target object matching the query request based on the codeword blocks stored in the target storage area.

[0131] In a possible implementation, the computing device determines a target storage area from multiple storage areas according to a query vector and a codebook; and retrieves a target object matching the query request from multiple candidate objects according to the codeword blocks stored in the target storage area.

[0132] The following describes the process of determining the target storage area by the computing device and the process of retrieving the target object according to the codeword blocks stored in the target storage area.

[0133] 1. The computing device determines a target storage area from multiple storage areas in a target database according to the query vector and the codebook.

[0134] The codebook of the embodiment of the present application includes multiple sub-codebooks, wherein the size of the sub-codebook is determined according to the preset subspace size. Exemplarily, if the preset subspace size is p, then one sub-codebook corresponds to p candidate objects, that is, the dimension of the matrix corresponding to one sub-codebook is d*p, where d is the feature dimension of the object data of the candidate object.

[0135] In a possible implementation, the computing device may determine the target storage area according to the following method:

[0136] Step 1b: The computing device determines the subspace distance between the query vector and each sub-codebook in the codebook according to the query vector and the codebook.

[0137] Optionally, the computing device may determine the first similarity information between the query vector and the sub-codebook according to the following formula;

[0138] The first similarity information is used to represent the similarity between the query vector and the sub-codebook.

[0139] In implementation, the computing device may determine the first similarity information between the query vector and the sub-codebook in a variety of different ways:

[0140] Method 1: A computing device determines a product of a query vector and a codebook to obtain an intermediate vector; divides the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook; and determines, according to the sub-intermediate vectors corresponding to each sub-codebook, first similarity information between the query vector and each sub-codebook in the codebook.

[0141] In this manner, a larger value of the first similarity information determined by the computing device indicates a higher similarity between the query vector and the sub-codebook.

[0142] Exemplarily, if the feature dimension of the query vector is d, the query vector can be represented as a 1*d matrix, and the structure of the codebook is a d*m matrix. The query vector is multiplied by the codebook, and the intermediate vector obtained is a 1*m matrix. According to the size of the sub-codebook, the intermediate vector is divided into multiple sub-intermediate vectors, and the divided sub-intermediate vectors can be a 1*p matrix. When the computing device determines the first similarity information between the query vector and each sub-codebook in the codebook according to the sub-intermediate vector corresponding to each sub-codebook, the L2 norm of the 1*p matrix can be used as the first similarity information.

[0143] Mode 2: The computing device determines a subspace distance between the query vector and each sub-codebook in the codebook, and uses the subspace distance as first similarity information between the query vector and each sub-codebook in the codebook.

[0144] Optionally, the computing device may calculate the subspace distance between the query vector and the subcodebook according to the following formula:

[0145]

[0146] Where D is the subspace distance between the query vector and the subcodebook, q is the query vector (feature dimension is d), A k is the matrix corresponding to the sub-codebook (whose dimension is d*p).

[0147] In this manner, the smaller the value of the first similarity information determined by the computing device is, the higher the similarity between the query vector and the sub-codebook is.

[0148] Step 2b: The computing device determines a target sub-codebook from the multiple sub-codebooks included in the codebook according to the first similarity information between the query vector and each sub-codebook in the codebook.

[0149] If in step 2b, the computing device adopts method 1 to determine the first similarity information between the query vector and each sub-codebook in the codebook, the computing device selects k0 sub-codebooks with large first similarity information values ​​from the multiple sub-codebooks included in the codebook as the target sub-codebook. If in step 2b, the computing device adopts method 2 to use the subspace distance between the query vector and each sub-codebook in the codebook as the first similarity information, the computing device selects k0 sub-codebooks with small subspace distances from the multiple sub-codebooks included in the codebook as the target sub-codebook.

[0150] Step 3b: The computing device determines the target codeword block corresponding to the target subcodebook according to the correspondence between the subcodebook and the codeword block, and uses the storage area storing the codeword block corresponding to the target codeword block as the target storage area.

[0151] After the computing device determines the target sub-codebook from the codebook, since the sub-codebook corresponds one-to-one to the codeword block in the codeword structure, the computing device determines the target codeword block corresponding to the target sub-codebook in the codeword structure. In addition, since each storage area of ​​the target database stores a codeword block, the storage area storing the target codeword block is determined as the target storage area.

[0152] like Fig.14 As shown, after the computing device determines the first similarity information between the query vector and each sub-codebook in the codebook, the multiple storage areas of the target database are sorted according to the first similarity information. In implementation, if the first similarity information is determined by the above method 1, the first similarity information is sorted in descending order; if the first similarity information is determined by the above method 2, the first similarity information is sorted in descending order. The top k0 storage areas are selected from the multiple storage areas as the target storage areas.

[0153] 2. The computing device retrieves a target object matching the query request from multiple candidate objects according to the codeword blocks stored in the target storage area.

[0154] In a possible implementation, the computing device determines, based on the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, where the second similarity information is used to characterize the degree of similarity between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area; the computing device determines the target object from the candidate objects corresponding to the target sub-codebook based on the second similarity information.

[0155] In implementation, the computing device may determine the second similarity information in a variety of different ways:

[0156] Method 1: The computing device determines the product of the sub-intermediate vector corresponding to the target sub-codebook and the codeword block stored in the target storage area, and determines the second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area according to the product.

[0157] Exemplarily, assuming that the target sub-codebook is a d*p matrix and the codeword blocks stored in the target storage area are a p*x matrix, the product of the target sub-codebook and the codeword blocks stored in the target storage area is a d*x matrix; wherein each column in the d*x matrix corresponds to a candidate object. Optionally, the computing device may determine the L2 norm for each column of the d*x matrix, thereby obtaining the second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area.

[0158] After the computing device obtains the second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area based on the above-mentioned method 1, it selects k1 candidate objects with the largest second similarity information values ​​from the candidate objects corresponding to the codeword blocks stored in the target storage area as target objects according to the second similarity information.

[0159] Method 2: The computing device determines, based on the codeword blocks stored in the target storage area, an adjustment weight for each candidate object corresponding to the codeword blocks stored in the target storage area; and determines the product of the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, and determines, based on the product and each adjustment weight, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area.

[0160] The following is a detailed description of method 2 for determining the second similarity information.

[0161] Step 1c: The computing device determines, according to the code word blocks stored in the target storage area, an adjustment weight of each candidate object corresponding to the code word blocks stored in the target storage area;

[0162] In the implementation, the computing device determines the L2 norm of each column of the codeword block stored in the target storage area; determines the adjustment weight of the candidate object according to the calculation result of the L2 norm; illustratively, if the codeword block stored in the target storage area is a p*x matrix, then for each column of the matrix, the L2 norm is determined, thereby obtaining x adjustment weights; optionally, assuming that the L2 norm of each column of the codeword block is y, 0.5*y is set. 2 The adjusted weight of the candidate object corresponding to this column.

[0163] Step 2c: The computing device determines the product of the sub-intermediate vector corresponding to the target sub-codebook and the codeword block stored in the target storage area;

[0164] Exemplarily, assuming that the target sub-codebook is a d*p matrix and the codeword blocks stored in the target storage area are a p*x matrix, the product of the target sub-codebook and the codeword blocks stored in the target storage area is a d*x matrix.

[0165] Step 3c: The computing device determines, based on the product and each adjusted weight, second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area;

[0166] If the product obtained in step 2c is a d*x matrix, the computing device determines the L2 norm of each column of the matrix, and uses the L2 norm of each column as the similarity corresponding to the column; the computing device takes the negative value of the similarity, and takes the sum of the similarity after taking the negative value and the corresponding adjustment parameter as the second similarity information.

[0167] After the computing device obtains the second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area based on the above-mentioned method 2, it selects k1 candidate objects with the smallest second similarity information values ​​from the candidate objects corresponding to the codeword blocks stored in the target storage area as target objects according to the second similarity information.

[0168] After the computing device in the embodiment of the present application determines the target object that matches the query request from multiple candidate objects, it can use the identification information of the target object as the retrieval result.

[0169] The retrieval method provided by the embodiment of the present application is that when the computing device obtains the query vector and retrieves the retrieval results matching the query vector from the database, the computing device retrieves from the target database after compression encoding; and the target database of the embodiment of the present application stores the codeword obtained after compression encoding of the original object data, and the codeword structure is specially designed so that the structure of the codeword includes a matrix of multiple codeword blocks, and each column in the matrix corresponds to a codeword block. The non-zero elements in the codeword of the embodiment of the present application are concentrated in the codeword blocks, and each codeword block in the codeword is a dense matrix. Since the operation speed of the dense matrix is ​​much higher than other operation forms, the computing device can significantly improve the operation speed based on the dense codeword matrix during the retrieval process, thereby improving the retrieval efficiency; in addition, since each column in the codeword matrix corresponds to a codeword block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device, so the retrieval efficiency can be further improved.

[0170] Based on the above content and the same technical concept, the embodiment of the present application provides a data processing device 1500, such as Fig.15 As shown, the data processing device 1500 includes an initialization unit 1501 and a processing unit 1502 .

[0171] An initialization unit 1501 is configured to generate an initial codebook, wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, the m is greater than p, the p is greater than 1, and the d is greater than 1;

[0172] The processing unit 1502 is configured to generate a codebook for compressing and encoding the object data of the candidate object stored in the original base library according to each initial sub-codebook in the initial codebook, and a codeword after the compression encoding; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, the codeword block includes a first element, and other positions in the matrix except the plurality of codeword blocks are second elements, and each column in the matrix corresponds to a codeword block.

[0173] In a possible implementation, the initialization unit 1501 is specifically configured to:

[0174] Selecting m sample objects' object data from the object data of the candidate objects stored in the original base database to form a plurality of sample sets, wherein each sample set includes p sample objects' object data;

[0175] Each initial sub-codebook included in the initial codebook is generated according to object data of the sample object included in each sample set.

[0176] In a possible implementation, the processing unit 1502 is specifically configured to:

[0177] Determine, according to the subspace distance between the object data of each sample object and each initial subcodebook, the sample object data set corresponding to each initial subcodebook;

[0178] For any initial sub-codebook, generating an updated sub-codebook and corresponding codeword blocks corresponding to the initial sub-codebook according to a sample object data set corresponding to the initial sub-codebook;

[0179] The codebook is generated according to the updated sub-codebook corresponding to each initial sub-codebook; and the codeword is generated according to the codeword block corresponding to each initial sub-codebook.

[0180] In a possible implementation, the processing unit 1502 is specifically configured to:

[0181] For object data of any sample object, determining an initial subcodebook corresponding to the object data of the sample object from a plurality of initial subcodebooks according to a subspace distance between the object data of the sample object and each initial subcodebook;

[0182] According to the correspondence between the object data of the sample object and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook is determined respectively, wherein the sample object data set includes the object data of the sample object corresponding to the initial sub-codebook.

[0183] In a possible implementation, the processing unit 1502 is specifically configured to:

[0184] Divide the codewords corresponding to each of the initial sub-codebooks into blocks to form reference codewords;

[0185] The reference codeword is quantized, and elements in the quantized reference codeword are adjusted to generate the codeword.

[0186] In a possible implementation manner, the structure of the codeword is a block diagonal matrix.

[0187] In a possible implementation, the present application embodiment further provides a data processing device 1600, such as Fig.16 As shown, the data processing device 1600 includes a receiving unit 1601 and a searching unit 1602; wherein the data processing device 1600 and the Fig.15 The data processing apparatus 1500 shown can be deployed on a device, or the data processing apparatus can be deployed on a device such as Fig.15 The data processing apparatus shown may be deployed separately.

[0188] The receiving unit 1601 is configured to receive a query request and determine a query vector corresponding to the query request;

[0189] The retrieval unit 1602 is configured to retrieve a target object matching the query request from a plurality of candidate objects stored in the original base database according to the query vector, the codeword, and the codebook.

[0190] In a possible implementation, each of the codewords is divided into blocks and stored in different storage areas.

[0191] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0192] Determine a target storage area according to the query vector and the codebook;

[0193] According to the codeword blocks stored in the target storage area, a target object matching the query request is retrieved from the multiple candidate objects.

[0194] In a possible implementation manner, the codebook includes multiple sub-codebooks, each sub-codebook corresponds to a codeword block;

[0195] The retrieval unit 1602 is specifically used for:

[0196] Determine, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, wherein the first similarity information is used to represent a degree of similarity between the query vector and the sub-codebook;

[0197] Determining a target subcodebook from a plurality of subcodebooks included in the codebook according to first similarity information between the query vector and each subcodebook in the codebook;

[0198] According to the correspondence between the sub-codebook and the block, the target codeword block corresponding to the target sub-codebook is determined, and the storage area storing the target codeword block is used as the target storage area.

[0199] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0200] Determine a product of the query vector and the codebook to obtain an intermediate vector;

[0201] Dividing the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook respectively;

[0202] Determine first similarity information between the query vector and each sub-codebook in the codebook according to the sub-intermediate vector corresponding to each sub-codebook.

[0203] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0204] Determining, according to the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, wherein the second similarity information is used to represent the degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area;

[0205] The target object is determined from candidate objects corresponding to the target sub-codebook according to the second similarity information.

[0206] Fig.17 A schematic diagram of a computing device provided in an embodiment of the present application. The computing device is used to perform the above Figure 5 , Fig.13 The method executed by the computing device in the method embodiment. The computing device 1700 includes a processor 1701 , a memory 1702 , and a communication interface 1703 . The processor 1701 , the memory 1702 , and the communication interface 1703 may be connected via a bus 1704 .

[0207] The processor 1701 is used to execute instructions stored in the memory 1702 so that the data processing device 1400 or the data processing device 1500 executes the data processing method provided in the present application. The processor 1701 can be but is not limited to: a central processing unit (CPU), a data processing unit (data processing unit, DPU), a system on chip (system on chip, SOC), a field programmable gate array (field programmable gate array, FPGA), a graphics processing unit (graphics processing unit, GPU), an application specific integrated circuit (application specific integrated circuit, ASIC) or an acceleration card and other processors. Any one or more.

[0208] Memory 1702 is used to store computer instructions and data, such as memory 1702 storing computer instructions and data required to implement the data processing method provided in the present application. Memory 1702 includes volatile memory (volatile memory), such as random access memory (random access memory, RAM), dynamic random access memory (dynamic random access memory, DRAM), etc. It can also be non-volatile memory (non-volatile memory), such as read-only memory (read-only memory, ROM), storage-class memory (storage-class memory, SCM), flash memory, hard disk drive (hard disk drive, HDD) or solid state drive (solid state drive, SSD).

[0209] The memory 1702 stores executable program codes, and the processor 1701 executes the executable program codes to implement the aforementioned data processing functions. The memory 1702 stores instructions for the data processing device 1500 and / or the data processing device 1600 to execute the data processing method provided in the present application.

[0210] The communication interface 1703 is used to communicate with internal devices or external devices, such as obtaining a read request / write request sent by a process, or communicating with a storage device to complete data access. Exemplarily, the communication interface 1703 may be a network card.

[0211] The bus 1704 may be a Peripheral Component Interconnect Express (PCIe) bus, a double data rate (DDR) bus, a serial advanced technology attachment (SATA) bus, a serial attached SCSI (SAS) bus, a controller area network (CAN) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.17 The bus 1704 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 1704 may include a path for transmitting information between each component of the data processing device 1700 (for example, the memory 1702, the processor 1701, and the communication interface 1703).

[0212] The present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above Figure 5 or Fig.13 The data processing method executed by the computing device in the embodiment is shown in Figure 5 or Fig.13 The description of each step is not repeated here.

[0213] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to perform the above Figure 5 or Fig.13The data processing method executed by the computing device in the embodiment is shown in Figure 5 or Fig.13 The description of each step is not repeated here.

[0214] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0215] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or the design of any combination of the above functions. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0216] The steps of the method or algorithm described in the embodiments of the present application can be directly embedded in the software unit executed by the hardware, the processor, or a combination of the two. The software unit can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC.

[0217] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0218] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: The method comprises: Generate an initial codebook; wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, the m is greater than p, the p is greater than 1, and the d is greater than 1; According to each of the multiple initial sub-codebooks, a codebook for compressing and encoding object data of the candidate object stored in the original base library and a codeword after the compression encoding are generated; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, and each column in the matrix corresponds to a codeword block.

2. The method according to claim 1, characterized in that The generating of the initial codebook comprises: Selecting m sample objects' object data from the object data of the candidate objects stored in the original base database to form a plurality of sample sets, wherein each sample set includes p sample objects' object data; Each initial sub-codebook included in the initial codebook is generated according to object data of the sample object included in each sample set.

3. The method according to claim 2, characterized in that The step of generating, according to each initial sub-codebook in the initial codebook, a codebook for compressing and encoding the object data of the candidate object stored in the original base library, and a codeword after the compression encoding, comprises: Determine, according to the subspace distance between the object data of each sample object and each initial subcodebook, a sample object data set corresponding to each initial subcodebook; For any initial sub-codebook, generating an updated sub-codebook and corresponding codeword blocks corresponding to the initial sub-codebook according to a sample object data set corresponding to the initial sub-codebook; The codebook is generated according to the updated sub-codebook corresponding to each initial sub-codebook; and the codeword is generated according to the codeword block corresponding to each initial sub-codebook.

4. The method according to claim 3, characterized in that The step of determining the sample object data set corresponding to each initial sub-codebook according to the subspace distance between the object data of each sample object and each initial sub-codebook comprises: For object data of any sample object, determining an initial subcodebook corresponding to the object data of the sample object from a plurality of initial subcodebooks according to a subspace distance between the object data of the sample object and each initial subcodebook; According to the correspondence between the object data of the sample object and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook is determined respectively, wherein the sample object data set includes the object data of the sample object corresponding to the initial sub-codebook.

5. The method according to claim 3, characterized in that The generating the codeword according to the codeword blocks corresponding to each initial sub-codebook includes: Divide the codewords corresponding to each of the initial sub-codebooks into blocks to form reference codewords; The reference codeword is quantized, and elements in the quantized reference codeword are adjusted to generate the codeword.

6. The method according to any one of claims 1 to 5, characterized in that: The structure of the codeword is a block diagonal matrix.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: receiving a query request and determining a query vector corresponding to the query request; According to the query vector, the codeword and the codebook, a target object matching the query request is retrieved from a plurality of candidate objects stored in the original base database.

8. The method according to claim 7, characterized in that Each codeword in the codeword is divided into blocks and stored in different storage areas respectively.

9. The method according to claim 8, characterized in that The step of retrieving a target object matching the query request from a plurality of candidate objects stored in the original base database according to the query vector, the codeword, and the codebook includes: Determine a target storage area according to the query vector and the codebook; According to the codeword blocks stored in the target storage area, a target object matching the query request is retrieved from the multiple candidate objects.

10. The method according to claim 9, characterized in that The codebook includes a plurality of sub-codebooks, each sub-codebook corresponding to a codeword block; The determining a target storage area according to the query vector and the codebook includes: Determine, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, wherein the first similarity information is used to represent a degree of similarity between the query vector and the sub-codebook; Determining a target subcodebook from a plurality of subcodebooks included in the codebook according to first similarity information between the query vector and each subcodebook in the codebook; According to the correspondence between the sub-codebook and the codeword block, the target codeword block corresponding to the target sub-codebook is determined, and the storage area storing the target codeword block is used as the target storage area.

11. The method according to claim 10, characterized in that The determining, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook includes: Determine a product of the query vector and the codebook to obtain an intermediate vector; Dividing the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook respectively; Determine first similarity information between the query vector and each sub-codebook in the codebook according to the sub-intermediate vector corresponding to each sub-codebook.

12. The method according to claim 11, characterized in that The step of determining a target object matching the query request from the plurality of candidate objects according to the codeword blocks stored in the target storage area comprises: Determining, according to the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, wherein the second similarity information is used to represent the degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area; The target object is determined from candidate objects corresponding to the target sub-codebook according to the second similarity information.

13. A data processing device, characterized in that: The device comprises: An initialization unit, configured to generate an initial codebook, wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, the m is greater than p, the p is greater than 1, and the d is greater than 1; A processing unit generates, according to each of the multiple initial sub-codebooks, a codebook for compressing and encoding object data of the candidate object stored in the original base library, and a codeword after the compression encoding; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, and each column in the matrix corresponds to a codeword block.

14. A computing device, characterized in that The computing device comprises a memory and at least one processor, wherein the memory is used to store a set of computer program instructions, and when the processor executes the set of program instructions, the method according to any one of claims 1 to 12 is performed.

15. A computer-readable storage medium, characterized in that: When the computer-readable storage medium is executed by a storage device, the storage device executes the method of any one of claims 1 to 12 above.

Images