Data processing method, and apparatus

Abstract

The present application provides a data processing method, and an apparatus, which are used for improving retrieval efficiency. The method comprises: a computing device generating an initial codebook, the structure of the initial codebook being a d*m matrix, the initial codebook comprising a plurality of initial sub-codebooks, and the structure of each initial sub-codebook being a d*p matrix, where m is greater than p, p is greater than 1, and d is greater than 1; and, on the basis of each initial sub-codebook, generating a codebook used for compressing and encoding object data of candidate objects stored in an original base library, and compressed and encoded codewords, the structure of the codewords being a matrix comprising a plurality of codeword blocks, the codeword blocks comprising first elements, other positions in the matrix except the plurality of codeword blocks being second elements, and each column in the matrix corresponding to one codeword block. By compressing and encoding the object data in the original base library to obtain the codebook and the codewords, and ensuring that the codeword blocks in the codewords are a dense matrix, the operation and memory access speed of the computing device can be greatly increased.

Description

Data processing method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on July 21, 2023, with application number 202310910436.3 and application name "A Data Processing Method and Device", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a data processing method and device. Background Art

[0004] The continuous development of computer technology has generated a vast amount of information data. Large-scale database retrieval is the process of retrieving the required data from this vast amount of information data. The process involves obtaining the object data of the query object, searching the information database for information data similar to the query object data, and returning the found information data as the search results.

[0005] A typical problem with large database retrieval is the sheer volume of data stored in the database. For example, a facial feature database can store the facial features of a large number of sample objects. Each sample object corresponds to a high-dimensional facial feature. For example, a sample object's facial features may be 256-dimensional, resulting in a database with a data volume in the billions.

[0006] Large database retrieval is a data-intensive task that generally runs on the accelerator card of the computing device. When performing the retrieval task, the accelerator card needs to load all the information data in the database into the video memory. Due to the large amount of data in the database, the retrieval efficiency is low.

[0007] Summary of the Invention

[0008] The present application provides a data processing method and device to improve retrieval efficiency.

[0009] In a first aspect, an embodiment of the present application provides a data processing method, which can be specifically executed by a computing device or by a chip inside the computing device, or by a processor in the computing device (exemplarily, the processor can be a central processing unit, an accelerator card, etc.). Taking the execution of a computing device as an example, the method includes: the computing device generates an initial codebook; wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes multiple initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, and m is greater than p, p is greater than 1, and d is greater than 1; the computing device generates a codebook for compressing and encoding the object data of the candidate object stored in the original base library according to each initial sub-codebook in the initial codebook, and a codeword after compression encoding; wherein the structure of the codeword is a matrix including multiple codeword blocks, the codeword block includes a first element, and the other positions in the matrix except the multiple codeword blocks are second elements, and each column in the matrix corresponds to a codeword block.

[0010] Through the above design, the computing device generates a codebook and codeword corresponding to the object data corresponding to multiple candidates stored in the original base library based on the initial codebook. Since the codeword is obtained by compressing and encoding the object data of the candidate objects stored in the original base library, the data volume of the codeword is much smaller than the data volume of the object data corresponding to multiple candidates stored in the original base library. During retrieval, retrieval can be performed based on the codeword and codebook obtained by compression encoding, thereby improving retrieval efficiency. In addition, the structure of the codeword is a matrix including multiple codeword blocks. For example, non-zero elements can be concentrated in the codeword blocks, and the codeword block structure is a dense matrix. In this way, when the computing device reads the codeword and performs calculation processing, the calculation speed of the dense matrix is ​​much higher than other calculation forms; and each column in the codeword matrix corresponds to a codeword block, that is, each column in the codeword matrix only includes an area with non-zero elements. Therefore, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device.

[0011] In one possible design, when generating the initial codebook, the computing device selects object data of m sample objects from the object data of candidate objects stored in the original base library to form multiple sample sets, where each sample set includes object data of p sample objects; and generates each initial sub-codebook included in the initial codebook based on the object data of the sample objects included in each sample set.

[0012] Through the above design, the computing device generates an initial codebook based on the object data of the sample objects selected from the original base library. This initial codebook generation method makes the obtained initial codebook more accurate.

[0013] In one possible design, when determining the initial sub-codebook corresponding to each sample set, the computing device performs, for any sample set, orthogonal basis initialization on a matrix composed of object data of sample objects included in the sample set to generate the initial sub-codebook corresponding to the sample set.

[0014] Through the above design, the initial sub-codebook obtained by performing an orthogonal basis initialization on the matrix composed of the object data of the sample object can better meet the requirements related to the codebook and improve the accuracy of the generated codebook.

[0015] In one possible design, a computing device determines, based on a subspace distance between object data of each sample object and each initial sub-codebook, a sample object dataset corresponding to each initial sub-codebook; for any initial sub-codebook, generates, based on the sample object dataset corresponding to the initial sub-codebook, an updated sub-codebook and a corresponding codeword block corresponding to the initial sub-codebook; generates a codebook based on the updated sub-codebook corresponding to each initial sub-codebook; and generates a codeword based on the codeword block corresponding to each initial sub-codebook.

[0016] Through the above design, when determining the codebook and codeword, the computing device first determines the sample object data set corresponding to the initial sub-codebook based on the subspace distance between the object data included in the sample set and the initial sub-codebook, so that the object data with high similarity to the initial sub-codebook can be used as a sample object data set, and a new sub-codebook and corresponding codeword blocks are generated by processing the sample object set. In this way, the object data of the candidate objects corresponding to the sub-codebook and the codeword blocks are data with high similarity. Therefore, the codebook and codewords generated in this manner can improve the retrieval efficiency in the subsequent retrieval process.

[0017] In one possible design, the computing device determines, for object data of any sample object, an initial sub-codebook corresponding to the object data of the sample object from multiple initial sub-codebooks based on a subspace distance between the object data of the sample object and each initial sub-codebook; and determines, based on a correspondence between the object data of the sample object and the initial sub-codebooks, a sample object dataset corresponding to each initial sub-codebook, where the sample object dataset includes the object data of the sample object corresponding to the initial sub-codebook.

[0018] Through the above design, the sample object data set corresponding to each initial sub-codebook determined by the computing device includes object data of relatively similar candidate objects. Based on this method, codebooks and codewords are generated, which can improve retrieval efficiency in subsequent retrieval processes.

[0019] In one possible design, when generating a codeword based on the codeword blocks corresponding to each initial sub-codebook, the computing device divides the codeword blocks corresponding to each initial sub-codebook into reference codewords, quantizes the reference codewords, and adjusts elements in the quantized reference codewords to generate the codewords. Exemplarily, the computing device adjusts the elements in the quantized reference codewords such that the structure of the adjusted codeword is a matrix including multiple codeword blocks, where each column in the matrix corresponds to a codeword block.

[0020] Through the above design, the computing device quantizes the reference codewords composed of each codeword block, which can achieve a higher compression rate. In addition, by adjusting the elements in the reference codeword, each column in the adjusted codeword corresponds to a codeword block. The adjusted codeword is a dense matrix, which can improve the computing speed when the computing device calls the codeword.

[0021] In one possible design, the codeword is structured as a block diagonal matrix.

[0022] In one possible design, after receiving a query request, the computing device determines a query vector corresponding to the query request; and retrieves a target object that matches the query request from multiple candidate objects stored in the original database based on the query vector, codeword, and codebook.

[0023] Through the above design, after obtaining the query vector, the computing device searches based on the codeword obtained after compressing and encoding the original object data. The codeword structure is specially designed so that the codeword structure includes multiple block matrices, and each column in the matrix corresponds to a block. The codeword structure is a dense matrix. Since the operation speed of a dense matrix is ​​much higher than other operation forms, the computing device can significantly increase the operation speed during the retrieval process based on this dense codeword matrix, thereby improving the retrieval efficiency. In addition, since each column in the codeword matrix corresponds to a block, that is, each column in the codeword matrix only includes one area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks. Continuous memory block access can greatly increase the memory access speed of the computing device, thereby further improving the retrieval efficiency.

[0024] In a possible design, each codeword in the codeword is divided into blocks and stored in different storage areas.

[0025] Through the above design, each codeword block in the codeword structure represents data with high similarity. By storing each codeword block in a different storage area, it is possible to centrally store data with high similarity, facilitating data retrieval in subsequent processes.

[0026] In one possible design, the computing device determines a target storage area based on a query vector and a codebook; and retrieves a target object matching the query request from multiple candidate objects based on codeword blocks stored in the target storage area.

[0027] With this design, when executing the search process, the computing device first determines the target storage area for storing the codeword blocks based on the query vector and the codebook. It then searches within the target storage area for target objects that match the query. This query method significantly improves search efficiency compared to searching across all codeword blocks.

[0028] In one possible design, the codebook includes multiple sub-codebooks, each sub-codebook corresponding to a codeword block; when determining the target storage area, the computing device determines, based on the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, where the first similarity information is used to represent a degree of similarity between the query vector and the sub-codebook; the computing device determines a target sub-codebook from the multiple sub-codebooks included in the codebook based on the first similarity information between the query vector and each sub-codebook in the codebook; the computing device determines, based on the correspondence between the sub-codebooks and the codeword blocks, a target codeword block corresponding to the target sub-codebook, and uses the storage area storing the target codeword block as the target storage area.

[0029] With the above design, when determining a target storage area from multiple storage areas, the computing device calculates the first similarity information between the query vector and each sub-codebook in the codebook to determine a target sub-codebook with a high similarity to the query vector. Then, based on the correspondence between the sub-codebooks and the codeword blocks, the computing device can determine the storage area storing the codeword blocks corresponding to the target sub-codebook as the target storage area. This design allows the codeword blocks stored in the target storage area to be determined to be most similar to the query vector from among the multiple storage areas, thereby simplifying the retrieval process.

[0030] In one possible design, a computing device determines a product of a query vector and a codebook to obtain an intermediate vector; divides the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook; and determines first similarity information between the query vector and each sub-codebook in the codebook based on the sub-intermediate vectors corresponding to each sub-codebook.

[0031] Through the above design, a solution is provided for accurately determining the first similarity information between the query vector and each sub-codebook in the codebook. Based on this design, the first similarity information between the query vector and each sub-codebook in the codebook can be accurately determined.

[0032] In one possible design, the computing device determines, based on the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area, where the second similarity information is used to characterize the degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area. The computing device then determines the target object from the candidate objects corresponding to the target sub-codebook based on the second similarity information.

[0033] Through the above design, after determining the target storage area, the computing device calculates the second similarity information between the query vector and the codeword blocks stored in the target storage area, and retrieves the target object that matches the query vector from the candidate objects corresponding to the target storage area; based on the second similarity information, the target object can be accurately retrieved.

[0034] In a second aspect, embodiments of the present application provide a data processing apparatus having the functionality to implement the method described in the first aspect or any possible implementation of the first aspect. The apparatus may be a computing device or a processor included in a computing device. The functionality of the data processing apparatus may be implemented via hardware or via hardware executing corresponding software. The hardware or software may include one or more modules, units, or means corresponding to the functionality described above.

[0035] In one possible implementation, in one possible design, the structure of the device includes an initialization unit and a processing unit. These units can perform the corresponding functions of the computing device in the above-mentioned first aspect method example. For details, please refer to the detailed description in the method example, which is not repeated here.

[0036] In another possible implementation, the apparatus includes a processor and may also include a memory. The processor is coupled to the memory and can be configured to execute computer program instructions stored in the memory, causing the apparatus to perform the method described in the first aspect or any possible implementation of the first aspect. Optionally, the apparatus further includes a communication interface, with the processor coupled to the communication interface. When the apparatus is a computing device, the communication interface may be a transceiver or an input / output interface.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program or instruction stored thereon. When the computer program or instruction is executed, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0039] The technical effects that can be achieved in any of the second to fourth aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a schematic diagram of an image retrieval scenario provided by an embodiment of the present application;

[0041] FIG2 is a schematic diagram of a bottom storage structure provided in an embodiment of the present application;

[0042] FIG3 is a schematic diagram of a product recommendation scenario provided in an embodiment of the present application;

[0043] FIG4 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0044] FIG5 is a schematic diagram of a process for establishing a compressed database according to an embodiment of the present application;

[0045] FIG6 is a schematic diagram of a sample set division method provided in an embodiment of the present application;

[0046] FIG7 is a schematic diagram of determining a subspace distance according to an embodiment of the present application;

[0047] FIG8 is a schematic diagram of performing PCA dimensionality reduction processing on a matrix according to an embodiment of the present application;

[0048] FIG9 is a codeword structure provided in an embodiment of the present application;

[0049] FIG10 is another codeword structure provided in an embodiment of the present application;

[0050] FIG11 is a schematic diagram of the relationship between an original base library, a codebook, and codewords provided in an embodiment of the present application;

[0051] FIG12 is a schematic diagram of a partitioned storage provided in an embodiment of the present application;

[0052] FIG13 is a search flow chart provided in an embodiment of the present application;

[0053] FIG14 is a schematic diagram of determining a target storage area according to an embodiment of the present application;

[0054] FIG15 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0055] FIG16 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0056] FIG17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] First, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0058] 1. Database Compression: Database compression compresses stored data to reduce storage space and improve database performance and efficiency. The primary goals of database compression are to optimize storage resource utilization, improve query performance, and reduce storage costs. Compressing a database reduces storage space usage, thereby lowering storage costs. Furthermore, a compressed database improves read and write efficiency due to the reduced data volume, accelerating query and transaction processing.

[0059] 2. Vector retrieval technology: In a given vector data set, vectors similar to the query vector are retrieved according to a certain measurement method.

[0060] 3. The k-means clustering algorithm is an iterative clustering analysis algorithm. Specifically, given the number of clusters, k, the entire dataset is clustered. The objective function is to minimize the sum of the distances from all samples to the cluster centers. The objective function is iteratively calculated and optimized to obtain k cluster centers and the category to which each sample belongs.

[0061] 4. Principal component analysis (PCA) dimensionality reduction is a common data analysis method that uses orthogonal transformation to convert data represented by possibly correlated variables into data represented by a small number of linearly uncorrelated variables. Linearly uncorrelated variables are called principal components. The number of principal components is usually smaller than the number of original variables, so principal component analysis is often used to reduce the dimensionality of high-dimensional data and extract the main characteristic components of the data. Common methods of PCA dimensionality reduction include the covariance matrix method and the eigenvalue decomposition method. Among them, the covariance matrix method uses the covariance matrix to solve the eigenvalue and eigenvector, and then obtains the reduced-dimensional data; the eigenvalue decomposition method can use the singular value decomposition method to solve, thereby obtaining the reduced-dimensional data.

[0062] 5. The base database is a database or data structure used to store extracted feature vectors. It is commonly used in tasks such as face recognition, image retrieval, and speech recognition to facilitate similarity matching or comparison. When building the base database, features must be extracted for each sample object and stored in the base database. For example, the base database can use a matrix structure to store data, with each column representing a feature vector for a sample object.

[0063] Figure 1 shows a schematic diagram of an image retrieval scenario. First, a collection of images is obtained, each of which includes a sample object. Object recognition is performed on each sample image in the collection. A feature extraction model is used to extract object data (also called object features) for the sample object in each sample image. The extracted object data for each sample object is stored to form a database. Figure 2 shows a schematic diagram of the database structure. The database structure can be a d*n matrix, where n represents the number of sample objects stored in the database, and d represents the dimension of each sample object's object data. Upon receiving a query request, the query request may include the image to be queried. Object features of the query object in the queried image are extracted to obtain a query vector. The database is searched based on the query vector to retrieve object data that meets a similarity requirement with the query vector. The target images from which these object data meeting the similarity requirement were extracted are determined, and the identified target images are returned as retrieval results. For example, in an internet application, based on a user-entered product image to be queried, images containing products with a similar appearance to the product in the user-entered product image are retrieved from the database.

[0064] Figure 3 shows a schematic diagram of a product recommendation scenario. First, a large number of product samples are obtained, and the product features of each product are extracted through a feature extraction model (for example, feature extraction can be performed on pictures of the product appearance, product description information, etc.), and the extracted product features of each product are stored to form a base database. When recommending products to users, the query request obtained includes a user identifier, and the user features of the user are extracted through a feature extraction model (in implementation, feature extraction can be performed on the user's historical behavior data, user portrait, etc. to extract user features), and the user features are used as query vectors; according to the query vector, a search is performed in the base database to retrieve product features that meet the similarity requirements with the query vector; it is determined from which target products these product features that meet the similarity requirements are extracted, and the determined target products are returned as search results.

[0065] During implementation, a query request can be received by a computing device. The computing device then retrieves object data from a database that meets the required similarity with the query vector based on the query vector corresponding to the query request, and uses the candidate objects of the object data that meet the required similarity with the query vector as the retrieval result. Since retrieval from the database is a data-intensive task, it can be run on the accelerator card of the computing device. When executing the retrieval task, the computing device loads the object data in the database into the video memory, so that object data that matches the query vector can be found from the large amount of object data in the database, thereby obtaining the retrieval result. When the amount of object data stored in the database is large, searching from all the object data stored in the database will result in low retrieval efficiency.

[0066] Based on the above problems, an embodiment of the present application provides a data processing method, which compresses and encodes the object data of multiple candidate objects through a codebook to obtain codewords. The compressed and encoded codewords can be stored in a base database. When performing a retrieval task, the retrieval results can be quickly obtained based on the query vector, codebook and codewords.

[0067] The following describes the system architecture applicable to the embodiments of this application. As shown in Figure 4, the system includes a terminal device 40, a computing device 41, and a storage device 42. Computing device 41 can receive a query request sent by terminal device 40 and determine a query vector corresponding to the query request. Storage device 42 stores codewords obtained by compressing and encoding the object data of multiple candidate objects, as well as a codebook corresponding to the codewords.

[0068] Illustratively, the query request may include a picture to be queried, and the computing device 41 may extract object data of the query object included in the picture to be queried, and use the extracted object data of the query object as a query vector.

[0069] The computing device 41 obtains the codebook and codeword from the storage device 42 and, based on the query vector, codeword, and codebook, retrieves a target object that matches the query request from multiple candidate objects. In implementation, when the computing device 41 performs a retrieval task, the retrieval task can be performed by a processor in the computing device 41 or by an accelerator card. If the retrieval task is performed by the processor, the processor loads the codebook and codeword from the storage device 42 into the memory. If the retrieval task is performed by the accelerator card, the processor loads the codebook and codeword from the storage device 42 into the video memory.

[0070] After obtaining the query vector, the computing device 41 searches from the compressed and encoded base library. The base library stores codewords obtained after compression encoding of the original object data, and the codeword structure is specially designed so that the codeword structure includes a matrix of multiple blocks, and each column in the matrix corresponds to a codeword block. Each codeword block obtained in the codeword structure is a dense matrix. Since the operation speed of the dense matrix is ​​much higher than other operation forms, the processor or acceleration card of the computing device 41 can significantly improve the operation speed based on the dense codeword matrix during the retrieval process, thereby improving the retrieval efficiency; in addition, since each column in the codeword matrix corresponds to a block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks. Continuous memory block access can greatly improve the memory access speed, thereby further improving the retrieval efficiency.

[0071] It should be noted that the system architecture shown in Figure 4 is only an example. The embodiments of the present application do not limit the specific form of the system architecture. The system architecture applicable to the embodiments of the present application may have more or fewer devices than shown in Figure 4.

[0072] Based on the system architecture shown in Figure 4 above, this embodiment of the present application provides a data processing solution. The implementation process of this solution can be divided into two phases: the first phase is the database construction phase, and the second phase is the retrieval phase. In the database construction phase, the object data of multiple candidate objects is compressed and encoded to obtain a compressed base database. In the retrieval phase, the target object that matches the query request is retrieved from the compressed base database based on the query vector corresponding to the query request. These two phases are described below.

[0073] 1. Database construction stage.

[0074] During the database building phase, the database building process may be executed by a computing device; wherein, the computing device executing the database building process and the computing device executing the retrieval process in FIG. 4 may be different computing devices.

[0075] During the database building phase, embodiments of the present application can collect a large number of samples; the computing device can extract object data for multiple candidate objects from the samples. For example, the samples can be images, and the object data for the candidate objects can be feature data for objects in the images, including but not limited to people, goods, buildings, animals, and plants.

[0076] The computing device can construct an original database based on the object data of the extracted candidate objects. The original database can be structured in a matrix format, with each column representing the object data of a candidate object. Because the number of candidate objects collected during the database construction phase is large, and the object data of each candidate object is high-dimensional, the original database will be very large.

[0077] In implementation, the computing device can extract object data of the candidate object from each collected sample based on the trained feature extraction model. The embodiment of the present application does not limit the type of feature extraction model used.

[0078] The following describes the database building process performed by the computing device in the embodiment of the present application in conjunction with Figure 5.

[0079] The schematic diagram of the process of establishing a compressed database as shown in FIG5 may specifically include the following steps:

[0080] Step 501: The computing device generates an initial codebook;

[0081] The initial codebook includes multiple initial sub-codebooks; optionally, the structure of the initial sub-codebook is a d*p matrix, and the structure of the initial codebook composed of the multiple initial sub-codebooks is a d*m matrix, where m is greater than p, p is greater than 1, and d is greater than 1. Optionally, the value of m can be an integer multiple of p, and the value of p can be a preset subspace size.

[0082] When a computing device generates an initial codebook, it can be implemented in a variety of different ways:

[0083] One possible implementation is to select object data of m sample objects from the object data of candidate objects stored in the original base database to form multiple sample sets; and generate an initial sub-codebook according to the object data of the sample objects included in each sample set.

[0084] During implementation, the computing device may select object data of m sample objects from the original base database, and organize the object data of the m sample objects into multiple sample sets based on the value of p, with each sample set including object data of p sample objects. The value of m is the number of columns corresponding to the codebook in the embodiment of this application, and the value of m is much smaller than the number of candidate objects stored in the original base database.

[0085] Assuming that the value of m is 9 and the preset subspace size is 3, the object data of 9 sample objects are selected from the original base library, and the object data of the 9 sample objects are divided into three groups. The grouping method can be as shown in Figure 6, where the object data of sample objects m1, m2, and m3 form sample set 1, the object data of sample objects m4, m5, and m6 form sample set 2, and the object data of sample objects m7, m8, and m9 form sample set 3.

[0086] During implementation, the computing device may randomly select object data of m sample objects from the original base database. For example, the original base database stores uncompressed object data of multiple candidate objects. For example, the data structure stored in the original base database is shown in FIG2 . The computing device may randomly select m sample objects from n candidate objects, where the value of m is significantly smaller than the value of n.

[0087] When generating the initial sub-codebook, the computing device may, for any sample set, perform orthogonal basis initialization on a matrix composed of object data of sample objects included in the sample set to obtain an initial sub-codebook; wherein the number of columns of the obtained initial sub-codebook is p.

[0088] During implementation, for each sample set, the object data of the sample objects included in the sample set are formed into a matrix, and the formed matrix is ​​initialized with an orthogonal basis to obtain the initial sub-codebook corresponding to the sample set. For example, as shown in Figure 6, the object data of the sample objects m1, m2, and m3 included in sample set 1 are initialized with an orthogonal basis to obtain the initial sub-codebook corresponding to sample set 1; the object data of the sample objects m4, m5, and m6 included in sample set 2 are initialized with an orthogonal basis to obtain the initial sub-codebook corresponding to sample set 2; and the object data of the sample objects m7, m8, and m9 included in sample set 3 are initialized with an orthogonal basis to obtain the initial sub-codebook corresponding to sample set 3.

[0089] When the computing device of the embodiment of the present application performs orthogonal basis initialization on the matrix composed of the object data included in the sample set, the orthogonal basis of the matrix can be obtained in a variety of different ways, the obtained orthogonal basis is used as the initialization matrix of the matrix, and the obtained initialization matrix is ​​used as the initial sub-codebook corresponding to the sample set.

[0090] The orthogonal basis can be initialized using QR decomposition. The principle of QR decomposition is that, for a given matrix A, it can be decomposed into the product of an orthogonal matrix Q and an upper triangular matrix R, i.e., A = QR. The orthogonal matrix Q can be used as the orthogonal basis to initialize the matrix. Alternatively, the orthogonal basis can be initialized using diagonalization. The principle of diagonalization is that, for certain special types of matrices (such as symmetric matrices or positive definite matrices), an orthogonal basis can be obtained through diagonalization. The eigenvectors of the symmetric matrix form an orthogonal basis to initialize the matrix.

[0091] When the computing device generates the initial codebook, another possible implementation method is to pre-set multiple sample data (the sample data may not be the object data in the original base library), divide the multiple sample data into sample sets, and randomly initialize the matrix composed of the sample data in the sample set to obtain each initial sub-codebook; and each obtained initial sub-codebook is combined into an initial codebook.

[0092] Step 502: The computing device generates a codebook for compressing and encoding the object data of the candidate object stored in the original base library, and a codeword after the compression encoding, based on each initial sub-codebook in the initial codebook.

[0093] In one possible implementation, the computing device determines, based on the subspace distance between the object data of each sample object and each initial sub-codebook, a sample object dataset corresponding to each initial sub-codebook; for any initial sub-codebook, generating, based on the sample object dataset corresponding to the initial sub-codebook, an updated sub-codebook and a corresponding codeword block corresponding to the initial sub-codebook; generating a codebook based on the updated sub-codebook corresponding to each initial sub-codebook; and generating a codeword based on the codeword block corresponding to each initial sub-codebook.

[0094] The following describes a process in which the computing device determines, based on the subspace distance between the object data included in each sample set and each initial subcodebook, a sample object data set corresponding to each initial subcodebook.

[0095] Step 1a: The computing device determines the subspace distance between the object data of each sample object in each sample set and each initial sub-codebook;

[0096] Here, the object data of m sample objects are selected from the original base library and the sample set is divided as an example; as shown in Figure 7, the object data of m candidate samples and m / p initial sub-codebooks, where the dimension of the initial sub-codebook is d*p. The subspace distance between the object data of each sample object and each initial sub-codebook is determined separately. An optional way to determine the subspace distance is to determine the subspace distance based on the product between the vector (with a dimension of 1*d) composed of the object data of a sample object and an initial sub-codebook and the initial sub-codebook for the object data of a sample object; as shown in Figure 7, the product between the 1*d vector composed of the object data of a sample object and the d*p initial sub-codebook is a 1*p vector. The computing device can calculate the L2 norm of the 1*p vector to obtain the subspace distance.

[0097] Step 2a: The computing device determines, from the multiple initial sub-codebooks, an initial sub-codebook corresponding to the object data of the sample object based on the subspace distance between the object data of the sample object and each initial sub-codebook;

[0098] For object data of any sample object, the subspace distance between the object data of the sample object and each initial subcodebook is determined in step 1. In one possible implementation, the initial subcodebook with the smallest subspace distance is bound to the sample object, thereby obtaining an initial subcodebook corresponding to the object data of the sample object.

[0099] Step 3a: The computing device determines, based on the correspondence between the object data of the sample objects and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook, where the sample object data set includes the object data of the sample objects corresponding to the initial sub-codebook.

[0100] Based on the initial sub-codebook corresponding to the object data of each sample object obtained in step 2a, a correspondence between the object data of the sample object and the initial sub-codebook can be generated. Based on the generated correspondence between the object data of the sample object and the initial sub-codebook, a sample object dataset corresponding to each initial sub-codebook can be obtained.

[0101] Exemplarily, assuming that the value of m is 9, the number of initial sub-codebooks obtained based on the object data of 9 sample objects is 3; if the correspondence between the object data of the sample objects generated based on step 2a and the initial sub-codebook is as shown in Table 1, then the sample object data set corresponding to the initial sub-codebook 1 includes the object data of the sample object 1, the object data of the sample object 3, and the object data of the sample object 7; the sample object data set corresponding to the initial sub-codebook 2 includes the object data of the sample object 2, the object data of the sample object 5, and the object data of the sample object 9; and the sample object data set corresponding to the initial sub-codebook 3 includes the object data of the sample object 4, the object data of the sample object 6, and the object data of the sample object 8.

[0102] Table 1

[0103] After determining the sample object data set corresponding to each initial sub-codebook, the computing device generates, for any initial sub-codebook, an updated sub-codebook corresponding to the initial sub-codebook and a corresponding codeword block according to the sample object data set corresponding to the initial sub-codebook.

[0104] Optionally, the computing device may perform PCA dimensionality reduction processing on the sample object data set corresponding to the initial sub-codebook to generate an updated sub-codebook corresponding to the initial sub-codebook and corresponding codeword blocks.

[0105] During implementation, when performing PCA dimensionality reduction processing on the sample object dataset corresponding to the initial sub-codebook, the computing device combines the sample object dataset corresponding to the initial sub-codebook into a matrix, performs PCA dimensionality reduction processing on the combined matrix, and obtains an updated sub-codebook and codeword blocks corresponding to the initial sub-codebook. The updated sub-codebook corresponds to a preset subspace size, the number of rows in the updated sub-codebook is the dimension d of the object data, and the number of columns in the updated sub-codebook is the preset subspace size p. The number of rows in the obtained codeword blocks is the preset subspace size p, and the number of columns in the codeword blocks is the number of sample objects included in the sample object dataset.

[0106] For example, assuming that the sample object dataset corresponding to the initial sub-codebook includes object data of x sample objects, the sample object dataset corresponding to the initial sub-codebook forms a d*x matrix. As shown in FIG8 , the process of performing PCA dimensionality reduction on the matrix formed by the sample object dataset corresponding to the initial sub-codebook reduces the d*x matrix into a d*p sub-codebook and a p*x codeword block.

[0107] It should be noted that after obtaining the updated sub-codebook and codeword block, the computing device of the embodiment of the present application can repeatedly execute the above-mentioned process of updating the sub-codebook and the corresponding codeword block according to the sample object data set based on the updated sub-codebook, and continuously update the sub-codebook and codeword block until the determined sub-codebook and codeword block meet the conditions.

[0108] The computing device of the embodiment of the present application may determine whether each updated sub-codebook and codeword block meets the conditions based on the loss function by minimizing the least squares reconstruction loss function;

[0109] The minimized least squares reconstruction loss function in the embodiment of the present application can be defined as follows:

[0110] Wherein, C is a dense codebook matrix, whose dimension can be d*m; B is a structured block sparse codeword matrix, whose dimension can be m*n; A is a matrix composed of object data of multiple candidate objects in the original base database; Represents a Frobenius norm operation. B includes multiple codebook blocks, and codebook blocks may include non-zero elements. Positions outside of the codebook blocks in B are zero, and each column of B corresponds to a codebook block (i.e., each column of B has only one contiguous region with non-zero elements). Optionally, the positions of non-zero elements in different columns of B are either identical or completely different, and the union of the positions of non-zero elements in different columns covers all dimensions.

[0111] After the computing device generates the updated sub-codebooks and the corresponding codeword blocks corresponding to the initial sub-codebooks, a codebook is generated according to the updated sub-codebooks corresponding to each initial sub-codebook; and a codeword is generated according to the codeword blocks corresponding to each initial sub-codebook.

[0112] When a computing device generates a codeword, one possible implementation method is that the computing device divides each codeword into blocks to form a reference codeword, quantizes the reference codeword, and adjusts elements in the quantized reference codeword to generate the codeword.

[0113] During implementation, the computing device may use standard int 8 quantization to quantize the reference codeword; or may use lower bit quantization to quantize the reference codeword, thereby achieving a higher compression rate.

[0114] Optionally, the structure of the codeword generated by the computing device in the embodiment of the present application is a matrix including multiple codebook blocks, the codebook blocks include first elements, and other positions in the matrix except for the multiple blocks are second elements, and each column in the matrix corresponds to a block.

[0115] Exemplarily, the first element is a non-zero element, and the second element is a 0 element. Since the codeword structure only contains non-zero elements in the codebook blocks, and the rest of the codeword contains 0 elements, the non-zero elements in the codeword are concentrated in each codebook block, and each codebook block in the codebook is a dense matrix.

[0116] When adjusting the elements in the reference codeword after quantization processing, the computing device may rearrange each column of the reference codeword according to a non-zero element pattern, thereby generating a codeword.

[0117] As shown in FIG9 , the codeword structure obtained in the embodiment of the present application is a matrix including multiple codeword blocks, and each column in the matrix corresponds to a codeword block, and the positions of different codeword blocks in the column can be exactly the same. Alternatively, as shown in FIG10 , the codeword structure obtained in the embodiment of the present application is a matrix including multiple codeword blocks, and each column in the matrix corresponds to a codeword block, and the positions of different codeword blocks in the column are completely different; the codeword structure shown in FIG10 is a block diagonal matrix.

[0118] Accordingly, based on the rearrangement of the reference codewords, the elements in each column of the original base database storing the object data of multiple candidate objects can also be rearranged in the same manner, so that the matrix corresponding to the original base database is approximately equal to the product of the generated codebook and the codeword. As shown in Figure 11, the matrix A corresponding to the original base database (whose dimension is d*n) is approximately equal to the product of codebook C (whose dimension is d*m) and codeword B, where codeword B is a block diagonal matrix. Codebook C includes multiple sub-codebooks, each sub-codebook has a number of columns equal to the preset subspace size p, and each codeword block of codeword B has a number of rows equal to the preset subspace size p. Each codeword block of codeword B corresponds one-to-one to each sub-codebook in codebook C.

[0119] After the computing device of the embodiment of the present application obtains the codebook and codeword corresponding to the object data of multiple candidate objects in the original base based on the above process, the codeword can be stored in the compressed base, and the base stores data in the form of a matrix. Since the object data of multiple candidate objects can be approximately equal to the product of the codebook and the codeword, when other devices read the codewords stored in the target database, the object data of multiple candidate objects can be restored based on the codebook. As can be seen from the above introduction, the codeword structure of the embodiment of the present application is a matrix including multiple codeword blocks, and only the first element (exemplarily, the first element is a non-zero element) is included in the codeword block, and the second element (exemplarily, the second element is a 0 element) is included in the other positions of the matrix, thereby greatly reducing the amount of data stored in the target database.

[0120] When storing codewords in a base database, the base database may be divided into multiple storage areas. Optionally, the storage areas in the embodiment of the present application may be buckets, where a bucket stores data with a high degree of similarity.

[0121] In implementation, after obtaining the codebooks and codewords corresponding to the object data of multiple candidate objects, a computing device may store the codewords in a base database. One possible implementation is to use each of the multiple storage areas to store a codeword block. As shown in Figure 12, each codeword block in the codeword structure is sequentially stored in different storage areas.

[0122] In the database building process provided by the embodiment of the present application, the original base of the storage object data is subjected to database compression processing, and the object data of multiple candidate objects are compressed and encoded into code words, and the structure of the code word is a matrix including multiple code word blocks. Since the non-zero elements in the code words of the embodiment of the present application are concentrated in the code word blocks, the various code word blocks in the code word are dense matrices, so that when the computing device reads the code word and performs arithmetic processing, the computing speed of the dense matrix is ​​much higher than other computing forms; in addition, since each column in the code word matrix corresponds to a code word block, that is, each column in the code word matrix only includes an area with non-zero elements, based on this code word structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device. In summary, the code word structure of the embodiment of the present application can greatly improve the computing speed and memory access speed of the computing device.

[0123] 2. Retrieval stage.

[0124] During the retrieval phase, the retrieval process may be executed by a computing device; wherein the computing device executing the retrieval process and the computing device executing the database building process during the database building phase may be different computing devices.

[0125] The following describes the retrieval process performed by the computing device in the embodiment of the present application in conjunction with Figure 13.

[0126] Step 1301: The computing device receives a query request and determines a query vector corresponding to the query request.

[0127] After receiving a query request, the computing device determines a query vector based on the query content. Optionally, the computing device may extract features from the query content based on a feature extraction model to generate the query vector. For example, if the query request is for products similar to a product in a query image, and the query content obtained by the computing device is an image, the computing device extracts product features from the image to generate the query vector.

[0128] Step 1302: The computing device retrieves a target object that matches the query request from a plurality of candidate objects stored in the original database according to the query vector, the codeword, and the codebook.

[0129] The codewords used by the computing device in step 1302 are obtained by compressing and encoding the object data of multiple candidate objects through a codebook. The codebook and codewords may be those generated in the database building phase.

[0130] Since the embodiment of the present application stores each codeword block in multiple storage areas of the base database, when the computing device searches for a target object that matches a query request, it first determines the target storage area from the multiple storage areas, and then searches for the target object that matches the query request based on the codeword blocks stored in the target storage area.

[0131] In one possible implementation, the computing device determines a target storage area from multiple storage areas based on a query vector and a codebook; and retrieves a target object matching the query request from multiple candidate objects based on codeword blocks stored in the target storage area.

[0132] The following describes the process of the computing device determining the target storage area and the process of retrieving the target object based on the codeword blocks stored in the target storage area.

[0133] 1. The computing device determines a target storage area from multiple storage areas in the target database according to the query vector and the codebook.

[0134] The codebook of the embodiment of the present application includes multiple sub-codebooks, wherein the size of the sub-codebook is determined according to a preset subspace size. For example, if the preset subspace size is p, then one sub-codebook corresponds to p candidate objects, that is, the dimension of the matrix corresponding to one sub-codebook is d*p, where d is the feature dimension of the object data of the candidate object.

[0135] In one possible implementation, the computing device may determine the target storage area according to the following method:

[0136] Step 1b: The computing device determines, based on the query vector and the codebook, a subspace distance between the query vector and each sub-codebook in the codebook.

[0137] Optionally, the computing device may determine the first similarity information between the query vector and the sub-codebook according to the following formula:

[0138] The first similarity information is used to represent the similarity between the query vector and the sub-codebook.

[0139] In implementation, the computing device may determine the first similarity information between the query vector and the sub-codebook in a variety of different ways:

[0140] Method 1: A computing device determines a product of a query vector and a codebook to obtain an intermediate vector; divides the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook; and determines first similarity information between the query vector and each sub-codebook in the codebook based on the sub-intermediate vector corresponding to each sub-codebook.

[0141] In this manner, a larger value of the first similarity information determined by the computing device indicates a higher similarity between the query vector and the sub-codebook.

[0142] For example, if the feature dimension of the query vector is d, the query vector can be represented as a 1*d matrix, and the codebook structure is a d*m matrix. Then, the query vector and the codebook are multiplied to obtain an intermediate vector of a 1*m matrix. Based on the size of the sub-codebook, the intermediate vector is divided into multiple sub-intermediate vectors, and the divided sub-intermediate vectors can be a 1*p matrix. When the computing device determines the first similarity information between the query vector and each sub-codebook in the codebook based on the sub-intermediate vector corresponding to each sub-codebook, the L2 norm of the 1*p matrix can be used as the first similarity information.

[0143] In approach 2, the computing device determines a subspace distance between the query vector and each sub-codebook in the codebook, and uses the subspace distance as first similarity information between the query vector and each sub-codebook in the codebook.

[0144] Optionally, the computing device may calculate the subspace distance between the query vector and the sub-codebook according to the following formula:

[0145] Where D is the subspace distance between the query vector and the subcodebook, q is the query vector (feature dimension is d), A k is the matrix corresponding to the sub-codebook (its dimension is d*p).

[0146] In this manner, the smaller the value of the first similarity information determined by the computing device is, the higher the similarity between the query vector and the sub-codebook is.

[0147] Step 2b: The computing device determines a target sub-codebook from the multiple sub-codebooks included in the codebook according to the first similarity information between the query vector and each sub-codebook in the codebook.

[0148] If, in step 2b, the computing device uses approach 1 to determine the first similarity information between the query vector and each sub-codebook in the codebook, the computing device selects k0 sub-codebooks with large first similarity information values ​​from the multiple sub-codebooks included in the codebook as target sub-codebooks. If, in step 2b, the computing device uses approach 2 to use the subspace distance between the query vector and each sub-codebook in the codebook as the first similarity information, the computing device selects k0 sub-codebooks with small subspace distances from the multiple sub-codebooks included in the codebook as target sub-codebooks.

[0149] Step 3b: The computing device determines the target codeword block corresponding to the target subcodebook according to the correspondence between the subcodebook and the codeword block, and uses the storage area storing the codeword block corresponding to the target codeword block as the target storage area.

[0150] After the computing device determines the target sub-codebook from the codebook, since the sub-codebooks correspond one-to-one to the codeword blocks in the codeword structure, the computing device determines the target codeword block in the codeword structure that corresponds to the target sub-codebook. Furthermore, since each storage area in the target database stores a codeword block, the storage area storing the target codeword block is determined as the target storage area.

[0151] As shown in Figure 14, after the computing device determines the first similarity information between the query vector and each sub-codebook in the codebook, it sorts the multiple storage areas of the target database according to the first similarity information. In implementation, if the first similarity information is determined using method 1 above, the storage areas are sorted in descending order based on the values ​​of the first similarity information; if the first similarity information is determined using method 2 above, the storage areas are sorted in ascending order based on the values ​​of the first similarity information. The top k0 storage areas in the sorting order are selected from the multiple storage areas as the target storage areas.

[0152] 2. The computing device retrieves a target object that matches the query request from multiple candidate objects based on the codeword blocks stored in the target storage area.

[0153] In one possible implementation, the computing device determines, based on the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area, where the second similarity information is used to characterize the degree of similarity between the query vector and each candidate object corresponding to the codeword block stored in the target storage area; and the computing device determines the target object from the candidate objects corresponding to the target sub-codebook based on the second similarity information.

[0154] In implementation, the computing device may determine the second similarity information in a variety of different ways:

[0155] Method 1: The computing device determines the product of the sub-intermediate vector corresponding to the target sub-codebook and the codeword block stored in the target storage area, and determines, based on the product, second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area.

[0156] For example, assuming that the target sub-codebook is a d*p matrix and the codeword blocks stored in the target storage area are a p*x matrix, the product of the target sub-codebook and the codeword blocks stored in the target storage area is a d*x matrix; each column in the d*x matrix corresponds to a candidate object. Optionally, the computing device can determine the L2 norm for each column of the d*x matrix to obtain second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area.

[0157] After the computing device obtains the second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area based on the above-mentioned method 1, it selects k1 candidate objects with the largest second similarity information values ​​from the candidate objects corresponding to the codeword blocks stored in the target storage area as target objects based on the second similarity information.

[0158] Method 2: The computing device determines, based on the codeword blocks stored in the target storage area, an adjustment weight for each candidate object corresponding to the codeword blocks stored in the target storage area; and determines a product of a sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, and determines, based on the product and each adjustment weight, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area.

[0159] The following is a detailed description of the method 2 for determining the second similarity information.

[0160] Step 1c: The computing device determines, based on the codeword blocks stored in the target storage area, an adjustment weight for each candidate object corresponding to the codeword blocks stored in the target storage area;

[0161] During implementation, the computing device determines the L2 norm of each column of the codeword block stored in the target storage area; determines the adjustment weight of the candidate object based on the calculation result of the L2 norm; illustratively, if the codeword block stored in the target storage area is a p*x matrix, then for each column of the matrix, the L2 norm is determined, thereby obtaining x adjustment weights; optionally, assuming that the L2 norm of each column of the codeword block is y, 0.5*y is added. 2The adjusted weight of the candidate object corresponding to this column.

[0162] Step 2c: The computing device determines the product of the sub-intermediate vector corresponding to the target sub-codebook and the codeword block stored in the target storage area;

[0163] Exemplarily, assuming that the target sub-codebook is a d*p matrix and the codeword blocks stored in the target storage area are a p*x matrix, the product of the target sub-codebook and the codeword blocks stored in the target storage area is a d*x matrix.

[0164] Step 3c: The computing device determines, based on the product and each adjusted weight, second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area;

[0165] If the product obtained in step 2c is a d*x matrix, the computing device determines the L2 norm of each column of the matrix and uses the L2 norm of each column as the similarity corresponding to the column; the computing device negates the similarity and uses the sum of the similarity after negation and the corresponding adjustment parameter as the second similarity information.

[0166] After the computing device obtains the second similarity information between the query vector and each candidate object corresponding to the codeword block stored in the target storage area based on the above-mentioned method 2, it selects k1 candidate objects with the smallest second similarity information values ​​from the candidate objects corresponding to the codeword blocks stored in the target storage area as target objects based on the second similarity information.

[0167] After the computing device in the embodiment of the present application determines the target object that matches the query request from multiple candidate objects, it can use the identification information of the target object as the retrieval result.

[0168] The retrieval method provided by the embodiment of the present application is that when the computing device obtains the query vector and retrieves the retrieval results matching the query vector from the database, the computing device searches from the target database after compression coding; and the target database of the embodiment of the present application stores the codeword obtained after compression coding of the original object data, and the codeword structure is specially designed so that the structure of the codeword includes a matrix of multiple codeword blocks, and each column in the matrix corresponds to a codeword block. The non-zero elements in the codeword of the embodiment of the present application are concentrated in the codeword blocks, and each codeword block in the codeword is a dense matrix. Since the operation speed of the dense matrix is ​​much higher than other operation forms, the computing device can significantly improve the operation speed based on the dense codeword matrix during the retrieval process, thereby improving the retrieval efficiency; in addition, since each column in the codeword matrix corresponds to a codeword block, that is, each column in the codeword matrix only includes an area with non-zero elements, based on this codeword structure, data can be stored in continuous memory blocks, and continuous memory block access can greatly improve the memory access speed of the computing device, thereby further improving the retrieval efficiency.

[0169] Based on the above content and the same technical concept, an embodiment of the present application provides a data processing device 1500 . As shown in FIG15 , the data processing device 1500 includes an initialization unit 1501 and a processing unit 1502 .

[0170] Initialization unit 1501 is configured to generate an initial codebook, wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes multiple initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, where m is greater than p, p is greater than 1, and d is greater than 1;

[0171] A processing unit 1502 is configured to generate, based on each initial sub-codebook in the initial codebook, a codebook for compressing and encoding object data of candidate objects stored in an original base library, and a codeword after the compression encoding; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, the codeword blocks including first elements, positions other than the plurality of codeword blocks in the matrix as second elements, and each column in the matrix corresponding to a codeword block.

[0172] In a possible implementation, the initialization unit 1501 is specifically configured to:

[0173] Selecting m sample objects' object data from the object data of the candidate objects stored in the original database to form multiple sample sets, where each sample set includes p sample objects' object data;

[0174] Each initial sub-codebook included in the initial codebook is generated according to object data of the sample object included in each sample set.

[0175] In a possible implementation, the processing unit 1502 is specifically configured to:

[0176] Determine, according to the subspace distance between the object data of each sample object and each initial subcodebook, the sample object data set corresponding to each initial subcodebook;

[0177] For any initial sub-codebook, generating an updated sub-codebook and corresponding codeword blocks corresponding to the initial sub-codebook according to the sample object data set corresponding to the initial sub-codebook;

[0178] Generating the codebook according to the updated sub-codebook corresponding to each initial sub-codebook; and generating the codeword according to the codeword block corresponding to each initial sub-codebook.

[0179] In a possible implementation, the processing unit 1502 is specifically configured to:

[0180] For object data of any sample object, determining an initial subcodebook corresponding to the object data of the sample object from a plurality of initial subcodebooks according to a subspace distance between the object data of the sample object and each initial subcodebook;

[0181] According to the correspondence between the object data of the sample objects and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook is determined respectively, where the sample object data set includes the object data of the sample objects corresponding to the initial sub-codebook.

[0182] In a possible implementation, the processing unit 1502 is specifically configured to:

[0183] Divide the codewords corresponding to each initial sub-codebook into blocks to form reference codewords;

[0184] The reference codeword is quantized, and elements in the quantized reference codeword are adjusted to generate the codeword.

[0185] In a possible implementation, the structure of the codeword is a block diagonal matrix.

[0186] In one possible implementation, an embodiment of the present application further provides a data processing device 1600, as shown in FIG16 , the data processing device 1600 includes a receiving unit 1601 and a retrieval unit 1602; wherein the data processing device 1600 and the data processing device 1500 as shown in FIG15 can be deployed on one device, or the data processing device and the data processing device as shown in FIG15 can be deployed separately.

[0187] The receiving unit 1601 is configured to receive a query request and determine a query vector corresponding to the query request;

[0188] The retrieval unit 1602 is configured to retrieve a target object matching the query request from a plurality of candidate objects stored in the original database according to the query vector, the codeword, and the codebook.

[0189] In a possible implementation, each codeword in the codeword is divided into blocks and stored in different storage areas.

[0190] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0191] determining a target storage area according to the query vector and the codebook;

[0192] According to the codeword blocks stored in the target storage area, a target object matching the query request is retrieved from the multiple candidate objects.

[0193] In a possible implementation, the codebook includes multiple sub-codebooks, each sub-codebook corresponds to a codeword block;

[0194] The retrieval unit 1602 is specifically configured to:

[0195] determining, based on the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, wherein the first similarity information is used to represent a degree of similarity between the query vector and the sub-codebook;

[0196] determining a target subcodebook from a plurality of subcodebooks included in the codebook according to first similarity information between the query vector and each subcodebook in the codebook;

[0197] According to the correspondence between the sub-codebook and the block, the target codeword block corresponding to the target sub-codebook is determined, and the storage area storing the target codeword block is used as the target storage area.

[0198] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0199] Determine a product of the query vector and the codebook to obtain an intermediate vector;

[0200] Dividing the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook respectively;

[0201] Determine, according to the sub-intermediate vector corresponding to each sub-codebook, first similarity information between the query vector and each sub-codebook in the codebook.

[0202] In a possible implementation, the retrieval unit 1602 is specifically configured to:

[0203] determining, based on the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, wherein the second similarity information is used to represent a degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area;

[0204] The target object is determined from candidate objects corresponding to the target sub-codebook according to the second similarity information.

[0205] Figure 17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. This computing device is used to execute the methods performed by the computing devices in the method embodiments of Figures 5 and 13 above. The computing device 1700 includes a processor 1701, a memory 1702, and a communication interface 1703. The processor 1701, the memory 1702, and the communication interface 1703 may be connected via a bus 1704.

[0206] The processor 1701 is used to execute instructions stored in the memory 1702 so that the data processing device 1500 or the data processing device 1600 executes the data processing method provided in this application. The processor 1701 can be, but is not limited to, any one or more of: a central processing unit (CPU), a data processing unit (DPU), a system on chip (SOC), a field programmable gate array (FPGA), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or an accelerator card.

[0207] Memory 1702 is used to store computer instructions and data, such as memory 1702 storing computer instructions and data required to implement the data processing method provided in this application. Memory 1702 includes volatile memory, such as random access memory (RAM), dynamic random access memory (DRAM), etc. It can also be non-volatile memory, such as read-only memory (ROM), storage-class memory (SCM), flash memory, hard disk drive (HDD) or solid state drive (SSD).

[0208] The memory 1702 stores executable program code, and the processor 1701 executes the executable program code to implement the aforementioned data processing functions. The memory 1702 stores instructions for the data processing device 1500 and / or the data processing device 1600 to execute the data processing method provided in this application.

[0209] The communication interface 1703 is used to communicate with internal or external devices, such as obtaining read requests / write requests sent by a process, or communicating with a storage device to complete data access. For example, the communication interface 1703 can be a network card.

[0210] Bus 1704 may be a Peripheral Component Interconnect Express (PCIe) bus, a Double Data Rate (DDR) bus, a Serial Advanced Technology Attachment (SATA) bus, a Serial Attached SCSI (SAS) bus, a Controller Area Network (CAN) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (UBus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), or the like. Buses may be classified as address buses, data buses, or control buses. For ease of illustration, FIG17 shows only one line, but this does not indicate that there is only one bus or only one type of bus. Bus 1704 may include a path for transmitting information between each component of data processing device 1700 (e.g., memory 1702, processor 1701, and communication interface 1703).

[0211] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the data processing method executed by the computing device in the embodiment of FIG. 5 or FIG. 13 . See the description of each step in FIG. 5 or FIG. 13 , which will not be repeated here.

[0212] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the data processing method executed by the computing device in the embodiment of Figure 5 or Figure 13 above. See the description of each step of Figure 5 or Figure 13, which will not be repeated here.

[0213] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0214] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or the design of any combination thereof. The general-purpose processor can be a microprocessor, alternatively, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0215] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, software units executed by a processor, or a combination of the two. The software units can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or other storage media in any form known in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and storage medium can be arranged in an ASIC.

[0216] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0217] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, the present application is intended to include such modifications and variations if they fall within the scope of the claims of the present application and their equivalents.

Claims

1. A data processing method, characterized in that: The method comprises: Generate an initial codebook; wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, the m is greater than p, the p is greater than 1, and the d is greater than 1; According to each initial sub-codebook in the initial codebook, a codebook for compressing and encoding object data of candidate objects stored in an original base library and a codeword after compression encoding are generated; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, the codeword blocks include a first element, other positions in the matrix except the plurality of codeword blocks are second elements, and each column in the matrix corresponds to a codeword block.

2. The method according to claim 1, characterized in that The generating of the initial codebook comprises: Selecting m sample objects' object data from the object data of the candidate objects stored in the original base database to form a plurality of sample sets, wherein each sample set includes p sample objects' object data; Each initial sub-codebook included in the initial codebook is generated according to object data of the sample object included in each sample set.

3. The method according to claim 2, characterized in that The step of generating, according to each initial sub-codebook in the initial codebook, a codebook for compressing and encoding the object data of the candidate object stored in the original base library, and a codeword after the compression encoding, comprises: Determine, according to the subspace distance between the object data of each sample object and each initial subcodebook, the sample object data set corresponding to each initial subcodebook; For any initial sub-codebook, generating an updated sub-codebook and corresponding codeword blocks corresponding to the initial sub-codebook according to a sample object data set corresponding to the initial sub-codebook; The codebook is generated according to the updated sub-codebook corresponding to each initial sub-codebook; and the codeword is generated according to the codeword block corresponding to each initial sub-codebook.

4. The method according to claim 3, characterized in that The step of determining the sample object data set corresponding to each initial sub-codebook according to the subspace distance between the object data of each sample object and each initial sub-codebook comprises: For object data of any sample object, determining an initial subcodebook corresponding to the object data of the sample object from a plurality of initial subcodebooks according to a subspace distance between the object data of the sample object and each initial subcodebook; According to the correspondence between the object data of the sample object and the initial sub-codebook, a sample object data set corresponding to each initial sub-codebook is determined respectively, wherein the sample object data set includes the object data of the sample object corresponding to the initial sub-codebook.

5. The method according to claim 3, characterized in that The generating the codeword according to the codeword blocks corresponding to each initial sub-codebook includes: Divide the codewords corresponding to each of the initial sub-codebooks into blocks to form reference codewords; The reference codeword is quantized, and elements in the quantized reference codeword are adjusted to generate the codeword.

6. The method according to any one of claims 1 to 5, characterized in that: The structure of the codeword is a block diagonal matrix.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: receiving a query request, and determining a query vector corresponding to the query request; According to the query vector, the codeword and the codebook, a target object matching the query request is retrieved from a plurality of candidate objects stored in the original base database.

8. The method according to claim 7, characterized in that Each codeword in the codeword is divided into blocks and stored in different storage areas respectively.

9. The method according to claim 8, characterized in that The step of retrieving a target object matching the query request from a plurality of candidate objects stored in the original base database according to the query vector, the codeword, and the codebook includes: Determine a target storage area according to the query vector and the codebook; According to the codeword blocks stored in the target storage area, a target object matching the query request is retrieved from the multiple candidate objects.

10. The method according to claim 9, characterized in that The codebook includes a plurality of sub-codebooks, each sub-codebook corresponding to a codeword block; The determining a target storage area according to the query vector and the codebook includes: Determine, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook, wherein the first similarity information is used to represent a degree of similarity between the query vector and the sub-codebook; Determining a target subcodebook from a plurality of subcodebooks included in the codebook according to first similarity information between the query vector and each subcodebook in the codebook; According to the correspondence between the sub-codebook and the codeword block, the target codeword block corresponding to the target sub-codebook is determined, and the storage area storing the target codeword block is used as the target storage area.

11. The method according to claim 10, characterized in that The determining, according to the query vector and the codebook, first similarity information between the query vector and each sub-codebook in the codebook includes: Determine a product of the query vector and the codebook to obtain an intermediate vector; Dividing the intermediate vector into sub-intermediate vectors corresponding to each sub-codebook respectively; Determine first similarity information between the query vector and each sub-codebook in the codebook according to the sub-intermediate vector corresponding to each sub-codebook.

12. The method according to claim 11, characterized in that The step of determining a target object matching the query request from the plurality of candidate objects according to the codeword blocks stored in the target storage area comprises: Determining, according to the sub-intermediate vector corresponding to the target sub-codebook and the codeword blocks stored in the target storage area, second similarity information between the query vector and each candidate object corresponding to the codeword blocks stored in the target storage area, wherein the second similarity information is used to represent the degree of similarity between the query vector and the candidate objects corresponding to the codeword blocks stored in the target storage area; The target object is determined from candidate objects corresponding to the target sub-codebook according to the second similarity information.

13. A data processing device, characterized in that: The device comprises: An initialization unit, configured to generate an initial codebook, wherein the structure of the initial codebook is a d*m matrix, the initial codebook includes a plurality of initial sub-codebooks, the structure of each initial sub-codebook is a d*p matrix, the m is greater than p, the p is greater than 1, and the d is greater than 1; A processing unit generates, according to each initial sub-codebook in the initial codebook, a codebook for compressing and encoding object data of a candidate object stored in an original base library, and a codeword after the compression encoding; wherein the structure of the codeword is a matrix including a plurality of codeword blocks, the codeword blocks include a first element, other positions in the matrix except the plurality of codeword blocks are second elements, and each column in the matrix corresponds to a codeword block.

14. A computing device, characterized in that The computing device comprises a memory and at least one processor, wherein the memory is used to store a set of computer program instructions, and when the processor executes the set of program instructions, the method according to any one of claims 1 to 12 is performed.

15. A computer-readable storage medium, characterized in that: When the computer-readable storage medium is executed by a storage device, the storage device executes the method according to any one of claims 1 to 12.

Images