An image encoding method and apparatus, and a storage medium

Abstract

Embodiments of the present application disclose an image coding method and device, and a storage medium. The image coding device trains an initial feature extraction model by using first training data, obtains a target feature extraction model; trains an initial clustering model by using second training data, obtains a first clustering model; and performs image coding processing on image data according to the target feature extraction model and the first clustering model, to obtain coded data corresponding to the image data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image coding, in particular to an image coding method and device, and a storage medium. BACKGROUND

[0002] Image coding technology refers to representing an image or information contained in the image with fewer bits under the premise of meeting certain accuracy. Existing image coding technology mainly captures images based on image pixel level features, structures, edge information, etc., and can better resist geometric deformation or brightness changes, but the existing image coding technology cannot detect the most meaningful part of the image and has little connection with the high-level semantics of the image.

[0003] Therefore, the existing image coding technology has poor image recognition capability, and the image coding obtained based on the existing image coding technology cannot well represent deep feature information of the image. SUMMARY

[0004] The embodiments of the present application provide an image coding method and device, and a storage medium, which can better utilize image coding to represent deep feature information of the image and improve the recognition capability of the image.

[0005] The technical scheme of the embodiments of the present application is as follows:

[0006] In a first aspect, the embodiments of the present application provide an image coding method, which comprises:

[0007] training an initial feature extraction model using first training data to obtain a target feature extraction model;

[0008] training an initial clustering model using second training data to obtain a first clustering model;

[0009] performing image coding processing on image data according to the target feature extraction model and the first clustering model to obtain coded data corresponding to the image data.

[0010] In a second aspect, the embodiments of the present application provide a terminal, which comprises a training unit and a coding unit,

[0011] The training unit is configured to train an initial feature extraction model using first training data to obtain a target feature extraction model, and train an initial clustering model using second training data to obtain a first clustering model.

[0012] The coding unit is configured to perform image coding processing on image data according to the target feature extraction model and the first clustering model to obtain coded data corresponding to the image data.

[0013] In a third aspect, an embodiment of the present application provides an image coding device, the image coding device further comprising a processor, and a memory storing instructions executable by the processor, when the instructions are executed by the processor, implementing the image coding method as described above.

[0014] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, having stored thereon a program, applied in an image coding device, when the program is executed by a processor, implementing the image coding method as described above.

[0015] The embodiments of the present application provide an image coding method and device, and a storage medium. The image coding device trains an initial feature extraction model by using first training data, to obtain a target feature extraction model; trains an initial clustering model by using second training data, to obtain a first clustering model; and performs image coding processing on image data according to the target feature extraction model and the first clustering model, to obtain coded data corresponding to the image data. That is, in the embodiments of the present application, the image coding device trains an initial feature extraction model and an initial clustering model to obtain a target feature extraction model and a first clustering model, and then codes image data according to the target feature extraction model and the first clustering model, so that the coded data has better feature expression capability. It can be seen that the target feature extraction model and the first clustering model constructed by the present application have better image coding capability, and the recognition capability of the image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 Implementation process of the image coding method provided by the embodiments of the present application Figure One

[0017] Figure 2 Implementation process of the image coding method provided by the embodiments of the present application Figure Two

[0018] Figure 3 Implementation process of the image coding method provided by the embodiments of the present application Figure Three

[0019] Figure 4 Implementation process of the image coding method provided by the embodiments of the present application Figure Four

[0020] Figure 5 Implementation process of the image coding method provided by the embodiments of the present application Figure Five

[0021] Figure 6 Implementation process of the image coding method provided by the embodiments of the present application Figure Six

[0022] ​​​​​​Figure 7 Implementation flow of the image encoding method proposed by the embodiment of the present application Figure Seven

[0023] Figure 8 Implementation flow of the image encoding method proposed by the embodiment of the present application Figure Eight ;

[0024] Figure 9 Implementation architecture of the image encoding method proposed by the embodiment of the present application Figure One ;

[0025] Figure 10 Implementation architecture of the image encoding method proposed by the embodiment of the present application Figure Two ;

[0026] Figure 11 Implementation architecture of the image encoding method proposed by the embodiment of the present application Figure Three ;

[0027] Figure 12 Composition structure of the image encoding device proposed by the embodiment of the present application Figure One ;

[0028] Figure 13 Composition structure of the image encoding device proposed by the embodiment of the present application Figure Two .DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings.

[0030] Image encoding refers to a technology for representing an image or information contained in the image with fewer bits under the premise of meeting a certain precision. Existing image encoding technologies include image compression technologies based on high-precision analog-digital conversion and signal encoding, such as prediction encoding, transform domain encoding, etc.; hash algorithms for measuring image similarity, such as average hash, perceptual hash, differential hash, etc.; and image processing related algorithms based on texture features, edge information extraction, such as scale-invariant feature transform, histogram of oriented gradients, and local binary, etc.

[0031] Existing image encoding technologies mainly capture the complex information of images based on the features or textures, structures, edge information, etc. at the pixel level of the images, and thus have good ability to resist geometric deformation or brightness change. However, the existing image encoding technologies usually cannot detect the most meaningful part of the image, and have little connection with the high-level semantics of the image.​

[0032] When facing a scenario containing an image of a hundred million level, for example, in an e-commerce scenario, a certain search result page can contain a large number of same main image goods, and some tail merchants steal the hot-selling main image of the head merchant, and upload it as the main image of their own goods by modifying the logo, cropping the background, or splicing multiple images. Based on the existing image coding technology, it is not possible to effectively determine whether there is a suspicion of plagiarism between the pictures of the search result page, and thus it is not possible to effectively manage such cases.

[0033] In order to solve the problems existing in the prior art, the embodiment of the present application provides an image coding method, an image coding device and a storage medium. Specifically, the image coding device trains an initial feature extraction model using first training data to obtain a target feature extraction model; trains an initial clustering model using second training data to obtain a first clustering model; and performs image coding processing on image data according to the target feature extraction model and the first clustering model to obtain coded data corresponding to the image data. Precise data transmission can be achieved, and data transmission efficiency can be improved.

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0035] Embodiment one

[0036] The embodiment of the present application provides an image coding method, Figure 1 The implementation process of the image coding method provided by the embodiment of the present application is shown in Figure One As shown in Figure 1 The image coding method can include the following steps:

[0037] Step 101, training an initial feature extraction model using first training data to obtain a target feature extraction model.

[0038] In the embodiment of the present application, the image coding device first trains an initial feature extraction model using first training data to obtain a target feature extraction model.

[0039] It can be understood that, in the embodiment of the present application, the first training data refers to the data for training the initial feature extraction model. Since the image coding device in the present application codes image data, the type of the first training data is image data. For example, the first training data can be a 64x64x3-dimensional image matrix, that is, the data input to the initial feature extraction model is image data.

[0040] Further, in the embodiment of the present application, the initial feature extraction model is a convolutional neural network (CNN). By training the initial feature extraction model, a target feature extraction model can be obtained, and the target feature extraction model is the model for finally extracting image features, so that the image deep semantic level features can be obtained according to the target feature extraction model.

[0041] It should be noted that, in the embodiment of the present application, the target feature extraction model is a trained CNN model, which has strong learning ability and generalization ability, and using the CNN model to process the image can represent the deep semantic level features of the image.

[0042] Step 102, training the initial clustering model by using the second training data to obtain a first clustering model.

[0043] In the embodiment of the present application, the image encoding device can also train the initial clustering model by using the second training data to obtain the first clustering model.

[0044] It can be understood that, in the embodiment of the present application, the second training data refers to the data for training the initial clustering model, and the second training data can be an image expression vector, for example, an image expression vector with a dimension of 256. That is, the data input to the initial clustering model is an image expression vector.

[0045] Further, in the embodiment of the present application, the initial clustering model is a product quantization (PQ), and by training the initial clustering model, a first clustering model can be obtained, so that the dimensionality of the feature information can be reduced according to the first clustering model to speed up the image encoding speed.

[0046] It should be noted that, in the embodiment of the present application, the first clustering model is a trained PQ model, and the PQ model is usually used for image retrieval, and the specific implementation process can be divided into a grouping quantization process and a category Cartesian product process. The PQ algorithm originally used in the field of image retrieval is used to realize image encoding in the present application, which can greatly improve the encoding speed.

[0047] Step 103, performing image encoding processing on the image data according to the target feature extraction model and the first clustering model to obtain encoded data corresponding to the image data.

[0048] In the embodiments of the present application, after the image coding device trains the initial feature extraction model by using the first training data, obtains the target feature extraction model, and trains the initial clustering model by using the second training data, obtains the first clustering model, the image coding device can perform image coding processing on the image data according to the target feature extraction model and the first clustering model, and obtain the coded data corresponding to the image data.

[0049] It should be noted that in the embodiments of the present application, the image coding processing refers to coding the input image. After the image coding processing in the present application, the coding of the image finally obtained has good image representation ability.

[0050] It can be understood that in the embodiments of the present application, the target feature extraction model and the first clustering model obtained by training can be directly applied to the image coding processing, so as to obtain the coded data corresponding to the image data.

[0051] Further, in the embodiments of the present application, the image coding processing on the image data according to the target feature extraction model and the first clustering model mainly includes three processes: performing feature extraction processing on the image data by using the target feature extraction model, then performing clustering processing by using the first clustering model, and finally performing clustering processing again by using the second clustering model.

[0052] Figure 2 Implementation process of the image coding method proposed in the embodiments of the present application Figure Two As shown in Figure 2 training the initial feature extraction model by using the first training data to obtain the target feature extraction model, that is, step 101, can include the following steps:

[0053] Step 101a, input the first training data into the initial feature extraction model to obtain the first feature map, the second feature map and the third feature map corresponding to the first training data.

[0054] In the embodiments of the present application, the image coding device trains the initial feature extraction model by using the first training data to obtain the target feature extraction model. Specifically, the image coding device first inputs the first training data into the initial feature extraction model to obtain the first feature map, the second feature map and the third feature map corresponding to the first training data.

[0055] It should be noted that in the embodiments of the present application, the first feature map, the second feature map and the third feature map refer to data obtained after convolution processing and pooling processing by the initial feature extraction model. The first feature map refers to data information obtained after convolution and pooling processing on the first training data, the second feature map refers to data information obtained after further convolution and pooling processing on the basis of the first feature map, and similarly, the third feature map refers to data information obtained after further convolution and pooling processing on the basis of the second feature map. The spatial dimensions of the first feature map, the second feature map and the third feature map show a decreasing trend.

[0056] Step 101b, calculating the first mutual information loss degree corresponding to the first feature map, the second mutual information loss degree corresponding to the second feature map and the third mutual information loss degree corresponding to the third feature map.

[0057] In the embodiments of the present application, after the image encoding device inputs the first training data into the initial feature extraction model to obtain the first feature map, the second feature map and the third feature map corresponding to the first training data, the image encoding device can calculate the first mutual information loss degree corresponding to the first feature map, the second mutual information loss degree corresponding to the second feature map and the third mutual information loss degree corresponding to the third feature map.

[0058] It should be noted that in the embodiments of the present application, mutual information refers to the amount of information of the feature map obtained after convolution and pooling processing with respect to the input image. Mutual information can be divided into global mutual information and local mutual information. For example, the first mutual information obtained according to the first feature map in the present application and the second mutual information obtained according to the second feature map belong to local mutual information, and the third mutual information obtained according to the third feature map belongs to global mutual information. In the present application, when the sum of the global mutual information and the local mutual information is large enough, it indicates that the output feature map can well represent the feature information of the input first training data.

[0059] It can be understood that in the embodiments of the present application, after obtaining the mutual information of the feature map, the mutual information loss degree corresponding to the feature map can be calculated according to the mutual information, i.e. the first mutual information loss degree corresponding to the first feature map, the second mutual information loss degree corresponding to the second feature map and the third mutual information loss degree corresponding to the third feature map. In the present application, the mutual information loss degree is used to calculate gradient information. The mutual information loss degree can represent the degree of information loss of the feature map after feature extraction processing with respect to the input image.

[0060] It can also be understood that in the embodiments of the present application, when the mutual information loss degree gradually converges, i.e. it basically no longer changes, the training can be stopped.

[0061] Step 101c: Calculate gradient information based on the first mutual information loss, the second mutual information loss, and the third mutual information loss.

[0062] In the embodiments of this application, after calculating the first mutual information loss corresponding to the first feature map, the second mutual information loss corresponding to the second feature map, and the third mutual information loss corresponding to the third feature map, the image encoding device can calculate gradient information based on the first mutual information loss, the second mutual information loss, and the third mutual information loss.

[0063] It is understood that in the embodiments of this application, when training the initial feature extraction model, the method of updating the model adopts the gradient descent algorithm. The core of the gradient descent algorithm is to calculate the gradient information and then update the initial feature extraction model according to the gradient information.

[0064] Step 101d: Update the initial feature extraction model based on the gradient information to obtain the target feature extraction model.

[0065] In the embodiments of this application, after calculating gradient information based on the first mutual information loss, the second mutual information loss, and the third mutual information loss, the image encoding device can update the initial feature extraction model based on the gradient information to obtain the target feature extraction model.

[0066] It is understood that in the embodiments of this application, the gradient information is calculated once each time the initial feature extraction model is updated, and then the model parameters are updated according to the corresponding components in the gradient information to finally obtain the target feature extraction model.

[0067] Figure 3 This is a schematic diagram of the implementation process of the image encoding method proposed in the embodiments of this application. Figure Three ,like Figure 3 As shown, the image encoding device inputs the first training data into the initial feature extraction model to obtain the first feature map, the second feature map, and the third feature map corresponding to the first training data, i.e., step 101a, which may include the following steps:

[0068] Step 101a1: Perform convolution processing on the first training data according to the first part of the convolutional layer to obtain the first information corresponding to the first training data.

[0069] In the embodiments of this application, the image encoding device inputs the first training data into the initial feature extraction model to obtain the first feature map, the second feature map and the third feature map corresponding to the first training data. Specifically, the image encoding device first performs convolution processing on the first training data according to the first part of the convolution layer to obtain the first information corresponding to the first training data.

[0070] It can be understood that, in the embodiments of the present application, since the initial feature extraction model is a convolutional neural network model, the core of which is to extract features from input data through a convolutional layer, therefore, the role of the convolutional layer is feature extraction.

[0071] It should be noted that, in the embodiments of the present application, the first part of the convolutional layer includes at least one convolutional layer, that is, the first part of the convolutional layer can include multiple convolutional layers, and the size of each convolutional layer is 3x3 convolution, and then the number of convolutional processing using each convolutional layer is also multiple. After the first training data is processed by the first part of the convolutional layer, the first information corresponding to the first training data can be obtained.

[0072] It can be understood that, in the embodiments of the present application, the first information refers to the data information obtained by the convolutional processing of the first training data.

[0073] Step 101a2, performing pooling processing on the first information according to the first pooling layer to obtain the first feature map corresponding to the first training data.

[0074] In the embodiments of the present application, after the image coding device obtains the first information corresponding to the first training data by performing convolutional processing on the first training data according to the first part of the convolutional layer, the first information can be processed by the first pooling layer to obtain the first feature map corresponding to the first training data.

[0075] It can be understood that, in the embodiments of the present application, adding a pooling layer in the initial feature extraction model can speed up the calculation and prevent overfitting, that is, the primary role of the pooling layer is downsampling, dimensionality reduction, removing redundant information, and compressing features, thereby reducing the amount of calculation. Therefore, after the first information is obtained, the first information can be processed by the first pooling layer.

[0076] Further, in the embodiments of the present application, the first feature map refers to the data information obtained by the pooling processing of the first information, for example, the first feature map can be a 4x4x256-dimensional image matrix.

[0077] It should be noted that, in the embodiments of the present application, the pooling layer can include an average pooling layer, a maximum pooling layer and a global pooling layer, wherein the average pooling layer can retain background information, slide in the form of a window on the feature map, take the average value in the window as the result, and after the operation, the feature map can be downsampled, reducing the overfitting phenomenon; the maximum pooling layer can extract feature texture and reduce the influence of useless information; the global pooling layer is used to obtain the global context relationship, and the average value is not taken in the form of a window, but is averaged in units of feature maps. In the embodiments of the present application, the first pooling layer and the second pooling layer are average pooling layers.

[0078] Step 101a3, performing convolution processing on the first feature map according to the second partial convolution layer to obtain second information corresponding to the first training data.

[0079] In the embodiment of the present application, after the image coding device performs pooling processing on the first information according to the first pooling layer to obtain the first feature map corresponding to the first training data, the image coding device can perform convolution processing on the first feature map according to the second partial convolution layer to obtain second information corresponding to the first training data.

[0080] It can be understood that in the embodiment of the present application, the second partial convolution layer can also include a plurality of convolution layers with a size of 3x3, that is, after the first feature map is obtained, the first feature map can be further processed by the second partial convolution layer to obtain the second information corresponding to the first training data.

[0081] It can be understood that in the embodiment of the present application, the second information refers to data information obtained by the convolution processing of the second partial convolution layer.

[0082] Step 101a4, performing pooling processing on the second information according to the second pooling layer to obtain a second feature map corresponding to the first training data.

[0083] In the embodiment of the present application, after the image coding device performs convolution processing on the first feature map according to the second partial convolution layer to obtain the second information corresponding to the first training data, the image coding device can perform pooling processing on the second information according to the second pooling layer to obtain a second feature map corresponding to the first training data.

[0084] It can be understood that in the embodiment of the present application, after the second information is obtained, the second information also needs to be processed by the second pooling layer to obtain the second feature map.

[0085] Further, in the embodiment of the present application, the second feature map refers to data information obtained by the pooling processing of the second information, for example, the second feature map can be a 2x2x256-dimensional image matrix.

[0086] Step 101a5, performing convolution processing on the second feature map according to the third partial convolution layer to obtain third information corresponding to the first training data.

[0087] In the embodiment of the present application, after the image coding device performs pooling processing on the second information according to the second pooling layer to obtain the second feature map corresponding to the first training data, the image coding device can perform convolution processing on the second feature map according to the third partial convolution layer to obtain third information corresponding to the first training data.

[0088] It should be noted that in the embodiment of the present application, the third part of the convolutional layer includes a plurality of 1x1 convolutional layers, that is, after the second feature map is obtained, the second feature map is convoluted by using a plurality of 1x1 convolutional layers, so as to obtain the third information.

[0089] It can be understood that in the embodiment of the present application, the third information refers to the data information obtained by the convolutional processing of the third part of the convolutional layer.

[0090] Step 101a6, performing pooling processing on the third information according to the third pooling layer to obtain a third feature map corresponding to the first training data; wherein the third pooling layer is a global pooling layer.

[0091] In the embodiment of the present application, after the image coding device convolutes the second feature map according to the third part of the convolutional layer to obtain the third information corresponding to the first training data, the image coding device performs pooling processing on the third information according to the third pooling layer to obtain the third feature map corresponding to the first training data; wherein the third pooling layer is a global pooling layer.

[0092] It can be understood that in the embodiment of the present application, similarly, after the third information is obtained, the third information needs to be processed by the third pooling layer to obtain the third feature map, and the third pooling layer is a global pooling layer.

[0093] It should be noted that in the embodiment of the present application, the third pooling layer is a global pooling layer, and since the global pooling is not in the form of window mean value, but is in the form of feature map mean value, the feature map output after the global pooling processing is a value. For example, the third feature map obtained after the pooling processing can be a 256-dimensional image matrix.

[0094] Further, in the embodiment of the present application, the third feature map is the data information finally output by the initial feature extraction model when the initial feature extraction model is trained.

[0095] Figure 4 The implementation flow of the image coding method proposed in the embodiment of the present application Figure Four As shown in FIG. 1, the image coding device trains the initial clustering model by using the second training data to obtain the first clustering model, that is, step 102, which can include the following steps: Figure 4

[0096] Step 102a, performing segmentation processing on the second training data to obtain a vector set corresponding to the second training data.

[0097] ​In the embodiments of the present application, the image coding device trains the initial clustering model by using the second training data to obtain the first clustering model. Specifically, the image coding device can first perform segmentation processing on the second training data to obtain a vector set corresponding to the second training data.

[0098] It should be noted that, in the embodiments of the present application, the segmentation processing refers to segmenting the second training data, for example, segmenting the second training data with a size of 256 dimensions into four 64-dimensional vector sets, and the four 64-dimensional vector sets are the vector set corresponding to the second training data.

[0099] It should be noted that, in the embodiments of the present application, the number of segments in the segmentation processing can be set according to specific requirements, and the present application does not make specific limitations.

[0100] Step 102b, performing clustering processing on the vector set to obtain the encoded training data corresponding to the second training data.

[0101] In the embodiments of the present application, after the image coding device performs segmentation processing on the second training data to obtain the vector set corresponding to the second training data, the image coding device can perform clustering processing on the vector set to obtain the encoded training data corresponding to the second training data.

[0102] It should be noted that, in the embodiments of the present application, after the vector set is obtained, clustering processing is performed in each vector set space. The clustering processing can use a k-means clustering algorithm. For example, in the process of clustering training on the four 64-dimensional vector sets, clustering training is performed on each 64-dimensional vector set to obtain an 8-bit codebook, each codebook contains 256 encodings, and the 256 encodings can be mapped to 256 64-dimensional vector set clustering center points, which loses a certain accuracy but greatly improves the training speed. Correspondingly, the four 8-bit codebooks are the encoded training data corresponding to the second training data.

[0103] Step 102c, calculating distance information corresponding to the encoded training data.

[0104] In the embodiments of the present application, after the image coding device performs clustering processing on the vector set to obtain the encoded training data corresponding to the second training data, the image coding device can calculate the distance information corresponding to the encoded training data.

[0105] It should be noted that in the embodiments of the present application, the initial clustering model, i.e., the PQ algorithm, the objective function is the distance between the sample and the class center. Therefore, in the embodiments of the present application, the distance information, i.e., the distance between the second training data and the encoded data, is used to update the initial clustering model according to the distance information calculated by continuous iteration.

[0106] Step 102d, updating the initial clustering model according to the distance information to obtain the first clustering model.

[0107] In the embodiments of the present application, the image coding device updates the initial clustering model according to the distance information after calculating the distance information corresponding to the encoded training data, and obtains the first clustering model.

[0108] It should be noted that in the embodiments of the present application, the process of training the initial clustering model is to update the objective function by continuously calculating the distance information, so as to update the initial clustering model and finally obtain the first clustering model.

[0109] Figure 5 Implementation flow of the image coding method proposed in the embodiments of the present application Figure Five As shown in Figure 5 The image coding device encodes the image data according to the target feature extraction model and the first clustering model to obtain the encoded data corresponding to the image data set, i.e., step 103, which can include the following steps:

[0110] Step 103a, performing feature extraction processing on the image data according to the target feature extraction model to obtain feature information corresponding to the image data.

[0111] In the embodiments of the present application, the image coding device encodes the image data according to the target feature extraction model and the first clustering model to obtain the encoded data corresponding to the image data set. Specifically, the image coding device first performs feature extraction processing on the image data according to the target feature extraction model to obtain feature information corresponding to the image data.

[0112] It should be noted that in the embodiments of the present application, when the model training is completed, the trained target feature extraction model can be used to perform feature extraction processing on the image data when implementing specific image coding.

[0113] It can be understood that in the embodiments of the present application, the image data is the data input into the target feature extraction model, that is, the image data is the original image data that needs to be encoded by the model.

[0114] Further, in the embodiments of the present application, the feature information is information obtained after feature extraction processing is performed on the image data, and can represent the features of the image data to the greatest extent, i.e., the deep semantic level features of the image.

[0115] Step 103b: performing clustering processing on the feature information according to a first clustering model to obtain initial encoded data corresponding to the image data.

[0116] In the embodiments of the present application, after the image encoding device performs feature extraction processing on the image data according to the target feature extraction model to obtain the feature information corresponding to the image data, the image encoding device can perform clustering processing on the feature information according to a first clustering model to obtain initial encoded data corresponding to the image data.

[0117] It can be understood that, in the embodiments of the present application, the initial encoded data is the encoded data corresponding to the image data obtained after clustering processing. For example, if the first clustering model is used to perform clustering processing on the 256-dimensional vectors of the order of 100 million existing in the e-commerce scenario, the 256-dimensional vectors of the order of 100 million can be converted to the encoded data of the order of 10 million, thereby greatly improving the image encoding speed.

[0118] It should be noted that, in the embodiments of the present application, the initial encoded data is equivalent to a coarse-grained cluster set, and the final result of image encoding in the present application is a composite encoding obtained on the basis of the initial encoded data.

[0119] Step 103c: performing clustering processing on the initial encoded data according to a second clustering model to obtain encoded data corresponding to the image data.

[0120] In the embodiments of the present application, after the image encoding device performs clustering processing on the feature information according to the first clustering model to obtain the initial encoded data corresponding to the image data, the image encoding device can perform clustering processing on the initial encoded data according to a second clustering model to obtain the encoded data corresponding to the image data.

[0121] It should be noted that, in the embodiments of the present application, after the first clustering model is used to perform clustering processing on the feature information, the speed of image encoding can be greatly improved, but in the embodiments of the present application, in order to obtain image encoding with stronger recognition capability, clustering processing needs to be performed on the initial encoded data by using the second clustering model on the basis of the clustering processing performed by the first clustering model, so that precision compensation can be performed.

[0122] In the embodiments of the present application, the second clustering model is a balanced iterative reducing and clustering using hierarchies (BIRCH) algorithm. The BIRCH algorithm is generally used for image retrieval, and does not need to specify the number of clustering centers, and has a faster calculation speed. The BIRCH algorithm is used as the second clustering model in the present application, which can further improve the accuracy of image coding.

[0123] It should be noted that, in the embodiments of the present application, since the initial coded data is equivalent to a coarse-grained cluster set, the clustering processing of the initial coded data by the second clustering model can obtain fine-grained clusters, so that the coarse-grained cluster set and the fine-grained clusters are combined to obtain the final coded data.

[0124] In the embodiments of the present application, the image coding device performs feature extraction processing on the image data according to the target feature extraction model to obtain feature information corresponding to the image data, i.e., step 103a, which can include the following steps:

[0125] Step 103a1, performing convolution processing on the image data according to the first partial convolution layer to obtain fourth information corresponding to the image data.

[0126] In the embodiments of the present application, the image coding device performs feature extraction processing on the image data according to the target feature extraction model to obtain feature information corresponding to the image data. Specifically, the image coding device can first perform convolution processing on the image data according to the first partial convolution layer to obtain fourth information corresponding to the image data.

[0127] It can be understood that, in the embodiments of the present application, when the image data is processed by the target feature extraction model, all the convolution layers are trained and determined convolution layers, i.e., the settings of the related parameters in the convolution layers are determined. In the embodiments of the present application, the first partial convolution layer is first used to perform convolution processing on the image data, thereby obtaining the fourth information. Further, the fourth information is the data information obtained by performing convolution processing on the image data according to the determined first partial convolution layer.

[0128] Step 103a2, performing pooling processing on the fourth information according to the first pooling layer to obtain a fourth feature map corresponding to the image data.

[0129] In the embodiments of the present application, after the image coding device performs convolution processing on the image data according to the first partial convolution layer to obtain the fourth information corresponding to the image data, the image coding device can perform pooling processing on the fourth information according to the first pooling layer to obtain a fourth feature map corresponding to the image data.

[0130] It can be understood that, in the embodiment of the present application, when the image data is processed by the target feature extraction model, the pooling layer used for the pooling processing is a determined pooling layer after training. In the embodiment of the present application, after obtaining the fourth information corresponding to the image data, the fourth information is processed by the determined first pooling layer to obtain the fourth feature map.

[0131] Step 103a3, the fourth feature map is processed by the second part of the convolution layer to obtain the fifth information corresponding to the image data.

[0132] In the embodiment of the present application, after the image coding device processes the fourth information by the first pooling layer to obtain the fourth feature map corresponding to the image data, the fourth feature map can be processed by the second part of the convolution layer to obtain the fifth information corresponding to the image data.

[0133] Further, the fifth information is the data information obtained by processing the convolution according to the determined second part of the convolution layer.

[0134] Step 103a4, the fifth information is processed by the second pooling layer to obtain the fifth feature map corresponding to the image data.

[0135] In the embodiment of the present application, after the image coding device processes the fourth feature map by the second part of the convolution layer to obtain the fifth information corresponding to the image data, the fifth information can be processed by the second pooling layer to obtain the fifth feature map corresponding to the image data.

[0136] Step 103a5, the fifth feature map is processed by the third part of the convolution layer to obtain the sixth information corresponding to the image data.

[0137] In the embodiment of the present application, after the image coding device processes the fifth information by the second pooling layer to obtain the fifth feature map corresponding to the image data, the fifth feature map can be processed by the third part of the convolution layer to obtain the sixth information corresponding to the image data.

[0138] Further, the sixth information is the data information obtained by processing the convolution according to the determined third part of the convolution layer.

[0139] Step 103a6, the sixth information is processed by the third pooling layer to obtain the feature information corresponding to the image data; wherein the third pooling layer is a global pooling layer.

[0140] In the embodiment of the present application, after the image coding device performs convolution processing on the fifth feature map according to the third partial convolution layer to obtain the sixth information corresponding to the image data, the image coding device can perform pooling processing on the sixth information according to the third pooling layer to obtain the feature information corresponding to the image data; wherein the third pooling layer is a global pooling layer.

[0141] It should be noted that, in the embodiment of the present application, after the convolution processing of the last partial convolution layer, i.e., the third partial convolution layer, all convolution operations on the image data are completed, i.e., the feature extraction of the image data is completed; and then, after the pooling processing of the third pooling layer on the sixth information, the data information obtained is the feature information obtained by performing feature extraction processing according to the target feature extraction model.

[0142] Figure 6 Implementation process of the image coding method proposed in the embodiment of the present application Figure Six As shown in Figure 6 In the embodiment of the present application, the image coding device performs clustering processing on the feature information according to the first clustering model to obtain the initial coded data corresponding to the image data, i.e., step 103b, can include the following steps:

[0143] Step 103b1, performing segmentation processing on the feature information according to the first clustering model to obtain a vector set corresponding to the image data.

[0144] In the embodiment of the present application, the image coding device performs clustering processing on the feature information according to the first clustering model to obtain the initial coded data corresponding to the image data, specifically, the image coding device can first perform segmentation processing on the feature information according to the first clustering model to obtain a vector set corresponding to the image data.

[0145] It can be understood that, in the embodiment of the present application, the first clustering model trained is used to perform segmentation processing on the feature information, so as to perform clustering processing according to the obtained vector set.

[0146] Step 103b2, performing clustering processing on the vector set to obtain the initial coded data corresponding to the image data.

[0147] In the embodiment of the present application, after the image coding device performs segmentation processing on the feature information according to the first clustering model to obtain a vector set corresponding to the image data, the image coding device can perform clustering processing on the vector set to obtain the initial coded data corresponding to the image data.

[0148] It can be understood that, in the embodiment of the present application, the initial coded data is a coarse-grained cluster set, for example, the initial coded data can be expressed as 4 8-bit codes: 001 128 233 255.

[0149] Figure 7 Implementation flow of the image coding method proposed in the embodiments of the present application Figure Seven As shown in the embodiments of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model to obtain coded data corresponding to the image data, i.e., step 103c, which can include the following steps: Figure 7 As shown in the embodiments of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model to obtain coded data corresponding to the image data, i.e., step 103c, which can include the following steps:

[0150] Step 103c1, generating a cluster number corresponding to the initial coded data according to the second clustering model.

[0151] In the embodiments of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model to obtain coded data corresponding to the image data, specifically, the image coding device generates a cluster number corresponding to the initial coded data according to the second clustering model.

[0152] It should be noted that in the embodiments of the present application, the cluster number refers to a number generated by the second clustering model for the initial coded data, which is a fine-grained cluster. For example, if the initial coded data is represented as a string of 001 128 233 255, the cluster number generated according to the initial coded data can be represented as 188.

[0153] Step 103c2, generating coded data according to the initial coded data and the cluster number.

[0154] In the embodiments of the present application, the image coding device generates coded data according to the initial coded data and the cluster number after generating a cluster number corresponding to the initial coded data according to the second clustering model.

[0155] It can be understood that in the embodiments of the present application, the coded data is a composite code generated by combining the initial coded data and the cluster number, and the coded data can correspond to the image data, so that the image data can be identified based on the coded data, improving the efficiency of identifying the image data.

[0156] Further, in the embodiments of the present application, Figure 8 Implementation flow of the image coding method proposed in the embodiments of the present application Figure Eight As shown in the embodiments of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model to obtain coded data corresponding to the image data, i.e., step 103c, which can include the following steps: Figure 8 As shown in the embodiments of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model to obtain coded data corresponding to the image data, i.e., step 103c, which can include the following steps:

[0157] Step 104, if at least two coded data corresponding to at least two image data are the same, determining that the at least two image data are the same.

[0158] In the embodiment of the present application, the image coding device performs clustering processing on the initial coded data according to the second clustering model, and determines that at least two image data are the same if at least two coded data corresponding to the at least two image data are the same.

[0159] It can be understood that in the embodiment of the present application, each image data corresponds to a coded data, and the corresponding image data can be identified according to the coded data. Therefore, if at least two coded data corresponding to at least two image data are the same, it can be determined that the two image data are the same image.

[0160] Further, in the embodiment of the present application, if it is determined that the corresponding image data are the same according to the at least two coded data in the e-commerce scenario, the pictures ranked later can be suppressed for cheating, and the pictures suspected of cheating can be ranked later or even not appear on the search result page. Therefore, whether there is cheating or plagiarism in the image data can be effectively determined based on image coding, and the image recognition ability is improved.

[0161] The embodiment of the present application provides an image coding method. An image coding device trains an initial feature extraction model by using first training data to obtain a target feature extraction model, trains an initial clustering model by using second training data to obtain a first clustering model, and performs image coding processing on image data according to the target feature extraction model and the first clustering model to obtain coded data corresponding to the image data. That is, in the embodiment of the present application, the image coding device obtains the target feature extraction model and the first clustering model by training the initial feature extraction model and the initial clustering model, and then codes the image data according to the target feature extraction model and the first clustering model, so that the coded data has better feature expression ability. It can be seen that the target feature extraction model and the first clustering model constructed in the present application have better image coding ability, and the image recognition ability is improved.

[0162] Embodiment Two

[0163] In another embodiment of the present application, an exemplary image coding method is provided. Figure 9 Implementation architecture of the image coding method provided in the embodiment of the present application Figure One For example, Figure 9As shown, in the embodiments of the present application, when encoding the image data, the process of image encoding can be: first, inputting the image data into the CNN; after obtaining the hidden layer feature information, processing the hidden layer feature information according to the PQ algorithm to obtain the PQ code, this process can map the hidden layer feature information with a number of hundreds of millions to the PQ code with a number of millions; then processing the PQ code according to the BIRCH algorithm to finally obtain the composite code composed of the PQ code and the cluster number, this process can map the PQ code with a number of millions to the composite code with a number of ten millions, that is, the whole image encoding process can finally map the picture with a number of hundreds of millions to the composite code with a number of ten millions, and the composite code is the final image code.

[0164] Exemplarily, Figure 10 Implementation architecture of the image encoding method proposed in the embodiments of the present application Figure Two As shown, Figure 10 As shown, the CNN model in the embodiments of the present application can be divided into three convolutional layers, each of which can include multiple convolutional layers, for example, multiple 3x3 convolutional layers, but the last convolutional layer includes multiple 1x1 convolutional layers, and a pooling layer is arranged after each convolutional layer, and the last pooling layer is a global pooling layer; if a 64x64x3-dimensional image matrix is input into the CNN model in the present application, after the multi-layer convolution and pooling processing of the first convolutional layer and the pooling layer, a 4x4x256-dimensional feature map can be obtained, then the multi-layer convolution and pooling processing of the second convolutional layer and the pooling layer are performed on the 4x4x256-dimensional feature map to obtain a 2x2x256-dimensional feature map, and finally the convolution processing of the multi-layer 1x1 convolutional layer and the global pooling processing of the global pooling layer are performed on the 2x2x256-dimensional feature map to obtain a 256-dimensional feature map. That is, the CNN model in the present application can include two intermediate layers, i.e. the 4x4x256-dimensional feature map and the 2x2x256-dimensional feature map, and finally the mutual information loss (loss) of the 4x4x256-dimensional feature map, the 2x2x256-dimensional feature map and the 256-dimensional feature map relative to the input image matrix needs to be calculated; wherein the mutual information loss of the 4x4x256-dimensional feature map and the 2x2x256-dimensional feature map is the local mutual information, and the mutual information loss of the 256-dimensional feature map is the global mutual information. Thus, the introduction of local mutual information is more abundant, and the robustness of the CNN model is enhanced.

[0165] Exemplarily, Figure 11 Implementation architecture of the image encoding method proposed in the embodiments of the present application Figure Three As shown, Figure 11As shown, in the embodiment of the present application, the process of encoding the CNN hidden layer features by using the PQ algorithm can be: first, vector segmentation is performed on the initial vector of the CNN hidden layer features, the 256-dimensional initial vector is segmented into 4 64-dimensional vector sets, each 64-dimensional vector set respectively searches for the nearest center point to obtain 4 8-bit encodings, the encoding can be expressed in the form of a string as the splicing of 4 0-255 integer strings, for example, "001 128 233 255".

[0166] Exemplarily, the image encoding method proposed in the embodiment of the present application is applied to a certain e-commerce scene for experiments, and the effectiveness of the image encoding method proposed in the embodiment of the present application is verified through the experimental results generated in the e-commerce scene. Table 1 is the experimental results generated by applying the image encoding method proposed in the embodiment of the present application to a certain e-commerce scene, and the image is identified based on the image encoding technology of the present application, and then the search results with suspected cheating are suppressed. As shown in Table 1, among them, the search UV value represents the order amount generated by each search user, the higher the better; the search UCVR represents the order line number generated by each search user, the higher the better; the search UCTR represents the search result click number generated by each search user, the higher the better; the average click position of the order represents the position of the ordered goods in the search result page of each search user, the lower the better. It can be seen that the image encoding technology proposed in the embodiment of the present application has strong image recognition ability and can have a positive impact on search value income and user experience.

[0167] Table 1

[0168]

[0169] The embodiment of the present application provides an image encoding method, and an image encoding device trains an initial feature extraction model by using first training data to obtain a target feature extraction model; trains an initial clustering model by using second training data to obtain a first clustering model; and performs image encoding processing on image data according to the target feature extraction model and the first clustering model to obtain encoded data corresponding to the image data. That is, in the embodiment of the present application, the image encoding device obtains the target feature extraction model and the first clustering model by training the initial feature extraction model and the initial clustering model, and then encodes the image data according to the target feature extraction model and the first clustering model, so that the encoded data has better feature expression ability. It can be seen that the target feature extraction model and the first clustering model constructed in the present application have better image encoding ability and improve the recognition ability of the image.

[0170] Embodiment three

[0171] Based on the above embodiment, in another embodiment of the present application, Figure 12Structure of image coding device Figure One As shown in Figure 12 The image coding device 10 can include a training unit 11, a coding unit 12, and a judging unit 13.

[0172] The training unit 11 is configured to train an initial feature extraction model using first training data to obtain a target feature extraction model, and train an initial clustering model using second training data to obtain a first clustering model.

[0173] The coding unit 12 is configured to perform image coding processing on image data according to the target feature extraction model and the first clustering model to obtain coded data corresponding to the image data.

[0174] Further, in the embodiment of the present application, the training unit 11 is specifically configured to input the first training data into the initial feature extraction model to obtain first feature maps, second feature maps, and third feature maps corresponding to the first training data, calculate a first mutual information loss degree corresponding to the first feature maps, a second mutual information loss degree corresponding to the second feature maps, and a third mutual information loss degree corresponding to the third feature maps, calculate gradient information according to the first mutual information loss degree, the second mutual information loss degree, and the third mutual information loss degree, and update the initial feature extraction model according to the gradient information to obtain the target feature extraction model.

[0175] Further, in the embodiment of the present application, the training unit 11 is also specifically configured to perform convolution processing on the first training data according to a first part of convolution layers to obtain first information corresponding to the first training data, perform pooling processing on the first information according to a first pooling layer to obtain first feature maps corresponding to the first training data, perform convolution processing on the first feature maps according to a second part of convolution layers to obtain second information corresponding to the first training data, perform pooling processing on the second information according to a second pooling layer to obtain second feature maps corresponding to the first training data, perform convolution processing on the second feature maps according to a third part of convolution layers to obtain third information corresponding to the first training data, and perform pooling processing on the third information according to a third pooling layer to obtain third feature maps corresponding to the first training data, wherein the third pooling layer is a global pooling layer.

[0176] Further, in the embodiments of the present application, the training unit 11 is further specifically configured to perform segmentation processing on the second training data to obtain a vector set corresponding to the second training data; perform clustering processing on the vector set to obtain encoded training data corresponding to the second training data; calculate distance information corresponding to the encoded training data; and update the initial clustering model according to the distance information to obtain the first clustering model.

[0177] The encoding unit 12 is configured to perform feature extraction processing on image data according to the target feature extraction model to obtain feature information corresponding to the image data; perform clustering processing on the feature information according to the first clustering model to obtain initial encoded data corresponding to the image data; and perform clustering processing on the initial encoded data according to the second clustering model to obtain encoded data corresponding to the image data.

[0178] Further, in the embodiments of the present application, the encoding unit 12 is specifically configured to perform convolution processing on the image data according to a first part of convolution layers to obtain fourth information corresponding to the image data; perform pooling processing on the fourth information according to a first pooling layer to obtain a fourth feature map corresponding to the image data; perform convolution processing on the fourth feature map according to a second part of convolution layers to obtain fifth information corresponding to the image data; perform pooling processing on the fifth information according to a second pooling layer to obtain a fifth feature map corresponding to the image data; perform convolution processing on the fifth feature map according to a third part of convolution layers to obtain sixth information corresponding to the image data; and perform pooling processing on the sixth information according to a third pooling layer to obtain the feature information corresponding to the image data; wherein the third pooling layer is a global pooling layer.

[0179] Further, in the embodiments of the present application, the encoding unit 12 is further specifically configured to perform segmentation processing on the feature information according to the first clustering model to obtain a vector set corresponding to the image data; and perform clustering processing on the vector set to obtain the initial encoded data corresponding to the image data.

[0180] Further, in the embodiments of the present application, the encoding unit 12 is further specifically configured to generate a clustering number corresponding to the initial encoded data according to the second clustering model; and generate the encoded data according to the initial encoded data and the clustering number.

[0181] The judging unit 13 is configured to, after the encoding unit 12 performs image encoding processing on image data according to the target feature extraction model and the first clustering model to obtain encoded data corresponding to the image data, determine that at least two image data are the same if at least two encoded data corresponding to the at least two image data are the same.

[0182] Figure 13 A composition structure of the image coding device according to an embodiment of the present application is shown in the following figure Figure Two As shown in the figure, the image coding device 10 according to an embodiment of the present application can further include a processor 14, a memory 15 storing executable instructions of the processor 14, and further include a communication interface 16, and a bus 17 for connecting the processor 14, the memory 15 and the communication interface 16. Figure 13 In an embodiment of the present application, the processor 14 can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device for realizing the function of the processor can also be other devices, which are not limited in the embodiments of the present application. The processor 14 can also include a memory 15, which can be connected with the processor 14, wherein the memory 15 is used to store executable program codes, the program codes including computer operation instructions, and the memory 15 can include a high-speed RAM memory and can also include a non-volatile memory, for example, at least two disk memories.

[0183] In an embodiment of the present application, the bus 17 is used to connect the communication interface 16, the processor 14 and the memory 15, and the mutual communication among these devices.

[0184] In an embodiment of the present application, the memory 15 is used to store instructions and data.

[0185] Further, in an embodiment of the present application, the processor 14 is used to train an initial feature extraction model by using first training data to obtain a target feature extraction model, train an initial clustering model by using second training data to obtain a first clustering model, and perform image coding processing on image data according to the target feature extraction model and the first clustering model to obtain corresponding coded data of the image data.

[0186]

[0187] ​In practical applications, the memory 15 can be a volatile memory, such as a random-access memory (RAM), or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or a combination of the above types of memories, and provides instructions and data to the processor 14.

[0188] In addition, each functional module in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional module.

[0189] The integrated unit, if realized in the form of a software functional module and not sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiment can essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the embodiment method. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0190] The embodiment of the present application provides a kind of image coding device, image coding device utilizes first training data and trains initial feature extraction model, obtains target feature extraction model;Utilize second training data and train initial clustering model, obtain first clustering model;According to target feature extraction model and first clustering model, image coding processing is carried out to image data, obtains the corresponding data after coding of image data.That is to say, in the embodiment of the present application, image coding device is obtained by training initial feature extraction model and initial clustering model, target feature extraction model and first clustering model, and then according to target feature extraction model and first clustering model, the coding of image data is realized, so that the data after coding has more optimal feature expression ability, it can be seen that, the target feature extraction model and first clustering model constructed in the present application have better image coding ability, improve the recognition ability of image.

[0191] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the image coding method described above.

[0192] Specifically, the program instructions corresponding to the image coding method in the embodiment can be stored on optical disc, hard disk, U disk and other storage media, when the program instructions corresponding to the image coding method in the storage medium are read or executed by an electronic device, the following steps are included:

[0193] Utilize first training data and train initial feature extraction model, obtain target feature extraction model;

[0194] Utilize second training data and train initial clustering model, obtain first clustering model;

[0195] According to the target feature extraction model and the first clustering model, image coding processing is carried out to image data, obtains the corresponding data after coding of image data.

[0196] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can be in the form of hardware embodiments, software embodiments or embodiments combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program code.

[0197] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks.

[0198] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks.

[0199] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure One The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks.

[0200] The above-described embodiments are merely preferred embodiments of the present application, but not to confine the present application.

Claims

1. An image encoding method, the method being applied to image recognition in e-commerce scenarios, characterized in that, The method includes: The initial feature extraction model is trained using the first training data to obtain the target feature extraction model; An initial clustering model is trained using the second training data to obtain a first clustering model; wherein, the initial clustering model is a product quantization model; The image data is processed by the target feature extraction model to obtain the feature information corresponding to the image data. The feature information is clustered according to the first clustering model to obtain the initial encoded data corresponding to the image data; Generate cluster numbers corresponding to the initially encoded data based on the second clustering model; Generate encoded data corresponding to the image data based on the initial encoded data and the cluster number; The step of training an initial feature extraction model using the first training data to obtain a target feature extraction model includes: The first training data is processed by convolutional layer in the first part to obtain the first information corresponding to the first training data. The first information is pooled according to the first pooling layer to obtain the first feature map corresponding to the first training data; the first pooling layer is an average pooling layer. The first feature map is convolved according to the second part of the convolutional layer to obtain the second information corresponding to the first training data; The second information is pooled using the second pooling layer to obtain the second feature map corresponding to the first training data; the second pooling layer is an average pooling layer. The second feature map is convolved according to the third part of the convolutional layer to obtain the third information corresponding to the first training data; The third information is pooled using the third pooling layer to obtain the third feature map corresponding to the first training data; wherein the third pooling layer is a global pooling layer. Calculate the first mutual information loss corresponding to the first feature map, the second mutual information loss corresponding to the second feature map, and the third mutual information loss corresponding to the third feature map; the first mutual information loss, the second mutual information loss, and the third mutual information loss respectively characterize the degree of information loss of the first feature map, the second feature map, and the third feature map after feature extraction processing relative to the input image. Gradient information is calculated based on the first mutual information loss, the second mutual information loss, and the third mutual information loss. The initial feature extraction model is updated based on the gradient information to obtain the target feature extraction model.

2. The method according to claim 1, characterized in that, The step of training an initial clustering model using the second training data to obtain a first clustering model includes: The second training data is segmented to obtain the vector set corresponding to the second training data. Clustering is performed on the vector set to obtain the encoded training data corresponding to the second training data; Calculate the distance information corresponding to the encoded training data; The initial clustering model is updated based on the distance information to obtain the first clustering model.

3. The method according to claim 1, characterized in that, The step of performing feature extraction processing on the image data according to the target feature extraction model to obtain the feature information corresponding to the image data includes: The image data is convolved according to the first part of the convolutional layer to obtain the fourth information corresponding to the image data; The fourth information is pooled according to the first pooling layer to obtain the fourth feature map corresponding to the image data; The fourth feature map is convolved according to the second part of the convolutional layer to obtain the fifth information corresponding to the image data; The fifth information is pooled according to the second pooling layer to obtain the fifth feature map corresponding to the image data; The fifth feature map is convolved according to the third part of the convolutional layer to obtain the sixth information corresponding to the image data; The sixth information is pooled according to the third pooling layer to obtain the feature information corresponding to the image data; wherein, the third pooling layer is a global pooling layer.

4. The method according to claim 1, characterized in that, The step of clustering the feature information according to the first clustering model to obtain the initial encoded data corresponding to the image data includes: The feature information is segmented according to the first clustering model to obtain the vector set corresponding to the image data; Clustering is performed on the vector set to obtain the initial encoded data corresponding to the image data.

5. The method according to claim 1, characterized in that, After performing image encoding processing on the image data based on the target feature extraction model and the first clustering model to obtain the encoded data corresponding to the image data, the method further includes: If at least two image data correspond to at least two encoded data that are the same, then the at least two image data are determined to be the same.

6. An image encoding device, the device being applied to image recognition in e-commerce scenarios, characterized in that, The image encoding device includes: a training unit and an encoding unit. The training unit is used to train an initial feature extraction model using the first training data to obtain a target feature extraction model; and to train an initial clustering model using the second training data to obtain a first clustering model, wherein the initial clustering model is a product quantization model. The encoding unit is configured to perform feature extraction processing on image data according to the target feature extraction model to obtain feature information corresponding to the image data; perform clustering processing on the feature information according to the first clustering model to obtain initial encoded data corresponding to the image data; generate cluster number corresponding to the initial encoded data according to the second clustering model; and generate encoded data corresponding to the image data according to the initial encoded data and the cluster number. The training unit is specifically configured to: perform convolution processing on the first training data according to a first convolutional layer to obtain first information corresponding to the first training data; perform pooling processing on the first information according to a first pooling layer to obtain a first feature map corresponding to the first training data; wherein the first pooling layer is an average pooling layer; perform convolution processing on the first feature map according to a second convolutional layer to obtain second information corresponding to the first training data; perform pooling processing on the second information according to a second pooling layer to obtain a second feature map corresponding to the first training data; wherein the second pooling layer is an average pooling layer; perform convolution processing on the second feature map according to a third convolutional layer to obtain third information corresponding to the first training data; and perform pooling processing on the third information according to a third pooling layer. Pooling is performed to obtain a third feature map corresponding to the first training data; wherein the third pooling layer is a global pooling layer; a first mutual information loss corresponding to the first feature map, a second mutual information loss corresponding to the second feature map, and a third mutual information loss corresponding to the third feature map are calculated; the first mutual information loss, the second mutual information loss, and the third mutual information loss respectively characterize the degree of information loss of the first feature map, the second feature map, and the third feature map after feature extraction processing relative to the input image; gradient information is calculated based on the first mutual information loss, the second mutual information loss, and the third mutual information loss; the initial feature extraction model is updated based on the gradient information to obtain the target feature extraction model.

7. An image encoding device, characterized in that, The image encoding device further includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a program stored thereon, which is applied in an image encoding device, wherein when the program is executed by a processor, it implements the image encoding method as described in any one of claims 1-5.

Images