Image compression and classification methods, apparatus and electronic equipment

Abstract

This invention provides an image compression and classification method, apparatus, and electronic device. The image compression and classification method includes acquiring a convolutional neural network model to be trained. The model includes an encoder composed of convolutional layers, a generator sharing a common encoder, and a classifier. Image samples are acquired; the image samples are convolved using the encoder to obtain feature images; the feature images are input into the generator and classifier, and a compressed image of the image samples and a classification result are output. A first error between the compressed image and the image samples, and a second error in the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first and second errors. The adjusted model parameters are then used to acquire image samples again, and the image samples are convolved using the encoder to obtain feature images, until model parameters that meet the training target values ​​are obtained. This invention uses the same encoder for image compression and image classification, which can greatly reduce computational resources.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image compression and classification method, apparatus, and electronic device. Background Technology

[0002] In recent years, with the deepening research into deep learning, image compression and classification methods based on deep learning have been proposed. Deep learning is considered to have unique advantages in image compression due to its ability to extract and represent image features, as well as its ability to process high-dimensional data. Currently, common deep learning architectures such as Convolutional Neural Networks (CNNs) have achieved remarkable performance in image processing applications such as image encoding, compression, and classification.

[0003] Using deep learning technology, a network model is first trained using image samples to adjust its parameters, resulting in a well-trained model. Then, the image to be processed is input into the trained network model, which outputs the image processing result, such as image compression or classification labels.

[0004] Therefore, how to save computational resources in image processing tasks based on deep learning technology is a topic that is widely considered in the industry. Summary of the Invention

[0005] This invention provides an image compression and classification method, apparatus, and electronic device to address the shortcomings of existing technologies where image processing requires a large amount of computing resources, thereby achieving resource savings during computation.

[0006] This invention provides an image compression and classification method, comprising:

[0007] Obtain a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0008] Image samples are acquired, and the image samples are convolved using the encoder to obtain feature images;

[0009] The feature image is input into the generator, which outputs a compressed image of the image sample; and the feature image is input into the classifier, which outputs the classification result of the image sample.

[0010] The first error between the compressed image and the image sample, and the second error of the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first error and the second error. The adjusted model parameters are used to return the obtained image sample. The image sample is then convolved using the encoder to obtain a feature image. This process continues until the model parameters that reach the training target value are obtained.

[0011] According to an image compression and classification method provided by the present invention, obtaining the first error between the compressed image and the image sample includes:

[0012] The compressed image and the image sample are input into the discriminator, and the discriminator outputs the distinction result between the compressed image and the image sample, and the distinction result is used as the first error;

[0013] The generator and discriminator are respectively the generator network and the discriminator network in the generative adversarial network.

[0014] According to an image compression and classification method provided by the present invention, adjusting the model parameters of the convolutional neural network model using the first error and the second error includes:

[0015] The parameters of the discriminator are adjusted using the first error.

[0016] According to an image compression and classification method provided by the present invention, after returning the acquired image samples using the adjusted model parameters, performing convolution processing on the image samples using the encoder to obtain feature images, and until obtaining model parameters that achieve the training objective, the method further includes:

[0017] The test image is input into the test network obtained based on the model parameters of the convolutional neural network model to be trained, and the test value of the classification label is output.

[0018] The obtained image samples are returned using the adjusted model parameters. The image samples are then convolved using the encoder to obtain feature images, until model parameters that meet the training objective are obtained, including:

[0019] If the test value of the classification label does not reach the training target value, the obtained image sample is returned using the adjusted model parameters. The image sample is then convolved using the encoder to obtain a feature image, until the model parameters that reach the training target are obtained.

[0020] According to an image compression and classification method provided by the present invention, adjusting the model parameters of the convolutional neural network model using the first error and the second error includes:

[0021] The parameters of the encoder and generator are adjusted using the first error;

[0022] The parameters of the encoder and classifier are adjusted using the second error.

[0023] An image compression and classification method includes: obtaining a trained convolutional neural network model, wherein the convolutional neural network model includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0024] The image to be processed is acquired, and the encoder is used to perform a convolution operation on the image to be processed to obtain a feature image.

[0025] The feature image is input into the generator, which outputs a compressed image of the image to be processed; and the feature image is input into the classifier, which outputs a classification label for the image to be processed.

[0026] The convolutional neural network model is trained using image samples.

[0027] The present invention also provides an image compression and classification apparatus, comprising: an acquisition module for acquiring a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0028] The feature extraction module acquires image samples and uses the encoder to perform convolution processing on the image samples to obtain feature images.

[0029] The training module inputs the feature image into the generator and outputs a compressed image of the image sample; and inputs the feature image into the classifier and outputs the classification result of the image sample.

[0030] The model parameter adjustment module obtains a first error between the compressed image and the image sample, and a second error of the classification result. It then uses the first and second errors to adjust the model parameters of the convolutional neural network model. The adjusted model parameters are then used to return the obtained image sample. The encoder is used to perform convolution processing on the image sample to obtain a feature image, until the model parameters that reach the training target value are obtained.

[0031] The present invention also provides an image compression and classification apparatus, including an acquisition module for acquiring a trained convolutional neural network model, wherein the convolutional neural network model includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier.

[0032] The feature extraction module acquires the image to be processed and performs a convolution operation on the image to be processed using the encoder to obtain a feature image.

[0033] The image compression and classification module inputs the feature image into the generator and outputs a compressed image of the image to be processed; and inputs the feature image into the classifier and outputs a classification label of the image to be processed.

[0034] The convolutional neural network model obtained through training is trained using image samples.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the image compression and classification methods described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image compression and classification method as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the image compression and classification methods described above.

[0038] The image compression and classification method, apparatus, and electronic device provided by this invention utilize multi-task joint training technology to train a convolutional neural network model for image compression and image classification. The generator for image compression and the classifier for image classification share an encoder composed of convolutional layers. Thus, the trained convolutional neural network model can simultaneously perform image compression and image classification on the image to be processed.

[0039] In the technical solution of this invention, the same encoder is used for image compression and image classification. This design can greatly reduce computing resources. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts illustrating the image compression and classification method provided by the present invention;

[0042] Figure 2 This is one of the framework diagrams of the image compression and classification method provided by the present invention;

[0043] Figure 3 This is the second framework diagram of the image compression and classification method provided by the present invention;

[0044] Figure 4 This is the third framework diagram of the image compression and classification method provided by the present invention;

[0045] Figure 5 This is the second flowchart illustrating the image compression and classification method provided by the present invention;

[0046] Figure 6 This is the third flowchart illustrating the image compression and classification method provided by the present invention;

[0047] Figure 7 This is a network model diagram of the image compression and classification method provided by the present invention;

[0048] Figure 8 This is one of the structural schematic diagrams of the image compression and classification device provided by the present invention;

[0049] Figure 9 This is the second schematic diagram of the image compression and classification device provided by the present invention;

[0050] Figure 10 This is the third schematic diagram of the image compression and classification device provided by the present invention;

[0051] Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] The following is combined with Figures 1-7 This invention describes an image compression and classification method. The method is implemented in an image processing system or a control module running an image processing system.

[0054] like Figure 1 As shown, the present invention provides an image compression and classification method, comprising the following steps:

[0055] Step 110: Obtain the convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0056] Step 120: Obtain image samples, and use the encoder to perform convolution processing on the image samples to obtain feature images;

[0057] Step 130: Input the feature image into the generator to output a compressed image of the image sample, and input the feature image into the classifier to output the classification result of the image sample;

[0058] Step 140: Obtain the first error between the compressed image and the image sample, and the second error of the classification result, and adjust the model parameters of the convolutional neural network model using the first error and the second error, return the obtained image sample using the adjusted model parameters, and perform convolution processing on the image sample using the encoder to obtain the feature image, until the model parameters that reach the training target value are obtained.

[0059] Specifically, a convolutional neural network (CNN) is a deep feedforward neural network that includes convolutional computations. It possesses representation learning capabilities and can perform translation-invariant classification of input information according to its hierarchical structure. The convolutional layers of a CNN extract features from the input data. Each layer contains multiple convolutional kernels, and each element of a kernel corresponds to a weight coefficient and a bias. These weight coefficients and biases constitute the first part of the model parameters to be trained.

[0060] like Figure 2 As shown, the convolutional neural network framework of the image compression and classification method provided in this embodiment includes an encoder 210, a generator 220, and a classifier 230. The encoder 210 takes image samples as input and outputs feature images, which also serve as inputs to the generator 220 and the classifier 230.

[0061] In this embodiment, the encoder consists of multiple convolutional layers. The encoder performs convolution processing on the input image samples to obtain the feature images corresponding to the image samples. The convolution processing is the encoding process, and its main purpose is to extract features from the input image samples to obtain the feature images.

[0062] The generator and classifier are used to compress and classify the feature images, respectively. The generator and classifier share the same encoder. Therefore, the feature images generated by the encoder can be used for both image compression and image classification.

[0063] Specifically, the generator reconstructs the image based on the feature image output by the encoder, generating a compressed image similar to the image sample. The generator then performs deconvolution on the feature image to obtain the compressed image. The similarity difference between the compressed image and the image sample is defined as the first error. Based on this first error, the model parameters, including the corresponding model parameters in the encoder and generator, can be adjusted.

[0064] Specifically, the classifier performs target recognition and classification on the feature image output by the encoder, obtaining the classification result of the image sample. The classification accuracy is the second error, and the model parameters, including those of the encoder and classifier, are adjusted based on the first error.

[0065] In this embodiment, reaching the training target value means that the convolutional neural network model has converged, which is specifically manifested as the error of both the image compression result and the classification result being less than a preset threshold, indicating that the model training is complete.

[0066] The image compression and classification method provided in this embodiment utilizes a multi-task joint training technique to train the convolutional neural network model to be trained for image compression and image classification. The generator for image compression and the classifier for image classification share an encoder composed of convolutional layers. This allows the training process for image compression and image classification to simultaneously train the model parameters in the encoder, thereby significantly improving the accuracy of the model parameters in the encoder and further enhancing the accuracy of the convolutional neural network model. In this embodiment, the same encoder is used for image compression and image classification, a design that greatly reduces computational resources.

[0067] In this way, the trained convolutional neural network model can simultaneously perform image compression and image classification on the image to be processed.

[0068] Optionally, obtaining the first error between the compressed image and the image sample includes:

[0069] The compressed image and the image sample are input into the discriminator, and the discriminator outputs the distinction result between the compressed image and the image sample, and the distinction result is used as the first error;

[0070] The generator and discriminator are respectively the generator network and the discriminator network in the generative adversarial network.

[0071] Optionally, adjusting the model parameters of the convolutional neural network model using the first error and the second error includes:

[0072] The parameters of the encoder and generator are adjusted using the first error;

[0073] The parameters of the encoder and classifier are adjusted using the second error.

[0074] The parameters of the discriminator are adjusted using the first error.

[0075] like Figure 3 As shown, with Figure 2In contrast, the convolutional neural network model in this embodiment also includes: a discriminator 310, which takes the compressed image generated by the generator 320 and the original image sample as input, and outputs the distinction result between the compressed image and the image sample.

[0076] The generator is used for image reconstruction, generating a compressed image similar to the image sample. The discriminator is used to distinguish between the compressed image and the image sample, and to determine whether the second error between the compressed image and the image sample meets the target.

[0077] Specifically, the generator's goal is to generate compressed images similar to the image samples, thus deceiving the discriminator. During training, the discriminator receives both the image samples and the compressed images generated by the generator—that is, it receives real data and fake data produced by the generator. The discriminator then determines whether the input compressed image is real or fake data. Based on the discriminator's output, the parameters of both the generator and the discriminator are adjusted. If the discriminator determines it to be real data, the generator's parameters need to be adjusted to make the generated fake data more realistic; if it determines it to be fake data, the discriminator's parameters need to be adjusted to avoid errors in future judgments. In this embodiment, model training continues until the generator and discriminator reach a balanced and harmonious state.

[0078] In this embodiment, the discriminator outputs a distinction result based on the input image sample and the compressed image, and adjusts the parameters of the generator and the discriminator according to the distinction result until the model parameters that reach the training target value are obtained.

[0079] like Figure 4 As shown, the convolutional neural network model in this embodiment also includes a quantizer 420. The model includes the following steps: quantizing the feature image acquired by the encoder 410; processing the feature image using the quantizer 420 to obtain a quantized representation of the feature image; mapping the feature image to multiple quantization levels using the quantizer 420; simultaneously inputting the quantized representation to the generator 430 and the classifier 440; generating a compressed image based on the quantized representation; and obtaining a classification label for the feature image based on the quantized representation.

[0080] Reference Figure 5 This embodiment provides an image compression and classification method including the following steps:

[0081] Step 510: Obtain the convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0082] Step 520: Obtain image samples, and use the encoder to perform convolution processing on the image samples to obtain feature images;

[0083] Step 530: Input the feature image into the generator to output a compressed image of the image sample, and input the feature image into the classifier to output the classification result of the image sample;

[0084] Step 540: Obtain the first error between the compressed image and the image sample, and the second error of the classification result, and adjust the model parameters of the convolutional neural network model using the first error and the second error;

[0085] Step 550: Input the test image into the test network obtained based on the model parameters of the convolutional neural network model to be trained, and output the classification label test value;

[0086] Step 560: If the test value of the classification label does not reach the training target value, return the obtained image sample using the adjusted model parameters, and use the encoder to perform convolution processing on the image sample to obtain the feature image, until the model parameters that reach the training target are obtained.

[0087] The parameters trained by the classifier are input into the test network; the test image is input into the test network with the trained parameters to obtain the classification label test value of the test image.

[0088] Optionally, the average of all classification label test values ​​is calculated, and then compared with the target value to determine whether the training has reached the target. If the calculated average reaches the target value, training ends; otherwise, training the model parameters continues. The preset threshold in this embodiment can be set according to different situations and is not specifically limited here.

[0089] like Figure 6 As shown, in an optional embodiment, the present invention provides an image compression and classification method. This embodiment describes the application process of a trained convolutional neural network, including the following steps:

[0090] Step 610: Obtain the trained convolutional neural network model, wherein the convolutional neural network model includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0091] Step 620: Obtain the image to be processed, and perform a convolution operation on the image to be processed using the encoder to obtain a feature image;

[0092] Step 630: Input the feature image into the generator to output a compressed image of the image to be processed, and input the feature image into the classifier to output the classification label of the image to be processed;

[0093] The convolutional neural network model is trained using image samples.

[0094] The image compression and classification method provided in this embodiment utilizes a multi-task joint convolutional neural network model for image compression and classification, wherein the generator for image compression and the classifier for image classification share an encoder composed of convolutional layers. In this way, the convolutional neural network model can simultaneously perform image compression and image classification on the image to be processed. In this embodiment, the same encoder is used for image compression and image classification, a design that significantly reduces computational resources.

[0095] like Figure 7 As shown in the diagram, combined with the convolutional neural network model of the present invention, the image compression and classification method provided by the present invention will be described in detail below:

[0096] It should be noted that the convolutional neural network architecture of the present invention includes an encoder 710, a quantizer 720, a generator 730, a classifier 740, and a discriminator 750.

[0097] The encoder is mainly composed of convolutional blocks, each of which contains a convolutional layer, a data normalization layer, and an activation function layer. The first layer is a convolutional block layer with a kernel size of 7×7, 32 channels, and a stride of 1. The second layer is a convolutional block layer with a kernel size of 3×3, 64 channels, and a stride of 1, and a max pooling layer with a size of 2×2 and a stride of 2. The third layer consists of two convolutional block layers, with a kernel size of 3×3, 125 channels, and strides of 1 and 2, respectively. The fourth layer consists of two convolutional block layers, with a kernel size of 3×3, 256 channels, and strides of 1 and 2 respectively. The fifth layer consists of two convolutional block layers with a kernel size of 3×3, 512 channels, and a stride of 1. Following this are two residual blocks and one average pooling layer. Each residual block has a kernel size of 3×3, a stride of 1, and 512 channels. The average pooling layer has a size of 2×2 and a stride of 2. Finally, there is a convolutional block layer with a kernel size of 3×3, C channels, and a stride of 1.

[0098] The size of the feature image is obtained using the following formula;

[0099]

[0100] Where z represents the information corresponding to the feature image, W and H are the length and width of the image sample, respectively, m is the scaling factor, and C is the sampling factor.

[0101] After obtaining the feature image, the generator and classifier simultaneously compress and classify the feature image.

[0102] Specifically, in this embodiment, the quantizer receives the feature image generated by the encoder and performs adaptive bit allocation. Assume there are L center vectors. The encoder reshapes Z into a matrix. And each column It is assigned to the nearest index in C to perform encoding. A soft assignment is defined as follows:

[0103]

[0104] Where softmax is the standard softmax operator, making All are positive elements, and use To define The j-th element,

[0105]

[0106] Make Converging to the one-hot encoding in C that is closest to Z, with parameter σ > 0 as the hardness of the soft assignment, the soft quantization of Z is defined using this soft assignment as follows:

[0107]

[0108] The center is written in matrix form. The corresponding hard allocation is used in The center of C that is closest to Z can be written as:

[0109]

[0110] In this embodiment, nearest neighbor assignment is used for calculation. Soft allocation is used to approximate it with smooth relaxation, quantizing the feature representation of the image into a differentiable approximation. Five quantization centers {-2, -1, 0, 1, 2} are set to reduce the amount of data transmitted, mapping the feature image to the quantized latent representation.

[0111] Specifically, in this embodiment, the generator corresponding to the network is the reverse process of the encoder. Through upsampling and deconvolution, the generator encodes and compresses the image. The first layer of the generator network is a deconvolutional layer with a kernel size of 3×3 and 512 channels. The following network consists of 9 residual blocks, each composed of two convolutional blocks with a stride of 1 and a kernel size of 3×3. Next are four upsampling layers with kernel sizes of 3×3 and a stride of 2. The last layer is a convolutional layer with a kernel size of 7×7, a stride of 1, and 3 channels. Through this network, the feature image can be reconstructed to obtain the compressed image.

[0112] Specifically, for the discriminator corresponding to the discriminator network in this embodiment, both the image sample and the compressed image are fed into the discriminator, allowing the discriminator to distinguish between the image sample and the compressed image. This embodiment employs a multi-scale discriminator technique, using the Leaky ReLU activation function. First, two 3×3 average pooling layers with a stride of 2 are performed. Then, the discriminator network is entered. The first layer is a convolutional layer with a stride of 2, a kernel size of 4×4, and 64 channels. The second part consists of three convolutional blocks with kernel sizes of 4×4, strides of 2, and channel numbers of 125, 256, and 512 respectively. The last layer is a convolutional layer with a kernel size of 4×4, a stride of 1, and 1 channel.

[0113] Specifically, for the classifier in this embodiment, the quantized representation is input into the classifier for flattening; the first fully connected layer with 4096 nodes uses dropout technology to prevent overfitting and sets the probability of elements being retained; the second fully connected layer with 100 nodes uses the Softmax classifier to obtain the classification label of the feature image.

[0114] During training of the convolutional neural network in this embodiment, the generation loss value is obtained according to the loss function corresponding to the generator; the classification loss value is obtained according to the loss function corresponding to the classifier; and the discrimination loss value is obtained according to the loss function corresponding to the discriminator.

[0115] The overall loss is calculated by weighting the generated loss value, classification loss value, and discriminant loss value as shown in the following formula:

[0116]

[0117] Where L is the overall loss, λ G λ is the weight of the generator. D λ is the weight of the discriminator. C L represents the weights of the classifier. G To generate the loss value, L D To determine the loss value, L C Classification loss value.

[0118] Specifically, based on the multi-task learning network implemented in this paper, different loss functions are designed for different tasks. For image compression, adversarial loss, generator loss, and discriminator loss are used to improve perceptual quality at low bit rates. For image classification, cross-entropy loss is employed to ensure classification accuracy.

[0119] The method for calculating the loss value is as follows:

[0120] L G =λ GAN L GAN +λFM L FM

[0121] Where, λ GAN λ is the weight of the adversarial loss. FM L represents the weights of the feature matching loss. GAN To counteract the loss value, L FM The feature matching loss value;

[0122] The formula for calculating the loss value of feature matching is:

[0123]

[0124] It is the i-th feature among the k features of the discriminator, E x It is the expected value of the input image.

[0125] The method for calculating the discriminant loss value is as follows:

[0126]

[0127] in, Here, x represents the parameters for compressing the image, and x represents the parameters for the image sample.

[0128] Specifically, for the discriminator, a multi-scale discriminator is used, comprising three discriminators with the same network structure but operating at different image scales. In this embodiment, the image samples and compressed images are downsampled by 2x and 4x respectively, creating image pyramids at three scales. The discriminators then distinguish between the image samples and compressed images at different scales, guiding the generator to produce globally consistent images. Furthermore, guiding the generator to generate finer details also simplifies generator training.

[0129] The method for calculating the classification loss value is as follows:

[0130]

[0131] Where y is the label corresponding to image sample x, and m is the number of images.

[0132] In this embodiment, the method for calculating the classification loss value can be the cross-entropy loss method.

[0133] The effectiveness of the image compression and classification method provided by this invention will be demonstrated below with actual data.

[0134] To improve the experimental results of this invention, a dataset containing images of 1000 object classes was used. This dataset is part of a larger image network. 100 classes were randomly selected, with 1300 images per class. The images were then randomly divided into training and test sets. Each image had a different size; the images were preprocessed to resize them to 256×256×3 pixels. The dataset and test set were then randomly divided in a 9:1 ratio. Over 117,000 images were used for training, and the remainder for testing. The model parameters were set as follows: batch size was set to 32, and feature matching weight λ... FM =10, the number of channels in the last layer of the encoder C = 16, 8, 4. Different bit rates are obtained using different values ​​of C. When C = 4, the bit rate is 0.0362; when C = 8, the bit rate is 0.0725; when C = 16, the bit rate is 0.145. The quantization center is set to {-2, -1, 0, 1, 2}, and the learning rate for the generator, discriminator, and classifier is 2 × 10⁻⁶. -4 The Adam optimizer with a momentum of 0.9 was used, and the entire model was trained for more than 100 epochs on a TITAN XP GPU.

[0135] The proposed method is compared with the image compression standard JPEG2000. First, the image is compressed to a fixed bit rate for easy comparison with the algorithm of this invention. Then, the classic image classification algorithm VGG (Visual Geometry Group) network is used for image classification. Structural similarity (SSIM) is used to evaluate compression performance, and classification accuracy is used to evaluate classification accuracy. Firstly, using GANs for image compression achieves extremely low bit rates and visually acceptable reconstruction. The table below shows the compression performance and classification accuracy at different compression levels:

[0136]

[0137] As can be seen from the table, at a bitrate of 0.1450, the compression performance is comparable to JPEG 2000. Secondly, at significantly lower bitrates, the classification accuracy at different bitrates was tested, and its performance is superior to JPEG 2000. In particular, at a bitrate of 0.0362, the classification accuracy of this method is three times that of the JPEG 2000 method.

[0138] The following shows a comparison of parameters for different tasks.

[0139] Task method parameter compression GAN <![CDATA[2.91*7 7 / 111.01M]]> Classification VGG19 <![CDATA[1.38*7 8 / 527.75M]]> Separate compression and classification GAN+VGG19 <![CDATA[1.67*7 8 / 638.76M]]> Collaborative compression and classification This invention <![CDATA[2.96*7 7 / 112.92M]]>

[0140] For individual compression and classification, the parameter is 638.76M. For collaborative compression and classification, the parameter is 112.92M. This shows that the parameter significantly saves computational resources. Extensive experiments have demonstrated the effectiveness and practicality of the image compression and classification method provided by this invention.

[0141] The image compression and classification apparatus provided by the present invention will be described below. The image compression and classification apparatus described below can be referred to in correspondence with the image compression and classification method described above.

[0142] like Figure 8 As shown, the present invention provides an image compression and classification apparatus for training the convolutional neural network of the present invention, comprising the following modules:

[0143] The acquisition module 810 acquires a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier.

[0144] The feature extraction module 820 acquires image samples and performs convolution processing on the image samples using the encoder to obtain a feature image.

[0145] The training module 830 inputs the feature image into the generator and outputs a compressed image of the image sample; and inputs the feature image into the classifier and outputs the classification result of the image sample.

[0146] The model parameter adjustment module 840 obtains the first error between the compressed image and the image sample, and the second error of the classification result, and adjusts the model parameters of the convolutional neural network model using the first error and the second error. The adjusted model parameters are then used to return the obtained image sample, and the image sample is convolved using the encoder to obtain a feature image, until the model parameters that reach the training target value are obtained.

[0147] The image compression and classification apparatus provided by this invention extracts features from image samples through convolution using a convolutional neural network model, obtaining corresponding feature images. These feature images can be used simultaneously for image compression and image classification. This invention coordinates image compression and classification, allowing the two tasks to be performed concurrently. It achieves both image compression and classification with a relatively small amount of data, thus significantly saving computational resources.

[0148] Optionally, the model parameter adjustment module 840 is specifically used for:

[0149] The compressed image and the image sample are input into the discriminator, and the discriminator outputs the distinction result between the compressed image and the image sample, and the distinction result is used as the first error;

[0150] The generator and discriminator are respectively the generator network and the discriminator network in the generative adversarial network.

[0151] Optionally, the model parameter adjustment module 840 adjusts the parameters of the discriminator using the first error; adjusts the parameters of the encoder and generator using the first error; and adjusts the parameters of the encoder and classifier using the second error.

[0152] Optionally, such as Figure 9 As shown, with Figure 8 In comparison, this device also includes:

[0153] The testing module 910 is used to input the test image into the test network obtained based on the model parameters of the convolutional neural network model to be trained, and output the classification label test value.

[0154] The model parameter adjustment module 920 is specifically used to: when the test value of the classification label does not reach the training target value, return the obtained image sample using the adjusted model parameters, and use the encoder to perform convolution processing on the image sample to obtain a feature image, until the model parameters that reach the training target are obtained.

[0155] The image compression and classification apparatus provided by this invention utilizes a multi-task joint training technique to train a convolutional neural network model for image compression and image classification. The generator for image compression and the classifier for image classification share an encoder composed of convolutional layers. Thus, the trained convolutional neural network model can simultaneously perform image compression and image classification on the image to be processed.

[0156] In this embodiment, the same encoder is used for image compression and image classification, which can greatly reduce computing resources.

[0157] like Figure 10 As shown, the present invention provides an image compression and classification device for use after training the convolutional neural network model of the present invention, including the following modules.

[0158] The module 1010 acquires the trained convolutional neural network model, which includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier.

[0159] The feature extraction module 1020 acquires the image to be processed and performs a convolution operation on the image to be processed using the encoder to obtain a feature image;

[0160] The image compression and classification module 1030 inputs the feature image into the generator and outputs a compressed image of the image to be processed, and inputs the feature image into the classifier and outputs a classification label of the image to be processed;

[0161] The convolutional neural network model obtained through training is trained using image samples.

[0162] The image compression and classification apparatus provided by this invention utilizes a multi-task joint convolutional neural network model for image compression and classification, wherein the generator for image compression and the classifier for image classification share an encoder composed of convolutional layers. In this way, the convolutional neural network model can simultaneously perform image compression and image classification on the image to be processed. In this embodiment, the same encoder is used for image compression and image classification, a design that can significantly reduce computational resources.

[0163] In the technical solution of this invention, the same encoder is used for image compression and image classification. This design can greatly reduce computing resources.

[0164] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute an image compression and classification method, which includes:

[0165] Obtain a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0166] Image samples are acquired, and the image samples are convolved using the encoder to obtain feature images;

[0167] The feature image is input into the generator, which outputs a compressed image of the image sample; and the feature image is input into the classifier, which outputs the classification result of the image sample.

[0168] The first error between the compressed image and the image sample, and the second error of the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first error and the second error. The adjusted model parameters are used to return the obtained image sample. The image sample is then convolved using the encoder to obtain a feature image. This process continues until the model parameters that reach the training target value are obtained.

[0169] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the image compression and classification methods provided by the above methods, the method comprising:

[0171] Obtain a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0172] Image samples are acquired, and the image samples are convolved using the encoder to obtain feature images;

[0173] The feature image is input into the generator, which outputs a compressed image of the image sample; and the feature image is input into the classifier, which outputs the classification result of the image sample.

[0174] The first error between the compressed image and the image sample, and the second error of the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first error and the second error. The adjusted model parameters are used to return the obtained image sample. The image sample is then convolved using the encoder to obtain a feature image. This process continues until the model parameters that reach the training target value are obtained.

[0175] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image compression and classification methods provided by the methods described above, the method comprising:

[0176] Obtain a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier;

[0177] Image samples are acquired, and the image samples are convolved using the encoder to obtain feature images;

[0178] The feature image is input into the generator, which outputs a compressed image of the image sample; and the feature image is input into the classifier, which outputs the classification result of the image sample.

[0179] The first error between the compressed image and the image sample, and the second error of the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first error and the second error. The adjusted model parameters are used to return the obtained image sample. The image sample is then convolved using the encoder to obtain a feature image. This process continues until the model parameters that reach the training target value are obtained.

[0180] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image compression and classification method, characterized in that, include: Obtain a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier; Image samples are acquired, and the image samples are convolved using the encoder to obtain feature images; The feature image is input into the generator, which outputs a compressed image of the image sample; and the feature image is input into the classifier, which outputs the classification result of the image sample. The first error between the compressed image and the image sample, and the second error of the classification result are obtained. The model parameters of the convolutional neural network model are adjusted using the first error and the second error. The adjusted model parameters are used to return the obtained image sample. The image sample is convolved using the encoder to obtain the feature image until the model parameters that reach the training target value are obtained. In terms of image compression, adversarial loss, generator loss, and discriminator loss are utilized. The method for calculating the loss value is as follows: ； in, As the weight for adversarial loss, The weights for feature matching loss, To combat the loss value, The feature matching loss value; The formula for calculating the loss value of feature matching is: ； in, It is the expected value of the input image; The method for calculating the discriminant loss value is as follows: ； in, For parameters of the compressed image, These are the parameters of the image samples; For image classification, cross-entropy loss is used; The method for calculating the classification loss value is as follows: ； in, The labels corresponding to the image samples. Number of images; The generated loss value, classification loss value, and discriminant loss value are weighted to calculate the overall loss: ； in, For the overall loss, The weights of the generator, The weights of the discriminator, The weights of the classifier, To generate loss values, To determine the loss value, Loss value; The convolutional neural network model also includes: a discriminator, which takes as input the compressed image generated by the generator and the original image sample, and outputs the distinction between the compressed image and the image sample.

2. The image compression and classification method according to claim 1, characterized in that, The process of obtaining the first error between the compressed image and the image sample includes: The compressed image and the image sample are input into the discriminator, which outputs the distinction result between the compressed image and the image sample. The distinction result is used as the first error.

3. The image compression and classification method according to claim 2, characterized in that, The step of adjusting the model parameters of the convolutional neural network model using the first error and the second error includes: The parameters of the discriminator are adjusted using the first error.

4. The image compression and classification method according to claim 1, characterized in that, The method further includes, before obtaining model parameters that achieve the training objective, returning the acquired image samples using the adjusted model parameters, performing convolution processing on the image samples using the encoder to obtain feature images, and so on. The test image is input into the test network obtained based on the model parameters of the convolutional neural network model to be trained, and the test value of the classification label is output. The obtained image samples are returned using the adjusted model parameters. The image samples are then convolved using the encoder to obtain feature images, until model parameters that meet the training objective are obtained, including: If the test value of the classification label does not reach the training target value, the obtained image sample is returned using the adjusted model parameters. The image sample is then convolved using the encoder to obtain a feature image, until the model parameters that reach the training target are obtained.

5. The image compression and classification method according to claim 1, characterized in that, The step of adjusting the model parameters of the convolutional neural network model using the first error and the second error includes: The parameters of the encoder and generator are adjusted using the first error; The parameters of the encoder and classifier are adjusted using the second error.

6. An image compression and classification method, characterized in that, include: Obtain the trained convolutional neural network model, wherein the convolutional neural network model includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier; The image to be processed is acquired, and the encoder is used to perform a convolution operation on the image to be processed to obtain a feature image. The feature image is input into the generator, which outputs a compressed image of the image to be processed; and the feature image is input into the classifier, which outputs a classification label for the image to be processed. The convolutional neural network model is trained using image samples. In image compression, adversarial loss, generator loss, and discriminator loss are utilized; The method for calculating the loss value is as follows: ； in, As the weight for adversarial loss, The weights for feature matching loss, To combat the loss value, The feature matching loss value; The formula for calculating the loss value of feature matching is: ； in, It is the expected value of the input image; The method for calculating the discriminant loss value is as follows: ； in, For parameters of the compressed image, These are the parameters of the image samples; For image classification, cross-entropy loss is used; The method for calculating the classification loss value is as follows: ； in, The labels corresponding to the image samples. Number of images; The generated loss value, classification loss value, and discriminant loss value are weighted to calculate the overall loss: ； in, For the overall loss, The weights of the generator, The weights of the discriminator, The weights of the classifier, To generate loss values, To determine the loss value, Loss value; The convolutional neural network model also includes: a discriminator, which takes as input the compressed image generated by the generator and the original image sample, and outputs the distinction between the compressed image and the image sample.

7. An image compression and classification apparatus, characterized in that, include: The acquisition module acquires a convolutional neural network model to be trained, wherein the convolutional neural network model to be trained includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier. The feature extraction module acquires image samples and uses the encoder to perform convolution processing on the image samples to obtain feature images. The training module inputs the feature image into the generator and outputs a compressed image of the image sample; and inputs the feature image into the classifier and outputs the classification result of the image sample. The model parameter adjustment module obtains a first error between the compressed image and the image sample, and a second error of the classification result. It then adjusts the model parameters of the convolutional neural network model using the first and second errors. The adjusted model parameters are then returned to the acquired image sample. The encoder performs convolution processing on the image sample to obtain a feature image, until model parameters that meet the training target value are obtained. In terms of image compression, adversarial loss, generator loss, and discriminator loss are utilized. The method for calculating the loss value is as follows: ； in, As the weight for adversarial loss, The weights for feature matching loss, To combat the loss value, The feature matching loss value; The formula for calculating the loss value of feature matching is: ； in, It is the expected value of the input image; The method for calculating the discriminant loss value is as follows: ； in, For parameters of the compressed image, These are the parameters of the image samples; For image classification, cross-entropy loss is used; The method for calculating the classification loss value is as follows: ； in, The labels corresponding to the image samples. Number of images; The generated loss value, classification loss value, and discriminant loss value are weighted to calculate the overall loss: ； in, For the overall loss, The weights of the generator, The weights of the discriminator, The weights of the classifier, To generate loss values, To determine the loss value, Loss value; The convolutional neural network model also includes: a discriminator, which takes as input the compressed image generated by the generator and the original image sample, and outputs the distinction between the compressed image and the image sample.

8. An image compression and classification apparatus, characterized in that, include: The acquisition module acquires the trained convolutional neural network model, which includes an encoder composed of convolutional layers, a generator sharing the encoder, and a classifier. The feature extraction module acquires the image to be processed and performs a convolution operation on the image to be processed using the encoder to obtain a feature image. The image compression and classification module inputs the feature image into the generator and outputs a compressed image of the image to be processed; and inputs the feature image into the classifier and outputs a classification label of the image to be processed. The convolutional neural network model obtained through training is trained using image samples; In image compression, adversarial loss, generator loss, and discriminator loss are utilized; The method for calculating the loss value is as follows: ； in, As the weight for adversarial loss, The weights for feature matching loss, To combat the loss value, The feature matching loss value; The formula for calculating the loss value of feature matching is: ； in, It is the expected value of the input image; The method for calculating the discriminant loss value is as follows: ； in, For parameters of the compressed image, These are the parameters of the image samples; For image classification, cross-entropy loss is used; The method for calculating the classification loss value is as follows: ； in, The labels corresponding to the image samples. Number of images; The generated loss value, classification loss value, and discriminant loss value are weighted to calculate the overall loss: ； in, For the overall loss, The weights of the generator, The weights of the discriminator, The weights of the classifier, To generate loss values, To determine the loss value, Loss value; The convolutional neural network model also includes: a discriminator, which takes as input the compressed image generated by the generator and the original image sample, and outputs the distinction between the compressed image and the image sample.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the image compression and classification method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image compression and classification method as described in any one of claims 1 to 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the image compression and classification method as described in any one of claims 1 to 6.

Images