Industrial image compression method, device and system, and storage medium
By using a neural network model with multi-path residual asymmetric convolutional blocks and enhanced local attention mechanisms, the problem of capturing details and structure in industrial image compression is solved, achieving higher quality image reconstruction results.
Patent Information
- Application Number
- CN202411157913.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing image compression technologies struggle to effectively capture details and structures at different scales and orientations when processing industrial images, leading to a decline in image quality.
A neural network model employing multi-path residual asymmetric convolutional blocks and enhanced local attention mechanisms is combined with a channel-aware compression entropy model. Key features are extracted through a multi-path residual asymmetric convolutional downsampling module, and upsampling is performed in the decoder to restore details. Asymmetric convolutional kernels are used to focus local key features in different directions, thereby enhancing the image compression network's ability to express local details.
It improves the accuracy and detail recovery capability of image reconstruction, enhances the reconstruction quality of compressed image targets, and has a clearer effect, especially in texture details, making it suitable for complex industrial images.
Smart Images

Figure CN118900334A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to an industrial image compression method and device, a system, and a storage medium. Background Art
[0002] In recent years, industrial image compression has been an important research topic in the field of signal processing, aiming to achieve high-quality image compression and efficient storage. Classic image compression technologies such as JPEG, JPEG2000, WebP and HEVC have achieved remarkable success. However, with the rapid development of industrialization, higher and higher requirements are placed on the effectiveness and efficiency of image compression coding. The traditional hybrid image codecs used have some limitations and are not suitable for all categories of images. For example, transforming and quantizing in the form of image blocks will cause blocking effects. Due to the limitation of network bandwidth, in order to achieve low bit rate encoding, the traditional hybrid image codec method will lose details, resulting in image blur and distortion.
[0003] With the development of deep learning technology in computer vision. Ball first proposed a learning image compression method based on variational autoencoder (VAE), which has better rate-distortion performance than traditional lossy image compression technology methods, measured by indicators such as signal-to-noise ratio (PSNR) and multi-scale structural similarity index metric (MS-SSIM), showing great potential for practical application. Subsequently, by introducing nonlinear transformation techniques such as convolution plus generalized division normalization GDN, differentiable quantization such as uniform noise approximate quantization UNAQ and soft-to-hard quantization, and conditional probability models following Bayesian generation rules such as hyper-prior and joint autoregressive context and hyper-prior based on 2D PixelCNN, significant performance improvements have been gradually accumulated. In addition to the common MSE or MS-SSIM loss functions, some other loss functions have also been used to improve image quality, especially subjective quality, such as feature loss and adversarial loss.
[0004] At present, deep learning image compression models have good performance in natural scenes, but there are still some problems in the compression of industrial images, which cannot fully meet the needs of intelligent manufacturing. The main reason is that images in industrial environments usually contain a lot of details, complex structures, and various noises and interferences, such as light changes, dust, vibration, etc. These factors may lead to a decrease in image quality. Convolutional neural network (CNN) plays an important role in extracting features. It can remove redundant spatial information based on global information. When it comes to image compression, images usually undergo multiple upsampling and downsampling operations to reduce the size of the image and remove redundant information. In traditional image compression models, symmetric convolution is usually used for sampling operations. Symmetric convolution uses the same size of convolution kernel to operate in the horizontal and vertical directions. Although this method has a good effect in extracting image features, the symmetric convolution kernel limits the model's ability to capture details in a specific direction. When processing complex image structures, it may not be able to effectively capture feature differences in different directions. This limitation is particularly prominent when processing complex structures and details in industrial images. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an industrial image compression method and device, system, and storage medium to better capture details and structures at different scales and directions.
[0006] To achieve the above object, the present invention adopts the following technical solution:
[0007] An industrial image compression method, comprising:
[0008] Step S1, obtaining historical industrial images;
[0009] Step S2: According to the historical industrial images, a neural network model based on asymmetric convolution of multi-way residuals and enhanced local attention mechanism is obtained;
[0010] Step S3: input the historical industrial image to be processed into the neural network model for industrial image compression.
[0011] Preferably, the neural network model includes: a multi-way residual asymmetric convolution block, an enhanced local attention mechanism module and a channel-aware squeezing entropy model; wherein, the multi-way residual asymmetric convolution block includes: a multi-way residual asymmetric convolution downsampling module and a multi-way residual asymmetric convolution upsampling module; wherein, the multi-way residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-way residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
[0012] Preferably, the loss function L of the neural network model is:
[0013] L=λ·D+R
[0014] Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image;
[0015]
[0016] Among them, C, W, and H represent the channel, height, and width of the image respectively.
[0017]
[0018] Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
[0019] The present invention also provides an industrial image compression device, comprising:
[0020] Acquisition module, used to acquire historical industrial images;
[0021] The training module is used to obtain a neural network model based on asymmetric convolution and enhanced local attention mechanism based on historical industrial images;
[0022] The compression module is used to input the historical industrial images to be processed into the neural network model for industrial image compression.
[0023] Preferably, the neural network model includes: a multi-way residual asymmetric convolution block, an enhanced local attention mechanism module and a channel-aware squeezing entropy model; wherein, the multi-way residual asymmetric convolution block includes: a multi-way residual asymmetric convolution downsampling module and a multi-way residual asymmetric convolution upsampling module; wherein, the multi-way residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-way residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
[0024] Preferably, the loss function L of the neural network model is:
[0025] L=λ·D+R
[0026] Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image;
[0027]
[0028] Among them, C, W, and H represent the channel, height, and width of the image respectively.
[0029]
[0030] Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
[0031] The present invention also provides an industrial image compression system, comprising: a memory and a processor, wherein a computer program executed by the processor is stored in the memory, and the computer program executes the industrial image compression method when executed by the processor.
[0032] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the industrial image compression method when running.
[0033] The present invention has the following technical effects:
[0034] 1. In order to solve the problem that it is difficult to accurately extract important feature information from industrial images in the current feature extraction stage, a multi-way residual asymmetric convolution block (MRACB) is proposed. The multi-way residual asymmetric convolution downsampling MRACD module is used for encoder downsampling operations and extracting key features. The multi-way residual asymmetric convolution upsampling (MRACU module is used for decoder upsampling operations, which can better utilize the previously extracted high-level features, improve the accuracy of image reconstruction and detail recovery capabilities. And it can be effectively used with existing image compression models.
[0035] 2. In order to better capture the correlation between adjacent elements in the industrial image space, a flexible enhanced local attention mechanism (ELAM) is proposed, which can allocate high bitrate to the target area, thereby effectively improving the reconstruction quality of compressed image targets. Compared with other learning-based methods and traditional codecs, the present invention has a clearer effect on texture details.
[0036] 3. In view of the fact that there are few attention modules adapted to the channel entropy model in the current image compression network, after channel compression of the channel entropy model, a parameter-efficient attention module (CLAB) is designed for the entropy model, which can obtain local information and thus better grasp the specific details of the image. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0038] Figure 1 A flowchart of an industrial image compression method according to an embodiment of the present invention;
[0039] Figure 2 Schematic diagram of MRACNN; MRACD↓2 is a multi-way residual asymmetric convolution downsampling module, MRACU↓2 is a multi-way residual asymmetric convolution upsampling module, IGDN is the inverse GDN, ELAM is the enhanced local attention module, Q represents quantization, AE and AD are the arithmetic encoder and arithmetic decoder;
[0040] Figure 3 Schematic diagram of symmetric convolution and asymmetric convolution structure;
[0041] Figure 4 Schematic diagram of multi-path residual based asymmetric convolution block (MRACB);
[0042] Figure 5 Schematic diagram of the enhanced local attention mechanism module;
[0043] Figure 6 is the channel-aware squeeze entropy model; where CLAB represents the compressed enhanced local attention module applied to the channel entropy model, E and D represent arithmetic coding and arithmetic decoding, μ and σ represent variance and mean, and LRP represents the potential residual error;
[0044] Figure 7 It is the enhanced local attention module (CLAB) in the channel entropy model;
[0045] Figure 8 The rate-distortion curves of different methods on the Kodak dataset; (a) represents the distortion measured by the peak signal-to-noise ratio (PSNR) indicator, and (b) represents the distortion measured by the structural similarity (MS-SSIM) indicator;
[0046] Fig. 9 Rate-distortion curves of different methods on the CLIC-PRO dataset; (a) represents the distortion measured by the peak signal-to-noise ratio (PSNR) indicator, and (b) represents the distortion measured by the structural similarity (MS-SSIM) indicator;
[0047] Fig.10 Reconstruct image visualization for industrial production scenes;
[0048] Fig.11 Illustration of a visual comparison of images reconstructed for an industrial production workshop. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0051] Embodiment 1:
[0052] like Figure 1 As shown, an embodiment of the present invention provides an industrial image compression method, comprising:
[0053] Step S1, obtaining historical industrial images;
[0054] Step S2: According to the historical industrial images, a neural network model based on asymmetric convolution of multi-way residuals and enhanced local attention mechanism is obtained;
[0055] Step S3: input the historical industrial image to be processed into the neural network model for industrial image compression.
[0056] As an implementation method of an embodiment of the present invention, a neural network model (MRACNN) includes: a multi-way residual asymmetric convolution block (MRACB), an enhanced local attention mechanism module (ELAM) and a channel-aware squeezing entropy model; wherein the multi-way residual asymmetric convolution block includes: a multi-way residual asymmetric convolution downsampling module (MRACD) and a multi-way residual asymmetric convolution upsampling module (MRACU); wherein the multi-way residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-way residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
[0057] In traditional CNN-based image compression, the historical industrial image x passes through the main encoder g a After that, the standard deviation response y with spatial variation is obtained, that is, the potential feature obtained by the encoder, which is quantized and encoded by the quantizer Q to obtain the output The image is entropy encoded using the quantized data probability model. However, since the actual distribution is unknown, there is a difference between the probability model and the actual distribution. In order to minimize this difference, the embodiment of the present invention introduces a new variable z, which is then quantized, compressed, and transmitted as auxiliary information to achieve accurate estimation of the probability model.
[0058] In the compression transformation of the image, the encoder g a Map the historical industrial image to the latent feature y, and after passing through the quantizer Q, obtain the discrete representation of the latent feature y Then use the decoder g s According to the generated reconstructed image, the process is as follows:
[0059]
[0060] in, represents the optimization parameter, represents quantization,
[0061]
[0062] During the training phase, uniform noise U(-0.5, 0.5) is used as the quantization function, and a circular function is used to quantize the potential representation. The bit stream is generated by arithmetic entropy coding (AE). When the image needs to be reconstructed, the image is decoded by arithmetic entropy decoding (AD), and the decoding module reconstructs the compressed image. The decoder consists of a deconvolution layer, three MRACB upsampling modules, and three ELAM modules; the reconstructed image It can be obtained by the following formula:
[0063]
[0064] Among them, θ represents the optimization parameter, and the dependency between y is solved by introducing a new variable z:
[0065]
[0066] Among them, h a and h s represents the analysis and synthesis transformations in the autoencoder, and θ h represents the optimization parameters, is the estimated distribution based on z;
[0067] The entropy decoding result is input to the main decoder gs At one end, reconstruct the source image.
[0068] Furthermore, in the task of image processing, convolution operations are usually used to extract features from images. Both asymmetric convolution and the square convolution (or cube convolution) used in most convolutional networks belong to rectangular convolution. Both obtain effective feature maps through convolution operations on local images. The difference is that the local convolution used in asymmetric convolution is a rectangular block, while the square convolution used in square convolution is a square convolution block. Figure 3 The figure shows the asymmetric convolution method. The network built based on asymmetric convolution has proven the effectiveness of asymmetric convolution. In a deep neural network, the convolutional layer has a convolutional kernel of N×M. This convolutional kernel slides on the image with a stride. When the convolutional kernel cannot ensure that the kernel can completely scan the entire image to extract features due to the size of the image, padding is pre-applied to the image or features to ensure that the field of view of the convolutional kernel can cover the entire image or features and ensure that information is not lost. When this convolutional kernel passes over the image, it can use the local features or image object information within the N×N area to obtain more advanced features. Stacking multiple convolutional layers can extract more advanced feature information. In modern deep learning networks, multiple convolutional networks are often stacked. The quality of the features extracted by the stacked convolutional networks continuously improves as the number of stacked layers increases. That is, the shallow convolutional network can only extract low-level features, while the deep convolutional network can extract high-level and abstract feature information, that is, the features that cannot be directly observed between pixels of objects in the image.
[0069] When N = M, the convolutional kernel is a square convolutional kernel, which can extract features equally in both the horizontal and vertical directions; when N≠M, the convolutional kernel is an asymmetric convolutional kernel. Since this convolutional kernel is non-square, the feature extraction capabilities of this convolutional kernel in different directions are different. When N < M, the features extracted by the convolutional kernel in the horizontal direction are more global; when N > M, the features extracted by the convolutional kernel in the vertical direction are more global than the features extracted in the horizontal direction, that is, the range of attention is wider and has a larger receptive field.
[0070] Traditional convolution operations perform equal aggregation among all pixels, which may ignore local features and overestimate the role of non-critical features. To address this problem, an embodiment of the present invention introduces asymmetric convolution. Compared with square kernel convolution, asymmetric convolution kernels can focus on local areas in the horizontal and vertical directions by using one-dimensional convolution kernels such as (1×3 and 3×1) to enhance the expression of important details. By applying one-dimensional asymmetric convolution in the horizontal and vertical directions, local key features can be better extracted and represented. Subsequently, the enhanced local key features are converged into a square convolution to achieve the combination of global and local information. This method not only reduces the number of model parameters, but also speeds up the training speed of deep compression networks.
[0071] The embodiment of the present invention adopts a multi-way residual asymmetric convolution block (MRACB) to capture long-range context information and emphasizes image details by enhancing the effect of local key features at low cost. Figure 4 The MRACB structure shown is shown. It includes an upsampling module and a downsampling module. For the downsampling module, (Conv, s = 2) uses a convolution layer with a kernel size of 3×3 to capture local information. The parameter s is used to adjust the spatial size of the feature map. The feature map is then fed into an asymmetric block, which consists of three convolutional layers. (Conv, 1×n) and (Conv, n×1) are one-dimensional asymmetric convolutions, each containing a 1×3 convolution kernel and a 3×1 convolution kernel. (Conv, n×n) is a 3×3 square convolution kernel, which will be affected by (Conv, 1×n) and (Conv, n×1) during residual learning, and the extracted local key features will be refined to enhance the image compression network's ability to express local details. In addition, a GDN is followed for normalization. The feature extraction process is defined as
[0072]
[0073] y h =C(x,θ v )
[0074] y v =C(x,θ v )
[0075] y 3×3 =C(x,θ v )
[0076] y acb =[y h ,y v ,y 3×3 ]
[0077]
[0078] Where μ is the mean of the historical industrial image feature map, σ is the standard deviation of the feature map, ε is a small constant for numerical stability, and γ and β are learnable parameters. h represents the feature map in the horizontal direction, θ h Represents the parameters of horizontal convolution, y v represents the feature map in the vertical direction, θ v Represents the parameters of the vertical convolution, y 3×3 represents the feature map of 3x3 convolution, θ v Represents the parameters of the 3x3 convolution.
[0079] The residual network helps solve the problem of gradient disappearance and inserts a residual path into MRACB. Figure 2 As shown in the figure, the 3×3 (Conv, s=2) module of the main path of the multi-path residual asymmetric convolutional downsampling (MRACD) and the 3×3 (Conv, s=2) module in the residual path are replaced with a deconvolution layer (TConv, s=2), and the GDN is replaced with an IGDN layer for upsampling. The new structure is called multi-path residual asymmetric convolutional upsampling (MRACU).
[0080] Furthermore, the enhanced local attention module is Figure 5 As shown on the left, it consists of two branches: the first branch includes 3 residual convolution blocks, which are used to generate feature maps; the second branch consists of ELSA module, 3 residual convolution blocks, 1 1×1 convolution layer and sigmoid activation function, which is used to generate attention mask. The key technology of ELSA module is Hadamard attention. Hadamrd product (⊙) generates attention more effectively while maintaining high-order mapping relationship. Ghost head combines attention and static matrix to increase channel capacity. These two advantages enable ELSA Block to better embed local details.
[0081] ELSA can be expressed as:
[0082]
[0083] Among them, H n-m is the Hadamard attention value, G(·) is the ghost head mapping, F m is the output feature at m pixels, V n is the value vector at n pixels.
[0084] Hadamard attention can be expressed as:
[0085]
[0086] Among them, ⊙ is the Hadamard product, q m⊙k m and q n ⊙k n This can be achieved by querying the key feature map and simply multiplying its elements, using 1×1 convolutional filters to represent them equivalently. and
[0087] R represents the relevant operation performed on X. The output features of the ELSA module are subjected to the operation consisting of three residual convolutions and 1×1 convolution cascade in the mask branch. RB represents a single residual convolution operation. Conv is a 1×1 convolution, and the information exchange between channels is enhanced through a 1×1 convolution kernel. The operation is to multiply each pixel at the corresponding position of the two feature maps, and generate an importance mask through the Sigmoid activation function, so that the learned model pays more attention to complex areas, thereby improving the encoding performance. M is a real number, ranging from 0 to 1. When the real number M is multiplied sequentially with each pixel of the feature map in the first branch, Y can be obtained, and then a residual structure is used to allow the network to converge at a higher speed during training.
[0088] R(X)=Conv(RB (3) (F m ))
[0089] M=Sigmoid(R(F m ))
[0090]
[0091] Further, if Figure 6 and 7 As shown in the figure, the channel-aware squeezed entropy model module processes the potential representation y generated by the main encoder. Specifically, on the one hand, y is fed into the entropy module for image entropy estimation; on the other hand, channel-level separation is performed, that is, the potential image is divided into i roughly equal slices along the channel dimension, and the mean and variance of each slice are estimated separately by the autoregressive module. In addition, the mean and variance predicted by the previous layer also need to be considered. Then, the parts of the autoregressive code stream are integrated through the network and input into the decoder for decoding. In addition, in order to improve the parallel capability of the framework, the entropy parameters of each slice are also adjusted on the previously decoded slices. An attention module (CLAB) is designed for the entropy model to obtain local information, so as to better grasp the specific details of the image. Finally, the reconstructed image is generated by the decoder, which can be obtained as follows:
[0092]
[0093] in, represents the discrete representation of the latent feature y, g sbit decoder, θ represents the optimization parameter.
[0094] Since the input y i The number of feature channels of increases with the increase of slice index i, so y i The number of input channels can be expressed as:
[0095] i×(M / / a)+M
[0096] Among them, M is the number of channels of the latent variable and a is the total number of slices.
[0097] The total number of slices is fixed to 5, and the input channels of all slices are compressed to 128, that is, the output channels of the first 1×1 convolutional layer are set to 128, and the output channels are desqueezed to the original number using the desqueezing operation, and the output channels of the last 1x1 convolutional layer are set to i×(M / / a)+M.
[0098] As an implementation method of an embodiment of the present invention, the loss function L of the neural network model is:
[0099] L=λ·D+R
[0100] Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image;
[0101]
[0102] Among them, C, W, and H represent the channel, height, and width of the image respectively.
[0103]
[0104] Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
[0105] In the experiment, the hardware environment of Intel(R) Xeon(R) Platinum 8124M CPU and NVIDIA GeForce RTX3090Ti GPU was used. Programming and experiments were carried out based on CUDA12.1, Pytorch1.10.0 and python3.8.
[0106] All experiments were programmed and implemented in Pytorch and Python. To ensure fair experimental comparison, all learning compression models used adopted the same training strategy.
[0107] The KodaK dataset and CLIC-PRO dataset are used to compare the method of the embodiment of the present invention with different learning-based methods and traditional compression standards. The KodaK dataset contains 24 color images of different subjects, each with a resolution of 768x512 pixels. The CLIC-PRO dataset contains a set of images with different characteristics and challenges, including high dynamic range (HDR) images, images under low light conditions, images with complex textures and structures, etc. These two datasets are often used to evaluate the performance and quality of image compression algorithms.
[0108] For training, 300K images were randomly selected from the openimages set, and images of size 256×256 were randomly cropped during training. The batch size was set to 4 for all model training, the initial learning rate was set to 0.00001, and the Adam optimiser was used to optimise the network. The model was optimised using two quality metrics (MSE and MS-SSIM) as supervision. When the model was optimised using MSE, the λ values λ belonged to {0.0018, 0.0035, 0.0067, 0.0130, 0.025, 0.05}. When optimised using MS-SSIM, the λ values λ were {3, 5, 8, 16, 32, 60}.
[0109] For the main codec (g a and g s ) in the MRACU and MRACD modules, kernel_size is set to 3. For the main codec (g a and g s ), the num_heads of the main path is set to 8, the kernel_size is set to 7, and the settings of the CLAB module are the same as above. The number of channels M of the latent feature y is set to 320, and the number of channels M of z is set to 192. The other hyperparameters in the entropy model follow the settings in Table 1.
[0110] Table 1
[0111]
[0112] The evaluation indicators used are bpp, PSNR, SSIM and MS-SSIM. The bit rate (bits per pixel, bpp) represents the average bit stream resource consumption, and its calculation method is:
[0113]
[0114] Image quality loss reflects the performance loss caused by the compression algorithm at a specific bit rate, so a reasonable image quality evaluation standard is very important for measuring the performance of the compression algorithm.
[0115] The calculation formula of PSNR (Peak Signal-to-Noi se Ratio) peak signal-to-noise ratio is:
[0116]
[0117] Among them, MAX is the maximum value of the image pixel, and MSE (Mean Squared Error) is the mean square error of the pixel values between the reconstructed image and the original image. The larger the PSNR, the smaller the error between the reconstructed image and the original image, and the better the image quality. It is currently one of the most widely used image quality evaluation criteria.
[0118] SSIM (Structural Similarity Index Measure) Structural similarity is an indicator to measure the similarity between two images. This indicator is a perception-based model that regards image degradation as a change in the perception of structural information and generally measures the loss in a local range. It comprehensively considers the three indicators of brightness contrast and structural similarity, and is expressed as the mean, standard deviation, and covariance of the pixels in the window. The specific calculation method is:
[0119] SSIM(x,y)=[l(x,y)]α[c(x,y)] β [s(x,y)] γ
[0120] in,
[0121]
[0122] Represents the comparison factors of window x and window y regarding brightness, contrast, and structure. μ and σ represent the mean and standard deviation, respectively. xy Represents covariance, c is a constant. In actual use, α=β=γ=1, c3=c2 / 2 is usually set, so the formula is simplified to:
[0123]
[0124] The window is moved by pixels each time until the local structural similarity index of all positions is calculated and the average value is the structural similarity of the two images.
[0125] MS-SSIM (Multi-Scale SSIM) is an improvement on SSIM. It divides the image into two M-1Downsampling is performed to calculate the structural similarity of images at multiple scales.
[0126] When M=1, it indicates the original image size.
[0127]
[0128] Among them, brightness comparison is only performed on scale M.
[0129] The rate-distortion performance of MRACNN, traditional image compression methods, and other CNN- and transformer-based compression methods are verified on different datasets. Figure 8 and Fig. 9 In the figure, the performance of the model on the Kodak dataset and the CLIC PRO dataset is shown. Multi-scale structural similarity (MS-SSIM) and peak signal-to-noise ratio (PSNR) are used to measure these performances. At the same time, in order to facilitate comparison with other methods, MS-SSIM is converted to -10log 10 (1-(MS-SSIM)) for a clearer comparison. It can be observed from the figure that the rate-distortion characteristics of the deep learning-based image compression method are slightly higher than those of the traditional image compression method. At the same time, the multi-way residual asymmetric convolutional compression network (MRACNN) of the embodiment method of the present invention outperforms all other methods, including deep learning-based image compression methods and traditional classical image compression methods. In addition, the embodiment method of the present invention achieves state-of-the-art compression performance in terms of both PSNR and MS-SSIM indicators, which verifies the effectiveness of the embodiment method of the present invention. The asymmetric convolution feature module based on multi-way residuals can better extract important feature information in the image, while the dual-branch enhanced local attention mechanism pays more attention to high-contrast areas, thereby allocating more bits to them. At the same time, the comparison results on the CLIC professional verification dataset also show the robustness of the model, which can maintain excellent performance even in complex scenes.
[0130] In image compression tasks, feature maps should focus more on regions with important details and structures in the image, because these regions are crucial for preserving image quality and reconstructing key features of the compressed image.
[0131] Then, the running time of the method of the embodiment of the present invention on the CPU was tested, and compared with the execution time of other traditional image compression methods vtm, av1, Webp, bpg, jpeg, and jpeg2000 on the CPU, and then the running time of the method of the embodiment of the present invention on the GPU was shown, and then the execution time of other deep learning-based image compression methods on the GPU was shown, as shown in Table 2. Through the comparison in Table 2, it can be clearly found that due to the high complexity of the CNN model, the execution speed of the method of the embodiment of the present invention on the CPU is slower than some traditional codecs, but is better than several traditional image compression methods such as VCC, AV1 and JPEG2000.
[0132] With the support of GPU, the model of the method of the embodiment of the present invention achieves faster encoding and decoding speed. Comparison with several learning image compression models is shown in Table 3. Although the method of the embodiment of the present invention is slightly inferior to SwinTChARM in encoding speed and decoding speed, it is still better than other image compression methods based on deep learning. The MRACB module only brings very little computing overhead, but gains performance benefits. In addition, the method of the embodiment of the present invention also has good scalability and flexibility, and can be easily applied to large-scale image compression scenarios. Although most of the image compression methods based on deep learning are shorter in encoding time than traditional algorithms under the support of GPU, due to the strict serial relationship of the decoder, the decoding time is far longer than the traditional algorithm.
[0133] Table 2
[0134] Method Encoding Decoding Ours 1.18s 0.21s vtm 129.21s 0.14s av1 22.24s 0.037s Webp 0.059s 0.007s bpg 0.66s 0.17s jpeg 0.012s 0.006s jpeg2000 0.50s 0.48s
[0135] Table 3
[0136]
[0137]
[0138] 1) Contribution of MRACB, ELAM, and CLAB to MRACNN: In order to prove that the MRACB module, ELAM module, and CLAB module can bring significant gains in image compression and improve RD performance, an ablation experiment was conducted on the Kodak dataset. The experimental results are shown in Table 4.
[0139] Table 4
[0140]
[0141] The Baseline+MRACB model only replaces the convolutional modules and GDN modules in the codec with multi-way residual asymmetric convolution for up and down sampling. For a fairer comparison, the Baseline+ELAM model replaces the WAM module in the Baseline model with the ELAM module. The final Baseline+CLAB model still uses WACNN, but only changes the entropy model.
[0142] The WRACB module achieved more RD improvement. Compared with the baseline, the PSNR increased by 0.16dB. At the same time, the ELAM module and the CLAB module also improved by about 0.01db. It is worth mentioning that according to the time analysis of different codecs in Table 3, MRACNN only increased the time cost by 0.02 seconds compared with the previous model, which is still at the same order of magnitude. This shows that the MRACB module, ELAM module, and CLAB module can indeed effectively improve the model performance while maintaining a low time cost. The introduction of these modules has brought new ideas and methods to the field of image compression, and is expected to be more widely used in the future.
[0143] 2) The size of the convolution kernel in ELAM: Next, the performance of the ELAM module under different convolution kernel sizes is compared, taking BD-rate as an indicator. Minnen2020+WAM is selected as the anchor point, and ELAMn is used to represent the size of the convolution kernel in ELAM. The convolution kernel of ELAM in ELAM3 is 3, the convolution kernel of ELAM in ELAM5 is 5, and the convolution kernel of ELAM in ELAM3 is 7. The results are shown in Table 5. It can be seen that ELAM7 has the best performance and ELAM3 has poor performance. And the model using ELAM7 can save 1.67% of the bit rate. Therefore, the kernel_size values of ELAM and CLAB of MRACNN in the experiment are both set to 7, and a larger receptive field leads to better rate-distortion performance.
[0144] Table 5
[0145]
[0146] 3) The impact of each module on industrial image compression:
[0147] Besides, the influence of each module on the image compression and reconstruction of industrial production workshops is further discussed.
[0148] Fig.10The following diagram shows the reconstruction images of industrial production scenes using different compression methods under the PSNR distortion evaluation scale under similar bit rates. The first image is the initial image, and the following images are the results of images compressed by traditional image compression methods and other compression methods. It can be observed that compared with other reconstructed images (kodim1.png) from industrial production scenes, the method of the embodiment of the present invention achieves the best performance at the lowest bit rate.
[0149] Fig.11 A comparison of the multi-channel residual asymmetric convolution module (MRACB), ELAM, CLAB and other modules for industrial image compression and reconstruction is given. Fig.11 The results show that compared with the traditional WACNN method, MRACB shows higher compression performance and better image quality in the compression and reconstruction of industrial production workshop images. The high-resolution industrial production images can be improved by 0.429db at a lower compression ratio. Specifically, MRACB can more effectively retain key details and structures, reduce the loss of information during compression, and thus provide clearer and more accurate visual effects during image reconstruction. It is worth noting that the methods using ELAM and CLAB can also retain more details at lower bpp.
[0150] Embodiment 2:
[0151] The embodiment of the present invention further provides an industrial image compression device, comprising:
[0152] Acquisition module, used to acquire historical industrial images;
[0153] The training module is used to obtain a neural network model based on asymmetric convolution and enhanced local attention mechanism based on historical industrial images;
[0154] The compression module is used to input the historical industrial images to be processed into the neural network model for industrial image compression.
[0155] As an implementation method of an embodiment of the present invention, a neural network model includes: a multi-channel residual asymmetric convolution block, an enhanced local attention mechanism module and a channel-aware squeezing entropy model; wherein, the multi-channel residual asymmetric convolution block includes: a multi-channel residual asymmetric convolution downsampling module and a multi-channel residual asymmetric convolution upsampling module; wherein, the multi-channel residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-channel residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
[0156] As an implementation method of an embodiment of the present invention, the loss function L of the neural network model is:
[0157] L=λ·D+R
[0158] Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image;
[0159]
[0160] Among them, C, W, and H represent the channel, height, and width of the image respectively.
[0161]
[0162] Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
[0163] Embodiment 3:
[0164] The present invention also provides an industrial image compression system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes the industrial image compression method when executed by the processor.
[0165] Embodiment 4:
[0166] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the industrial image compression method when running.
[0167] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.
Claims
1. An industrial image compression method, characterized in that: include: Step S1, obtaining historical industrial images; Step S2: According to the historical industrial images, a neural network model based on asymmetric convolution of multi-way residuals and enhanced local attention mechanism is obtained; Step S3: input the historical industrial image to be processed into the neural network model for industrial image compression.
2. The industrial image compression method according to claim 1, characterized in that: The neural network model includes: a multi-way residual asymmetric convolution block, an enhanced local attention mechanism module and a channel-aware squeeze entropy model; wherein, the multi-way residual asymmetric convolution block includes: a multi-way residual asymmetric convolution downsampling module and a multi-way residual asymmetric convolution upsampling module; wherein, the multi-way residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-way residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
3. The industrial image compression method according to claim 2, characterized in that: The loss function L of the neural network model is: L=λ·D+R Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image; Among them, C, W, and H represent the channel, height, and width of the image respectively. Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
4. An industrial image compression device, characterized in that: include: Acquisition module, used to acquire historical industrial images; The training module is used to obtain a neural network model based on asymmetric convolution and enhanced local attention mechanism based on historical industrial images; The compression module is used to input the historical industrial images to be processed into the neural network model for industrial image compression.
5. The industrial image compression device according to claim 4, characterized in that: The neural network model includes: a multi-way residual asymmetric convolution block, an enhanced local attention mechanism module and a channel-aware squeeze entropy model; wherein, the multi-way residual asymmetric convolution block includes: a multi-way residual asymmetric convolution downsampling module and a multi-way residual asymmetric convolution upsampling module; wherein, the multi-way residual asymmetric convolution downsampling module is used to downsample and extract key features in the encoder, and the multi-way residual asymmetric convolution upsampling module is used to upsample in the decoder to restore details and reconstruct the image.
6. The industrial image compression device according to claim 5, characterized in that: The loss function L of the neural network model is: L=λ·D+R Among them, R represents the compression bit rate, λ represents the balance between the control rate and distortion, and D represents the degree of distortion between the reconstructed image and the original image; Among them, C, W, and H represent the channel, height, and width of the image respectively. Among them, x c,i,j Represents the pixel value of the cth channel of the input image at position (i, j), Represents the pixel value of the cth channel of the reconstructed image at position (i, j), represents the potential representation of the quantized primary encoder output, represents the latent representation of the quantized hyper-prior encoder output, and They are and is the probability density model of , ψ represents the probability density model of the variable distribution parameter.
7. An industrial image compression system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the industrial image compression method according to any one of claims 1 to 3 is executed.
8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the industrial image compression method according to any one of claims 1 to 3 when running.
Citation Information
Patent Citations
End-to-end video compression method and device and computer readable storage medium
CN114095728A
Image compression model based on graph attention and asymmetric convolutional network
CN115512199A
Image compression method based on enhanced multi-scale residual generative adversarial network
CN117173263A
Image compression method based on frequency domain decomposition
CN117676149A
Image compression system and method based on wavelet diffusion model
CN117834923A
