Image compressed sensing reconstruction method and system based on dynamic convolution and feature fusion

Abstract

The present invention discloses a method and system for image compression sensing reconstruction based on dynamic convolution and feature fusion, and relates to the field of image processing technology. The method comprises the following steps: obtaining an original image to be compressed sensing and reconstructed; inputting the original image into a trained image compression sensing model, performing block sampling and extracting an initialized reconstructed feature map through a sampling subnet and an initialized reconstruction subnet in the model, and then inputting the initialized reconstructed feature map into a deep reconstruction subnet in the model, respectively extracting multi-scale dynamic convolution local features and global features in turn through parallel dynamic convolution branches and Transformer branches, and fusing two features of the same scale through a weighted feature fusion module, and finally outputting the fused features of the image. After reconstruction, the model outputs a high-quality reconstructed image. The present invention adopts dynamic convolution and Transformer branch structures, combined with feature adaptive fusion, to effectively improve the quality of the reconstructed image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image compression sensing reconstruction method and system based on dynamic convolution and feature fusion. Background Art

[0002] Compressed sensing (CS) is an innovative sampling method for video image signals. It can significantly improve the sampling speed and reconstruct the signal with high quality, so it has shown wide application potential in the field of video image processing. In recent years, deep learning technology has shown excellent performance far exceeding other methods in the field of computer vision images. At present, a variety of image compression sensing methods based on deep learning have been studied, among which convolutional neural network (CNN) occupies the mainstream. At the same time, the image compression sensing model based on Transformer has also been proposed. Although the above deep learning compressed sensing reconstruction model has made great progress in image reconstruction quality, there are still some problems in the model: (1) The image compressed sensing method based on CNN has made great use of the advantages of CNN in capturing local features, but CNN will ignore global features during processing; (2) Since Transformer is migrated from the field of natural language processing (NLP), it is more effective in similar tasks of capturing global sequences, but lacks the emphasis on local features; (3) The existing method of combining the advantages of CNN and Transformer needs to be further improved. It often has the problem that the Transformer branch accounts for too large a proportion and ignores the influence of some local features, and the existing global and local feature fusion effect is not good, resulting in low quality of final image reconstruction. Summary of the invention

[0003] To address the deficiencies of the above-mentioned prior art, the present invention provides an image compressed sensing reconstruction method and system based on dynamic convolution and feature fusion. By improving the network structure, a dynamic convolution and Transformer combined image compressed sensing network (DCCT-Net) model is proposed. In the process of image reconstruction, parallel multi-layer dynamic convolution (DCNN) and Transformer branch structures are used to extract dynamic convolution local features and global features respectively. Moreover, while combining the excellent local feature processing capability of CNN and the global feature grasping capability of Transformer, feature adaptive fusion is used to effectively improve the quality of the reconstructed image.

[0004] In a first aspect, the present invention provides an image compressed sensing reconstruction method based on dynamic convolution and feature fusion.

[0005] An image compression sensing reconstruction method based on dynamic convolution and feature fusion, comprising:

[0006] Obtaining the original image to be reconstructed by compressed sensing;

[0007] The original image is input into the trained image compression sensing model. The sampling subnet and the initialization reconstruction subnet in the model are used to perform block sampling and extract the initialization reconstruction feature map. The initialization reconstruction feature map is then input into the deep reconstruction subnet in the model. The multi-scale dynamic convolution local features and global features are extracted in turn through the parallel dynamic convolution branch and Transformer branch. The two features of the same scale are fused through the weighted feature fusion module, and the fusion features of the image are finally output. After reconstruction, the model outputs a high-quality reconstructed image.

[0008] In a second aspect, the present invention provides an image compression sensing reconstruction system based on dynamic convolution and feature fusion.

[0009] An image compression sensing reconstruction system based on dynamic convolution and feature fusion, comprising:

[0010] An image acquisition module, used to acquire the original image to be reconstructed by compressed sensing;

[0011] The image reconstruction module is used to input the original image into the trained image compression sensing model, and perform block sampling and extract the initialized reconstructed feature map through the sampling subnet and the initialized reconstruction subnet in the model. The initialized reconstructed feature map is then input into the deep reconstruction subnet in the model, and the multi-scale dynamic convolution local features and global features are extracted in turn through the parallel dynamic convolution branch and Transformer branch. The two features of the same scale are fused through the weighted feature fusion module, and finally the fused features of the image are output. After reconstruction, the model outputs a high-quality reconstructed image.

[0012] In a third aspect, the present invention further provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps of the method described in the first aspect are completed.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the method described in the first aspect.

[0014] One or more of the above technical solutions have the following beneficial effects:

[0015] 1. The present invention provides an image compression sensing reconstruction method and system based on dynamic convolution and feature fusion. By improving the network structure, a dynamic convolution and Transformer combined image compression sensing network (DCCT-Net) model is proposed. In the process of image reconstruction, a parallel multi-layer dynamic convolution (DCNN) and Transformer branch structure are used. The initialized reconstructed image signal is input into two branches for local and global feature extraction, and the extracted features of each layer are efficiently and adaptively fused to facilitate the feature extraction operation of the next layer of DCNN and Transformer. Finally, after multi-layer dual-structure parallel branches, fused features are obtained, and high-quality reconstructed images are obtained after reconstruction. The present invention adopts feature adaptive fusion while combining the excellent local feature processing capability of CNN and the global feature grasping capability of Transformer, so as to effectively improve the quality of the reconstructed image.

[0016] 2. Considering that the existing image compression sensing model uses the same operation to treat image blocks with very different contents during operation, which will lead to biased feature extraction, the present invention proposes a feature dynamic enhancement module (FDAM), which treats the salient contents of several image blocks differently, improves the CNN branch into a dynamic convolution DCNN branch, and considers the saliency of the content of each image block through convolution operations, further improving the performance of the image compression sensing model in local features and reducing the additional burden on calculations.

[0017] 3. Considering that most of the existing traditional feature fusion methods cannot obtain the optimal fusion solution of multiple features, the present invention proposes a weighted fusion module (WFM). Considering the different situations when fusion is required each time, an inconsistent fusion strategy is adopted to achieve full complementarity between the local features captured by DCNN and the global features captured by Transformer, so that the fusion results can better act on the next operation, thereby ensuring the comprehensiveness of the final extracted features and improving the image reconstruction quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 It is an overall block diagram of the image compression sensing reconstruction method based on dynamic convolution and feature fusion according to an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the structure of a feature dynamic enhancement module in an embodiment of the present invention;

[0021] Figure 3Schematic diagram of the structure of the weighted feature fusion module in an embodiment of the present invention;

[0022] Figure 4 The figure is a schematic diagram of the results of image reconstruction using the method described in the embodiment of the present invention and other existing methods. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary only, are intended to describe specific embodiments, are intended to provide further explanation of the present invention, and are not intended to limit exemplary embodiments according to the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] Embodiment 1

[0025] This embodiment provides an image compression sensing reconstruction method based on dynamic convolution and feature fusion, comprising the following steps:

[0026] Obtaining an original image to be reconstructed by compressed sensing;

[0027] The original image is input into the trained image compression sensing model. After the sampling subnet and the initialization reconstruction subnet in the model, block sampling is performed and the initialization reconstruction feature map is extracted. The initialization reconstruction feature map is then input into the deep reconstruction subnet in the model. After the parallel dynamic convolution DCNN branch and Transformer branch, multi-scale dynamic convolution local features and global features are extracted respectively. The two features of the same scale are fused through the weighted feature fusion module, and finally the fusion features of the image are output. After reconstruction, the model outputs a high-quality reconstructed image.

[0028] In the image compression sensing reconstruction method based on dynamic convolution and feature fusion proposed in this embodiment, a dynamic convolution and Transformer combined image compression sensing network (DCCT-Net) model, referred to as the image compression sensing model, is proposed through network structure improvement. The model is used to achieve high-quality reconstruction of the original image. The following content introduces the dynamic convolution and Transformer combined image compression sensing model proposed in this embodiment in more detail.

[0029] Similar to the traditional image compression sensing model based on deep learning, the overall framework of DCCT-Net proposed in this embodiment is as follows: Figure 1 As shown, it includes a sampling subnet, an initialization reconstruction subnet and a depth reconstruction subnet, where the first two subnets are used to reconstruct the original image. Sampling and initial reconstruction are performed, and the deep reconstruction subnet is the key research part of the model. This embodiment designs a parallel dynamic convolution (DCNN) branch and a Transformer branch in the deep reconstruction stage, and designs a weighted feature fusion module to combine the features of the two branches, so as to achieve complementary advantages of DCNN and Transformer and obtain high-quality reconstructed images.

[0030] Specifically, an original image to be compressed sensing reconstructed is obtained, and the original image is input into an image compression sensing model for image feature extraction and reconstruction, including:

[0031] First, the sampling subnet performs block sampling to obtain several non-overlapping image blocks; the sampled image is input to the initialization reconstruction subnet, and the initialization reconstruction is performed to obtain the initialization reconstruction feature map. Specifically, based on block compressed sensing (BCS), in the sampling stage of compressed sensing, a size of Original image Split into non-overlapping image blocks, the size of each image block is ; Then, in the process of initialization reconstruction and depth reconstruction, the operation is no longer based on the entire image but on small non-overlapping blocks after segmentation. After a simple initialization reconstruction, it is input into the deep reconstruction subnet. In the above process, Indicates that image compression sensing is performed by x The measured values ​​obtained by sampling, parameters Represents the sampling matrix, that is, the sampling stage of image compression sensing is performed: y= x , through the sampling matrix right x Sampling to obtain y , in the subsequent initialization and reconstruction phase, The reconstructed image is initialized and reconstructed, and the image is the initialized reconstructed feature map.

[0032] Secondly, the dual-branch parallel deep reconstruction network structure is as follows Figure 1As shown, the deep reconstruction subnet includes parallel dynamic convolution DCNN branches and Transformer branches, and a multi-layer weighted feature fusion module; wherein the dynamic convolution DCNN branch includes multiple layers of sequentially connected feature dynamic enhancement modules (FDAM), the feature dynamic enhancement module is used to extract the dynamic convolution local features of the input image, and each layer of feature dynamic enhancement module outputs dynamic convolution local features of different scales respectively; the Transformer branch includes multiple layers of Transformer blocks, the Transformer blocks are used to extract the global features of the input image, and each layer of Transformer blocks outputs global features of different scales; each layer of feature dynamic enhancement module corresponds to each layer of Transformer blocks respectively, in this embodiment, the DCNN and Transformer branches are composed of corresponding four layers; the extracted dynamic convolution local features and global features are feature fused through the weighted feature fusion module of the corresponding layer to obtain fused features, and the fused features are then input as input features to the next Transformer block.

[0033] Furthermore, in order to adapt to the characteristics of the larger size during the Transformer processing and to effectively fuse the features of the DCNN branch and the Transformer branch, an upsampling operation is performed at the end of each feature dynamic enhancement module in the DCNN branch, that is, the first layer feature dynamic enhancement module ( ) and Transformer blocks ( ) The output results DF1 (i.e., dynamic convolution local features) and TF1 (i.e., Transformer features) are of the same size, and the size of the results extracted and output by each subsequent layer (from DF1, TF1 to DF4, TF4) gradually increases, from the initial 8×8 to 64×64 at the output. At this time, DF i and TF i , i = 1234, and then they are input into the weighted feature fusion module (WFM) for feature fusion. The result of WFM is applied to the Transformer branch again, that is, WFM i The result acts on TF i+1 .

[0034] In the above process, considering the influence of the initialization reconstruction result on the final reconstruction quality, in the first layer weighted feature fusion module, i.e., the first WFM, the dynamic convolution local features and global features extracted by the first layer feature dynamic enhancement module and the Transformer block are combined with the initialization reconstruction result F init They are used as the input of WFM1 to perform multiple feature fusion inputs to further improve the reconstruction accuracy.

[0035] (1) Feature Dynamic Enhancement Module (FDAM)

[0036] Considering that after the image is divided into blocks, each small image block (a total of n The contents of each small image block are different, so the importance of each small image block during reconstruction is also different, such as Figure 2 As shown, the three image frames represent three image blocks with different reconstruction difficulties, including simple background blocks and complex portrait blocks. In order to reflect the difference in content of different image blocks, unlike the traditional convolutional neural network CNN that uses fixed convolution operations to process all image blocks, this embodiment introduces the idea of ​​dynamic convolution on the DCNN branch of the model and proposes a feature dynamic enhancement module FDAM, in which a dynamic convolution layer applicable to each image block is set.

[0037] Specifically, the image is input into the feature dynamic enhancement module. According to the different saliency between each image block in the input image, each image block is input into the set convolution layer. The content saliency of the image block is analyzed through the convolution operation of the 1×1 convolution kernel. The convolution kernel is dynamically generated for each image block through the convolution operation, that is, 、 ... , for all image blocks (i.e. 、 、... ), the shapes of the generated multiple convolution kernels are matched to the format of the spatial position of each image block; then, the input image is expanded, and the dynamically generated convolution kernel is used as the dynamic convolution layer, and each image block is convolved with the corresponding dynamic convolution layer to output the dynamic convolution local features. Further, the dynamic convolution local features of the entire input image are obtained by splicing, that is, the local feature map of the input image is extracted.

[0038] Through the above process, it is ensured that the convolution kernel can be dynamically adjusted according to the different features of the input. Each dynamic convolution layer will only serve the corresponding image block. Each time a different small image block is processed, only the corresponding dynamic convolution layer is called for operation.

[0039] In the compressed sensing reconstruction process, considering that the simple image block has less influence on the reconstruction quality than the complex image block due to its less details, the introduction of the dynamic convolution idea can greatly simplify the reconstruction effort of simple image blocks (such as smooth walls, calm sea surface, etc.), and spend more energy on complex image blocks with more details and greater reconstruction difficulty to improve their reconstruction quality, and finally achieve high-quality reconstruction of the entire picture.

[0040] (2) Weighted Feature Fusion Module (WFM)

[0041] Considering that the DCNN and Transformer branches in the model have different focuses, which respectively attach importance to the local features and global features of the image blocks, in order to combine the strengths of the two as much as possible and maximize the complementarity of the two in terms of characteristics and influence on features, on the one hand, it is necessary to design a simple traditional feature fusion scheme selection module in the model design process, that is, to select the method that can achieve the best image reconstruction quality from the existing mainstream feature fusion methods, so as to solve the problem that the model cannot break through the bottlenecks of peak to noise ratio (PSNR) and structural similarity (SSIM) in practical applications and cannot achieve the expected effect; on the other hand, Figure 1 As shown, the weighted feature fusion module WFM i The result is output to the Transformer branch and acts on the next layer of Transformer block (TB i+1 ).

[0042] Taking into account the better feature fusion effect mentioned above and the bias of the fused features focusing on the Transformer branch, such as Figure 3 As shown, this embodiment proposes a weighted feature fusion module (WFM). Specifically, the dynamic convolution local features and global features extracted by the DCNN and Transformer blocks are input into the weighted feature fusion module for feature fusion, wherein a plurality of feature fusion methods are used to fuse the dynamic convolution local features and global features to obtain a plurality of fusion features, and the plurality of feature fusion methods include direct splicing, element-by-element multiplication, bilinear pooling, etc.; by adaptively learning the fusion parameters (i.e., the weight of each feature fusion method) based on the fusion effect, a better fusion result is finally obtained, that is, in this step, based on the weight of each feature fusion method determined by adaptive learning during the training process, a plurality of fusion features are weightedly fused to obtain the final fusion feature. At the same time, the first layer of the weighted feature fusion module WFM in the model i At runtime, the previous layer of weighted feature fusion module WFM will be used by default. i-1 The parameters are adjusted, and on this basis, learning continues to adapt to the fusion features required in the next stage. This measure can maximize the fusion effect of DCNN and Transformer while reducing the running time of the model.

[0043] In the above method proposed in this embodiment, the original image is first sampled in blocks, and the initial reconstruction result is obtained by initializing and reconstructing the block sampling results, and the initial reconstruction result is input into the DCNN and Transformer branches at the same time; in the DCNN branch, a dynamic feature enhancement method is adopted to perform different convolution operations on regions with different significance, and in the Transformer branch, the global features of the input feature map are captured; in addition, the results obtained in each layer of the two branches are input into the weighted feature fusion module, and the advantages of the two are combined to obtain a more accurate reconstruction result.

[0044] Furthermore, to prove the effectiveness of this embodiment, the COCO dataset containing tens of thousands of images that can be used for multiple computer vision tasks is used to train the above-mentioned image compression sensing model. In the model performance testing phase, existing datasets such as Urban100, BSD100, BSD68, Set11, Set5, etc. are used to fully verify the reconstruction accuracy of the model on a large number of image tasks and a small number of image tasks to avoid the possibility of inaccurate results due to the small difference in the number of images in the test datasets. The model measures the performance of the model by calculating the peak noise ratio (PSNR) and structural similarity (SSIM) of the reconstructed image.

[0045] Step S1, obtain a number of original image samples and their corresponding reconstructed image samples to construct a training data set; and set training hyperparameters at the same time.

[0046] Step S1-1: Use the COCO dataset, a commonly used training dataset in the field of computer vision, to train the model.

[0047] Step S1-2: manually set the batch size of each round in the training process and the image block size according to the specific situation. Similarly, continuously adjust the relevant parameters in the Transformer according to the quality of the reconstructed image.

[0048] Step S2: Use the training data set to train the constructed image compression sensing model, extract the fusion features of the input image samples, and generate a reconstructed image based on the fusion features.

[0049] Specifically, the input image Get the measured value by sampling the subnet and initializing the reconstructed subnet And the initialization result ; Initialize the result (i.e., the initialized reconstructed feature map) is input into the deep reconstruction subnet for deep reconstruction, specifically including:

[0050] Step S2-1: The input is sent to the dynamic convolution branch. In the dynamic feature enhancement module, a convolution layer is used to generate a dynamic convolution layer suitable for each small image block according to the different saliency of each small image block. Then, each dynamic convolution layer is used to process each small image block corresponding to it to obtain the first dynamic convolution layer in the DCNN branch. i Layer Results .

[0051] Step S2-2: Input to the Transformer branch, operate on the input feature map through the Transformer block, and get the first i Results of the layer Transformer branch .

[0052] Step S2-3: connect the two branches i Layer Results and At the same time, enter i In the weighted feature fusion module, the two features are fused. In order not to ignore the initialization reconstruction result The impact on the quality of the final image reconstruction will be It is not only input into the DCNN and Transformer branches, but also into the first weighted feature fusion module. That is, the first weighted feature fusion module realizes , , A fusion of the three.

[0053] Step S2-4: In order to adapt to the size characteristics of the Transformer application, an upsampling operation is performed after each layer of DCNN.

[0054] In the above step S2-3, the specific operations in the weighted feature fusion module include:

[0055] Step S2-3-1: In the first weighted feature fusion module, first , , The initial weights of the three are all set to 1. By continuously comparing the fusion quality, the weights of the three are changed to obtain the final optimal quality, and this result is passed back to the Transformer branch for use in the next layer of Transformer blocks;

[0056] Step S2-3-2: In all weighted feature fusion modules except the first one, the weight values ​​in the previous WFM are inherited first, and learning and optimization are continuously performed on this basis to finally obtain the weight values ​​that are most suitable for the second one. i layer , The weights are calculated and the results are passed back to the Transformer.

[0057] Step S2-3-3: In the last WFM, the output result does not need to be passed back to the Transformer branch, but will be directly used as the output of the DCCT-Net model.

[0058] Furthermore, through the above steps, the fusion features of each image block are output, and all the fusion features are spliced ​​(Cat) to obtain the fusion features of the input image sample, which are reconstructed based on the fusion features (this reconstruction step is not shown in the figure) to generate a reconstructed image; as another implementation method, the fusion features of each image block are output, reconstructed separately and then spliced ​​to obtain the final reconstructed image.

[0059] Step S3, based on the generated reconstructed image and reconstructed image samples, calculate the peak-to-noise ratio and structural similarity of the reconstructed image; after cyclic iterative training, adjust the model parameters in each iteration process until the peak-to-noise ratio and structural similarity are maximized, and complete the iterative training; wherein the model parameters include the weight values ​​of different feature fusion methods in the weighted feature fusion module, and the hyperparameters of each module.

[0060] Through the above method, the training of the image compression sensing model is completed, and the trained model is used to perform compression sensing reconstruction on the original image to be processed to obtain a reconstructed image with better quality.

[0061] Furthermore, to prove the superior performance of the method described in this embodiment, this method is objectively compared (PSNR and SSIM) with other existing methods or network models, and the experimental results fully verify the superiority of the method proposed in this embodiment. The experimental results of five different sampling rates in the Urban100 dataset containing 100 pictures are shown in Table 1 below, and the results of comparison with different models in the Set11 dataset containing fewer pictures are shown in Table 2 below. The double underline in the table indicates the optimal result. According to Tables 1 and 2, it can be seen that the method proposed in this embodiment has a better implementation effect.

[0062] Table 1 Comparison results of different image compression sensing network models

[0063]

[0064] Table 2 Comparison results of different image compression sensing network models with fewer test images

[0065]

[0066] Table 3 Comparison results of different image compression sensing network models on other datasets

[0067]

[0068] At the same time, combined with the content of Table 3 above, it can be seen that the model proposed in this embodiment has a greatly improved ability in reconstructing pictures, especially in data sets with a large number of pictures.

[0069] like Figure 4 As shown, it intuitively reflects that the image compression sensing model proposed in this embodiment has a good reconstruction effect on the original image at different sampling rates, wherein, Figure 4 middle( a )~( f ) represent the original image and the image reconstructed based on the existing networks CSNet+, BCSNet, SCSNet, MSCRLNet, and DCCT-Net respectively. The model proposed in this embodiment significantly improves the processing of the overall image and image details.

[0070] Embodiment 2

[0071] This embodiment provides an image compression sensing reconstruction system based on dynamic convolution and feature fusion, including:

[0072] An image acquisition module, used to acquire the original image to be reconstructed by compressed sensing;

[0073] The image reconstruction module is used to input the original image into the trained image compression sensing model, and perform block sampling and extract the initialized reconstructed feature map through the sampling subnet and the initialized reconstruction subnet in the model. The initialized reconstructed feature map is then input into the deep reconstruction subnet in the model, and the multi-scale dynamic convolution local features and global features are extracted in turn through the parallel dynamic convolution DCNN branch and Transformer branch. The two features of the same scale are fused through the weighted feature fusion module, and finally the fused features of the image are output. After reconstruction, the model outputs a high-quality reconstructed image.

[0074] Embodiment 3

[0075] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps in the image compression sensing reconstruction method based on dynamic convolution and feature fusion as described above are completed.

[0076] Embodiment 4

[0077] This embodiment also provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the image compressed sensing reconstruction method based on dynamic convolution and feature fusion as described above are completed.

[0078] The steps involved in the above embodiments 2 to 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0079] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0080] The above description is only a preferred embodiment of the present invention. Although the specific implementation mode of the present invention is described in conjunction with the accompanying drawings, it is not a limitation of the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the protection scope of the present invention.

Claims

1. An image compression sensing reconstruction method based on dynamic convolution and feature fusion, characterized in that: include: Obtaining an original image to be reconstructed by compressed sensing; The original image is input into the trained image compression sensing model. The sampling subnet and the initialization reconstruction subnet in the model are used to perform block sampling and extract the initialization reconstruction feature map. The initialization reconstruction feature map is then input into the deep reconstruction subnet in the model. The multi-scale dynamic convolution local features and global features are extracted in turn through the parallel dynamic convolution branch and Transformer branch. The two features of the same scale are fused through the weighted feature fusion module, and the fusion features of the image are finally output. After reconstruction, the model outputs a high-quality reconstructed image. The deep reconstruction subnet includes parallel dynamic convolution branches and Transformer branches, and a multi-layer weighted feature fusion module; The dynamic convolution branch includes multiple layers of sequentially connected feature dynamic enhancement modules, the feature dynamic enhancement modules are used to extract dynamic convolution local features of the input image, and each layer of feature dynamic enhancement modules outputs dynamic convolution local features of different scales respectively; The Transformer branch includes multiple layers of Transformer blocks, which are used to extract global features of the input image, and each layer of Transformer blocks outputs global features of different scales; Each layer of feature dynamic enhancement module corresponds to each layer of Transformer block respectively, and the extracted dynamic convolution local features and global features are fused through the weighted feature fusion module of the corresponding layer to obtain fused features, which are then input as input features to the next Transformer block.

2. The image compression sensing reconstruction method based on dynamic convolution and feature fusion as claimed in claim 1, characterized in that: For the first-layer weighted feature fusion module, in addition to inputting the first-level feature dynamic enhancement module and the dynamic convolution local features and global features extracted by the Transformer block, it also includes the initialization of the reconstructed feature map.

3. The image compression sensing reconstruction method based on dynamic convolution and feature fusion as claimed in claim 1, characterized in that: The feature dynamic enhancement module includes a dynamic convolution layer applicable to each image block; The image is input into the feature dynamic enhancement module. According to the different saliency between each image block in the input image, a convolution kernel is dynamically generated for each image block through a convolution operation of a 1×1 convolution kernel. The dynamically generated convolution kernel is used as the dynamic convolution layer. Each image block is convolved through the corresponding dynamic convolution layer to output the dynamic convolution local features.

4. The image compression sensing reconstruction method based on dynamic convolution and feature fusion as claimed in claim 1, characterized in that: The dynamic convolution local features and global features are input into the weighted feature fusion module for feature fusion, including: For dynamic convolution local features and global features, multiple feature fusion methods are used to fuse them to obtain multiple fusion features; Based on the weight of each feature fusion method determined by training, multiple fusion features are weightedly fused to obtain the final fusion feature.

5. The image compression sensing reconstruction method based on dynamic convolution and feature fusion as claimed in claim 1, characterized in that: The training of the image compression sensing model includes: Obtain a number of original image samples and their corresponding reconstructed image samples to construct a training data set; The image compression sensing model constructed by training the training data set is used to extract the fusion features of the input image samples, and the reconstructed image is generated based on the fusion features; Based on the generated reconstructed image and the reconstructed image samples, a peak noise ratio and a structural similarity of the reconstructed image are calculated; After cyclic iterative training, the model parameters are adjusted in each iteration process until the peak noise ratio and structural similarity are maximized, and the iterative training is completed; wherein the model parameters include the weight values ​​of different feature fusion methods in the weighted feature fusion module.

6. An image compression sensing reconstruction system based on dynamic convolution and feature fusion, characterized in that: include: An image acquisition module, used to acquire the original image to be reconstructed by compressed sensing; The image reconstruction module is used to input the original image into the trained image compression sensing model, and perform block sampling and extract the initialization reconstruction feature map through the sampling subnet and the initialization reconstruction subnet in the model. The initialization reconstruction feature map is then input into the deep reconstruction subnet in the model, and the multi-scale dynamic convolution local features and global features are extracted in turn through the parallel dynamic convolution branch and the Transformer branch. The two features of the same scale are fused through the weighted feature fusion module, and finally the fusion features of the image are output. After reconstruction, the model outputs a high-quality reconstructed image; The deep reconstruction subnet includes parallel dynamic convolution branches and Transformer branches, and a multi-layer weighted feature fusion module; The dynamic convolution branch includes multiple layers of sequentially connected feature dynamic enhancement modules, the feature dynamic enhancement modules are used to extract dynamic convolution local features of the input image, and each layer of feature dynamic enhancement modules outputs dynamic convolution local features of different scales respectively; The Transformer branch includes multiple layers of Transformer blocks, which are used to extract global features of the input image, and each layer of Transformer blocks outputs global features of different scales; Each layer of feature dynamic enhancement module corresponds to each layer of Transformer block respectively, and the extracted dynamic convolution local features and global features are fused through the weighted feature fusion module of the corresponding layer to obtain fused features, which are then input as input features to the next Transformer block.

7. An electronic device, characterized in that: The invention comprises a memory and a processor and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of an image compression sensing reconstruction method based on dynamic convolution and feature fusion as described in any one of claims 1 to 5 are completed.

8. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of an image compression sensing reconstruction method based on dynamic convolution and feature fusion as described in any one of claims 1-5.

Images