An image compression sensing self-adaptive reconstruction method and device based on texture features

Abstract

The application discloses a texture feature-based image compression sensing adaptive reconstruction method and device, and the method comprises the following steps: obtaining an original image to be reconstructed; performing compression sensing sampling on an image block in the original image to obtain a measurement value of the image block; inputting the measurement value of the image block into a pre-trained reconstruction network model to obtain a reconstructed image block; and splicing the reconstructed image blocks in sequence to obtain a final reconstructed image; wherein the reconstruction network model comprises an initial reconstruction subnet, a gate subnet and a backbone reconstruction subnet; the gate subnet is used to mine texture features of initial reconstruction, output an activation vector to activate a corresponding part of the backbone reconstruction subnet, realize adaptive reconstruction of different image blocks, and improve the reconstruction quality and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to an image compression sensing adaptive reconstruction method and apparatus based on texture features. Background Technology

[0002] In recent years, compressed sensing theory, based on signal sparsity (sparse model), has proposed a novel signal sampling and reconstruction method: at the encoding end, a small number of measurement values ​​(or sampled values) are obtained by linearly and non-adaptively projecting the original image data through a measurement matrix; at the encoding end, the image is reconstructed from the small number of obtained measurement values ​​by iteratively solving an optimization problem. This strategy of direct compression without storage significantly reduces the memory resource requirements of the encoding end and can sample signals at a sampling rate lower than Nyquist, greatly improving coding efficiency. It has been applied in multiple fields, such as single-pixel imaging, accelerated magnetic resonance imaging, wireless remote monitoring, and cognitive radio communication.

[0003] Traditional compressed sensing reconstruction algorithms reconstruct images by iteratively solving optimization problems, resulting in low reconstruction efficiency and failing to meet the demands of real-time applications. In recent years, deep learning-based compressed sensing reconstruction algorithms have attracted widespread attention. These methods construct parameterized networks and learn the mapping from measurements to the original image through network parameter training, achieving higher reconstruction efficiency and meeting the needs of real-time applications. However, existing deep learning-based compressed sensing reconstruction methods utilize the same network to reconstruct different image patches, ignoring the image's texture features, thus requiring further improvement in reconstruction quality and efficiency. Summary of the Invention

[0004] Objective: In order to overcome the shortcomings of the existing technology, the present invention provides an image compression sensing adaptive reconstruction method and apparatus based on texture features.

[0005] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] Firstly, a Parkinson's disease speech recognition method is provided, including:

[0007] Obtain the original image to be reconstructed;

[0008] Compressed sensing sampling is performed on image patches in the original image to obtain the measurement values ​​of the image patches;

[0009] The measured values ​​of the image patch are input into the pre-trained reconstruction network model to obtain the reconstructed image patch;

[0010] The reconstructed image blocks are stitched together in sequence to obtain the final reconstructed image;

[0011] The reconstructed network model includes an initial reconstructed subnet, gate subnets, and a backbone reconstructed subnet;

[0012] The initial reconstruction subnetwork consists of one convolutional layer and one deformable layer; the measured value y is input into the initial reconstruction subnetwork, and the initial reconstruction result is output. The gate subnetwork consists of multiple convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer; the initial reconstruction result is input into the gate subnetwork, and the activation vector is output. The backbone reconstruction subnetwork consists of multiple weighted residual subnetworks; the initial reconstruction result and the activation vector are input into the backbone reconstruction subnetwork for adaptive reconstruction, and the final reconstruction result is output.

[0013] In some embodiments, performing compressed sensing sampling on image patches in the original image to obtain measurement values ​​for the image patches includes:

[0014] In compressed sensing, the signal acquisition is shown in the following formula:

[0015] y = Φx

[0016] Where y is the measured value, Φ is the sampling matrix, and x is the original image;

[0017] In this network, compressed sensing sampling is implemented by a convolutional layer for sampling image patches; SR is the sampling rate, and convolution is performed using b·b·SR convolutional kernels of size b×b with a stride of b, 1 input channel, and b·b·SR output channels.

[0018] The convolutional layer performs compressed sensing sampling on each b×b image block in the image and outputs an h×w×(SR·b·b) tensor, where h and w represent the number of blocks in the height and width dimensions of the input image.

[0019] In some embodiments, the method for constructing the reconstructed network model includes:

[0020] The reconstructed network model consists of an initial reconstructed subnet, a gate subnet, and a backbone reconstructed subnet.

[0021] The initial reconstruction subnet consists of one convolutional layer (Conv layer) and one reshape layer (Reshape layer). First, the measured value y is convolved using a first convolutional layer, wherein the number of convolutional kernels in the first convolutional layer is b·b, the kernel size is 1×1, the stride is 1, and the padding is valid. Then, the output tensor of the convolutional layer is deformed using a reshape layer, and the initial reconstruction result is output to the backbone reconstruction subnet.

[0022] The subnet consists of h fourth convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer. The fourth convolutional layers have 32 kernels, a kernel size of 3×3, a stride of 1, and the same padding. The average pooling layer has a 2×2 kernel size. The fully connected layer has k neurons. In the filtering layer, the output vector of the softmax layer is filtered: if the value is greater than or equal to a set threshold... If the value is positive, retain it; otherwise, set it to 0. Finally, obtain the final activation vector and output it to the backbone reconstruction subnet.

[0023] The backbone reconstruction subnetwork consists of k weighted residual subnetworks. In each weighted residual subnetwork, firstly, a second convolutional layer (Conv layer) is used to extract features from the input to obtain a feature map. The second convolutional layer has 32 kernels, a kernel size of 3×3, a stride of 1, and the same padding. Then, a third convolutional layer (Conv layer) is used for enhancement reconstruction to obtain the enhancement reconstruction result. The third convolutional layer has 1 kernel, a kernel size of 3×3, a stride of 1, and the same padding. Next, the value of the enhancement reconstruction result is multiplied by the corresponding value in the activation vector, and then added to the input of this weighted residual subnetwork to obtain the final output of this weighted residual subnetwork.

[0024] In some embodiments, the training method for the reconstructed network model includes:

[0025] (2a) Based on the dataset, the initial reconstruction subnet is pre-trained using the reconstruction loss function as the loss function and the gradient descent algorithm is employed; wherein the dataset consists of data pairs of image patches and their measurements.

[0026] (2b) Based on the dataset, an adaptive reconstruction loss function is constructed using the reconstruction loss function and the gate function, and the gradient descent algorithm is used to train the entire network end-to-end; wherein the dataset consists of data pairs of image patches and their measurements.

[0027] In some embodiments, the reconstruction loss function L recon for:

[0028]

[0029] Among them, f pre (y) represents the output of the initial reconstruction subnet, where x is the original image and y is the measurement value. In some embodiments, the adaptive reconstruction loss function L... adap for:

[0030] L adap =L recon +λ1Lgate

[0031] Where λ1 is the weight parameter, L recon To reconstruct the loss function, L gate For gate functions:

[0032]

[0033] Where, p i (y) represents the value in the output vector of the Softmax layer, where i represents the output vector p. i The i-th value in (y) is k, where k is the number of weighted residual subnets.

[0034] In some embodiments, the dataset is the BSD200 dataset.

[0035] In some embodiments, the image compression-sensing adaptive reconstruction method based on texture features has the following parameter settings: the number of convolutional layers h in the subnet is set to 5, the number of weighted residual subnets k is set to 9, the stride b is set to 32, and the weight parameter λ1 is set to 1.

[0036] In a second aspect, the present invention provides an image compression-sensing adaptive reconstruction device based on texture features, including a processor and a storage medium;

[0037] The storage medium is used to store instructions;

[0038] The processor is configured to operate according to the instructions to perform the steps of the method according to the first aspect.

[0039] Thirdly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0040] Beneficial Effects: The image compressed sensing adaptive reconstruction method and apparatus based on texture features provided by this invention have the following advantages: This method proposes a special end-to-end neural network, which consists of an initial reconstruction subnet, a gate subnet, and a backbone reconstruction subnet. The gate subnet is used to mine the texture features of the initial reconstruction and output activation vectors to activate the corresponding parts of the backbone reconstruction subnet, thereby achieving adaptive reconstruction of different image patches and improving reconstruction quality and efficiency. Attached Figure Description

[0041] Figure 1 This is an overall network framework diagram according to an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be used to limit the scope of protection of the present invention.

[0043] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0044] In the description of this invention, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0045] Example 1

[0046] An image compressed sensing adaptive reconstruction method based on texture features includes:

[0047] Obtain the original image to be reconstructed;

[0048] Compressed sensing sampling is performed on image patches in the original image to obtain the measurement values ​​of the image patches;

[0049] The measured values ​​of the image patch are input into the pre-trained reconstruction network model to obtain the reconstructed image patch;

[0050] The reconstructed image blocks are stitched together in sequence to obtain the final reconstructed image;

[0051] The reconstructed network model includes an initial reconstructed subnet, gate subnets, and a backbone reconstructed subnet;

[0052] The initial reconstruction subnetwork consists of one convolutional layer and one deformable layer; the measured value y is input into the initial reconstruction subnetwork, and the initial reconstruction result is output. The gate subnetwork consists of multiple convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer; the initial reconstruction result is input into the gate subnetwork, and the activation vector is output. The backbone reconstruction subnetwork consists of multiple weighted residual subnetworks; the initial reconstruction result and the activation vector are input into the backbone reconstruction subnetwork for adaptive reconstruction, and the final reconstruction result is output.

[0053] This embodiment provides an image compression-sensory adaptive reconstruction method based on texture features. The specific steps of this method are as follows:

[0054] (1) Building a network

[0055] An end-to-end deep network is constructed, consisting of an initial reconstruction subnet, gate subnets, and a backbone reconstruction subnet. The initial reconstruction subnet comprises one convolutional layer and one deformable layer. The measured value y is input to the initial reconstruction subnet, and the output is the initial reconstruction result. The gate subnet consists of multiple convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer. The initial reconstruction result is input to the gate subnet, and the output is the activation vector. The backbone reconstruction subnet consists of multiple weighted residual subnets. The initial reconstruction result and the activation vector are input to the backbone reconstruction subnet for adaptive reconstruction, and the output is the final reconstruction result.

[0056] (2) Training the network

[0057] (2a) Based on the dataset (consisting of data pairs of image patches and their measurements), the initial reconstruction subnet is pre-trained using the reconstruction loss function as the loss function and the gradient descent algorithm.

[0058] (2b) Based on the dataset (consisting of data pairs of image patches and their measurements), an adaptive reconstruction loss function is constructed using the reconstruction loss function and the gate function. The gradient descent algorithm is used to train the entire network end-to-end.

[0059] (3) Restructuring

[0060] The measurements of the image patches are input into the trained network to obtain reconstructed image patches. The reconstructed image patches are then stitched together in sequence to obtain the final reconstructed image.

[0061] In this embodiment, the process specifically includes obtaining the original measurement value, network construction, and training, with the following steps:

[0062] (1) Obtain the original measurement value

[0063] In compressed sensing, the signal acquisition is shown in the following formula:

[0064] y=Φx (1)

[0065] Where y is the measured value, Φ is the sampling matrix, and x is the original image. In this network, equation (1) is implemented by a convolutional layer for sampling image patches. Specifically, SR is the sampling rate, and convolution is performed using b·b·SR convolutional kernels of size b×b with a stride of b, 1 input channel, and b·b·SR output channels.

[0066] This convolutional layer is equivalent to compressed sensing sampling of each b×b image block in the image, and the output is a tensor of h×w×(SR·b·b), where h and w represent the number of blocks in the height and width dimensions of the input image (i.e., the width and height of the image are divided by b respectively).

[0067] (2) Network construction process

[0068] Building end-to-end deep networks such as Figure 1 As shown, the network consists of an initial reconfiguration subnet, gate subnets, and a backbone reconfiguration subnet:

[0069] The initial reconstruction subnetwork consists of one convolutional layer (Conv layer) and one reshape layer (Reshape layer). First, a convolutional layer (number of kernels: b·b, kernel size: 1×1, stride: 1, padding: valid) is used to perform a convolution operation on the measured value y. Then, a reshape layer is used to deform the output tensor of the convolutional layer, outputting the initial reconstruction result.

[0070] The backbone reconstruction subnetwork consists of k weighted residual subnetworks. In each weighted residual subnetwork, firstly, a single convolutional layer (Conv layer) (32 kernels, 3×3 kernel size, stride 1, padding same) is used to extract features from the input, obtaining a feature map. Then, another convolutional layer (Conv layer) (1 kernel, 3×3 kernel size, stride 1, padding same) is used for augmentation reconstruction, yielding the augmented reconstruction result. Next, the value of the augmented reconstruction result is multiplied by the corresponding value in the activation vector, and then added to the input of the weighted residual subnetwork to obtain the final output of the weighted residual subnetwork.

[0071] The network consists of h convolutional layers (Conv layers) (32 kernels, 3×3 kernel size, stride 1, padding same), one average pooling layer (pooling kernel size 2×2), one fully connected layer (FC layer) (k neurons), one softmax layer, and one filtering layer (GL layer). In the filtering layer, the output vector of the softmax layer is filtered: if the value is greater than or equal to a set threshold... If the value is positive, retain it; otherwise, set it to 0. Finally, the final activation vector is obtained.

[0072] (3) Training process

[0073] (3a) Construct the loss function L as shown below. recon Pre-train the initial reconstructed subnet:

[0074]

[0075] Among them, f pre (y) represents the output of the initial reconstructed subnet.

[0076] (3b) Construct the adaptive reconstruction loss function L as shown below. adap Train the entire network:

[0077] L adap =L recon +λ1L gate

[0078] Where λ1 is the weight parameter, L gate For gate functions:

[0079]

[0080] Where, p i (y) represents the numerical value in the output vector of the Softmax layer.

[0081] The effects of the method in the embodiments of the present invention will be further explained below.

[0082] Parameter settings: h is set to 5, k to 9, b to 32, and λ1 to 1. The BSD200 dataset is divided into training and validation sets (grayscale images) at 80% and 20% respectively. BSD100 is used as the test set (grayscale images). The Adam algorithm is used to train the network. Performance is compared with the mainstream reconstruction method ADMM-Net, as shown in Table 1.

[0083] Table 1. PSNR (dB) Performance Comparison

[0084]

[0085] As can be seen from Table 1, the method of the present invention has a significant improvement in reconstruction quality compared with the existing ADMM-Net method.

[0086] Example 2

[0087] Secondly, this embodiment provides an image compression-sensing adaptive reconstruction device based on texture features, including a processor and a storage medium;

[0088] The storage medium is used to store instructions;

[0089] The processor is configured to operate according to the instructions to perform the steps of the method according to Embodiment 1.

[0090] Example 3

[0091] Thirdly, this embodiment provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image compressed sensing adaptive reconstruction method based on texture features, characterized in that, include: Obtain the original image to be reconstructed; Compressed sensing sampling is performed on image patches in the original image to obtain the measurement values ​​of the image patches; The measured values ​​of the image patch are input into the pre-trained reconstruction network model to obtain the reconstructed image patch; The reconstructed image blocks are stitched together in sequence to obtain the final reconstructed image; The reconstructed network model includes an initial reconstructed subnet, gate subnets, and a backbone reconstructed subnet; The initial reconstruction subnetwork consists of one convolutional layer and one deformable layer; the measured value y is input into the initial reconstruction subnetwork, and the initial reconstruction result is output. The subnetting network consists of multiple convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer. The filtering layer filters the output vector of the softmax layer: if the value is greater than or equal to a set threshold... If the value is positive, retain it; otherwise, set it to 0. k is the number of neurons in the fully connected layer. The initial reconstruction result is input into the subnet and the output is the activation vector. The backbone reconstruction subnet consists of multiple weighted residual subnets; the initial reconstruction result and activation vector are input into the backbone reconstruction subnet for adaptive reconstruction, and the final reconstruction result is output.

2. The image compression-sensing adaptive reconstruction method based on texture features according to claim 1, characterized in that, The process of performing compressed sensing sampling on image patches in the original image to obtain measurement values ​​for the image patches includes: In compressed sensing, the signal acquisition is shown in the following formula: y = Φx Where y is the measured value, Φ is the sampling matrix, and x is the original image; Compressed sensing sampling is implemented by a convolutional layer for sampling image patches; SR is the sampling rate, which uses b·b·SR convolutional kernels of size b×b to perform convolution calculations with a stride of b, 1 input channel, and b·b·SR output channels. The convolutional layer performs compressed sensing sampling on each b×b image block in the image and outputs an h×w×(SR·b·b) tensor, where h and w represent the number of blocks in the height and width dimensions of the input image.

3. The image compression-sensing adaptive reconstruction method based on texture features according to claim 2, characterized in that, The method for constructing the reconstructed network model includes: The reconstructed network model consists of an initial reconstructed subnet, a gate subnet, and a backbone reconstructed subnet. The initial reconstruction subnet consists of one convolutional layer and one deformable layer. First, the measured value y is convolved using a first convolutional layer, wherein the number of convolutional kernels in the first convolutional layer is b·b, the kernel size is 1×1, the stride is 1, and the padding is valid. Then, the output tensor of the convolutional layer is deformed using a deformable layer, and the initial reconstruction result is output to the backbone reconstruction subnet. The subnetwork consists of h fourth convolutional layers, one average pooling layer, one fully connected layer, one softmax layer, and one filtering layer. The fourth convolutional layers have 32 kernels, a kernel size of 3×3, a stride of 1, and the same padding. The average pooling layer has a 2×2 kernel size. The fully connected layer has k neurons. In the filtering layer, the output vector of the softmax layer is filtered, and finally, the final activation vector is obtained and output to the backbone reconstruction subnetwork. The backbone reconstruction subnetwork consists of k weighted residual subnetworks. In each weighted residual subnetwork, firstly, a second convolutional layer is used to extract features from the input to obtain a feature map. The second convolutional layer has 32 kernels, a kernel size of 3×3, a stride of 1, and the same padding. Then, a third convolutional layer is used for enhancement reconstruction to obtain the enhancement reconstruction result. The third convolutional layer has 1 kernel, a kernel size of 3×3, a stride of 1, and the same padding. Next, the value of the enhancement reconstruction result is multiplied by the corresponding value in the activation vector, and then added to the input of this weighted residual subnetwork to obtain the final output of this weighted residual subnetwork.

4. The image compression-sensing adaptive reconstruction method based on texture features according to claim 1, characterized in that, The training method for the reconstructed network model includes: (2a) Based on the dataset, the initial reconstruction subnet is pre-trained using the reconstruction loss function as the loss function and the gradient descent algorithm is employed; wherein the dataset consists of data pairs of image patches and their measurements. (2b) Based on the dataset, an adaptive reconstruction loss function is constructed using the reconstruction loss function and the gate function, and the gradient descent algorithm is used to train the entire network end-to-end; wherein the dataset consists of data pairs of image patches and their measurements.

5. The image compression-sensing adaptive reconstruction method based on texture features according to claim 4, characterized in that, Reconstruction loss function L recon for: Among them, f pre (y) represents the output of the initial reconstructed subnet, where x is the original image and y is the measurement value.

6. The image compression-sensing adaptive reconstruction method based on texture features according to claim 5, characterized in that, Adaptive reconstruction loss function L adap for: L adap =L recon +λ1L gate Where λ1 is the weight parameter, L recon To reconstruct the loss function, L gate For gate functions: Where, p i (y) represents the value in the output vector of the Softmax layer, where i represents the output vector p. i The i-th value in (y) is k, where k is the number of weighted residual subnets.

7. The image compression-sensing adaptive reconstruction method based on texture features according to claim 4, characterized in that, The dataset used is the BSD200 dataset.

8. The image compression-sensing adaptive reconstruction method based on texture features according to claim 1, characterized in that, Parameter settings: The number of convolutional layers h in the subnet is set to 5, the number of weighted residual subnets k is set to 9, the stride b is set to 32, and the weight parameter λ1 is set to 1.

9. An image compression-sensing adaptive reconstruction device based on texture features, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images