High-resolution remote sensing image compression method and system based on block modulation imaging

Abstract

The invention discloses a high-resolution remote sensing image compression method and system based on block modulation imaging, a binary mask matrix is generated, an acquisition end configures an optical mask device according to the binary mask matrix, and when a high-resolution remote sensing image is acquired, an optical lens is adopted to acquire an optical signal matrix; after optical mask processing is carried out through the optical mask device, modulation is carried out through the relay lens, then the optical mask device is converted into an electrical signal matrix through the photoelectric sensor, then the electrical signal matrix is partitioned and superposed so as to complete compression, and an obtained measurement value matrix is sent to a receiving end; the receiving end extracts a block mask matrix corresponding to each block from the binary mask matrix according to the block mode of the acquisition end, then extracts each block signal matrix from the measured value matrix to form a three-dimensional signal matrix, and inputs the three-dimensional signal matrix into a pre-constructed and trained decoding network; and decoding to obtain an optical signal matrix of the high-resolution remote sensing image. According to the invention, block modulation imaging is provided, and the decoding restoration quality is improved while the compression speed is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image compression, and more specifically, relates to a high-resolution remote sensing image compression method and system based on block modulation imaging. Background Art

[0002] Remote sensing images play a key role in many fields, and their high-resolution data provide important support for decision-making, resource optimization and disaster risk reduction. In recent years, advances in remote sensing technology have significantly improved the spatial resolution of images and increased the amount of data collected, which has put tremendous pressure on satellite storage capacity, data transmission bandwidth and energy resources. As a result, the processing, storage and communication capabilities on satellites have been severely affected, limiting real-time processing capabilities and satellite-to-ground / satellite-to-satellite data transmission.

[0003] Remote sensing image compression algorithms are key technologies for addressing the challenges of storing and transmitting massive remote sensing images. Satellite payload image compression algorithms are mainly divided into two categories: lossless compression and lossy compression. Lossless compression reduces file size by exploiting redundancy in image data while perfectly reconstructing the original image without losing any information. However, lossless compression usually has a low compression ratio (Cr) and cannot meet the transmission bandwidth and storage limitations of satellites. In contrast, lossy compression can achieve significantly higher compression ratios by allowing a certain degree of controllable information loss. However, this advantage comes at the cost of increased computational requirements, which may put pressure on the limited processing power on satellites. The increased computational burden brought by complex compression algorithms may lead to a decline in satellite processor performance, which in turn affects the execution of other critical tasks. The academic community has conducted extensive research on the challenge of limited computing resources on specific platforms. Many strategies have been proposed to reduce the coding complexity of compression algorithms. However, these methods have not brought significant changes, and the burden on satellite processors during high-resolution imaging is still large.

[0004] Based on the principle of compressed sensing, single pixel imaging (SPI) proposes a new scheme that uses the inherent sparsity of the signal to reconstruct the signal from a number of samples far lower than the number required by the Nyquist-Shannon sampling theorem, providing a better choice for remote sensing image compression. Compared with traditional compression methods that perform electrical domain calculations after the sensor captures the image, SPI uses optical components to perform optical domain calculations before the sensor images, directly capturing the compressed representation of the scene. Compared with traditional compression methods such as JPEG and JPEG2000, SPI has a simpler encoding process and is very suitable for resource-constrained platforms. However, the application of SPI in satellite scenarios faces two major challenges. The first challenge is that a single pixel camera (SPC) can only capture one measurement value at a time. This requires a long acquisition time for time series measurement and requires static imaging, which is extremely unfavorable for moving satellites. The limited imaging speed will also seriously affect the execution of real-time tasks. The second challenge is that SPC relies on digital micromirror devices (DMD) for dynamic encoding. Unfortunately, the limited size of DMD cannot meet the high resolution requirements of remote sensing images. It is worth mentioning that the current SPI decoding algorithm has a very high computational complexity and is not suitable for high-resolution images. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a high-resolution remote sensing image compression method and system based on block modulation imaging, introduce compressed sensing theory into remote sensing image compression and propose block modulation imaging, as well as a corresponding decoding method, so as to improve the compression speed and the decoding restoration quality.

[0006] In order to achieve the above-mentioned object of the invention, the high-resolution remote sensing image compression method based on block modulation imaging of the present invention comprises the following steps:

[0007] S1: Generate a binary mask matrix Where H and W represent the height and width of the optical signal matrix in the process of high-resolution remote sensing image acquisition, respectively, and the binary mask matrix M is distributed to the acquisition end and the receiving end;

[0008] S2: The acquisition end configures the optical mask device according to the binary mask matrix M. When the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes through, and when the element value is 0, the optical signal of the corresponding channel does not pass through;

[0009] S3: The acquisition end uses an optical lens to collect the optical signal matrix of the high-resolution remote sensing image of the target area After optical mask processing by the optical mask device, it is modulated by the relay lens and then converted into an electrical signal matrix by the photoelectric sensor.

[0010] S4: The acquisition end divides the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i , i = 1, 2, ..., N, superimpose the N block electrical signal matrices and normalize them to a preset value range to obtain a measurement value matrix Y and send it to the receiving end;

[0011] S5: After receiving the measurement value matrix Y, the receiving end extracts the block mask matrix m corresponding to each block from the binary mask matrix M according to the block method of the acquisition end. i , and then extract each block signal matrix from the measurement matrix Y

[0012]

[0013] N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix respectively. The pre-built and trained decoding network is input to decode the optical signal matrix of the high-resolution remote sensing image.

[0014] The present invention also proposes a high-resolution remote sensing image compression system based on block modulation imaging, including a collection end compression module and a receiving end decoding module, wherein:

[0015] The compression module at the acquisition end includes an optical lens, an optical mask device, a relay lens, a photoelectric sensor and a compression transmission module, wherein:

[0016] Optical lens is used to collect optical signal matrix of high-resolution remote sensing images of target areas

[0017] The optical mask device generates a binary mask matrix based on the pre-generated Configuration is performed, when the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes, and when the element value is 0, the optical signal of the corresponding channel does not pass; the optical mask device performs optical mask processing on the optical signal matrix X;

[0018] The relay lens modulates the optical signal matrix processed by the optical mask;

[0019] Photoelectric sensors convert the modulated optical signal matrix into an electrical signal matrix

[0020] The compression transmission module is used to divide the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i, i = 1, 2, ..., N, superimpose the N block electrical signal matrices and normalize them to a preset value range to obtain a measurement value matrix Y and send it to the receiving end;

[0021] The receiving end includes a receiving module and a decoding network, where:

[0022] The receiving module is used to extract the block mask matrix m corresponding to each block from the binary mask matrix M according to the block mode of the acquisition end. i , extract each block signal matrix from the received measurement value matrix Y

[0023]

[0024] N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix, respectively;

[0025] The decoding network is used to decode the three-dimensional signal matrix S to obtain the optical signal matrix

[0026] The present invention provides a high-resolution remote sensing image compression method and system based on block modulation imaging, generates a binary mask matrix, and the acquisition end configures the optical mask device according to the binary mask matrix. When acquiring the high-resolution remote sensing image, an optical lens is used to acquire the optical signal matrix, which is modulated by a relay lens after optical mask processing by the optical mask device, and then converted into an electrical signal matrix by a photoelectric sensor. The electrical signal matrix is ​​divided into blocks and then superimposed to complete compression, and the obtained measurement value matrix is ​​sent to the receiving end; the receiving end extracts the block mask matrix corresponding to each block in the binary mask matrix according to the block mode of the acquisition end, and then extracts each block signal matrix from the measurement value matrix to form a three-dimensional signal matrix, which is input into a pre-built and trained decoding network, and decoded to obtain the optical signal matrix of the high-resolution remote sensing image.

[0027] The present invention has the following beneficial effects:

[0028] 1) The present invention realizes Hardman product in the optical domain by setting an optical mask device. When light passes through the optical mask device, the calculation is completed instantly. The whole process is isolated from other systems of the satellite, so no processor resources need to be allocated. After conversion to electrical signals, the block division and summation of the modulated image are the main sources of computational complexity in the encoding process. Compressed sensing theory makes this simple encoding method possible. Compared with traditional JPEG and JPEG2000 compression algorithms, compressed sensing transfers the computational burden from encoding to decoding, saving a lot of computing resources at the acquisition end.

[0029] 2) Compared with the limitation of SPI paradigm, the present invention only needs one exposure to obtain the compressed representation of the image, which not only speeds up the image acquisition, making it very suitable for real-time tasks, but also alleviates the image blurring problem caused by satellite motion;

[0030] 3) The present invention can eliminate the need for DMD and use a preset photomask device to achieve modulation, thereby solving the limitation of commercial DMD in processing high-resolution images. Eliminating DMD deployment can significantly save energy consumption;

[0031] 4) In terms of decoding network, the computational complexity of the present invention is smaller than that of SPI, and it is more suitable for high-resolution images. It can also introduce gated three-dimensional convolution and cross-attention mechanisms to further improve the decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a system schematic diagram of a high-resolution remote sensing image compression method based on block modulation imaging in the present invention;

[0033] Figure 2 It is a flowchart of a specific implementation method of the high-resolution remote sensing image compression method based on block modulation imaging of the present invention;

[0034] Figure 3 is a structural diagram of a decoding network in this embodiment;

[0035] Figure 4 is a structural diagram of a gated three-dimensional convolution module in this embodiment;

[0036] Figure 5 is a structural diagram of the 3D U-net network in this embodiment;

[0037] Figure 6 is a structural diagram of the bidirectional cross attention module in this embodiment;

[0038] Figure 7 It is a structural diagram of the high-resolution remote sensing image compression system based on block modulation imaging of the present invention.

[0039] Figure 8 It is a visual comparison diagram of the images reconstructed on the DOTA-v1.0 dataset under different compression ratios by the present invention and the SAUNet decoding network in this embodiment;

[0040] Fig. 9 is a visual comparison diagram of different tasks of the present invention at different compression ratios in this embodiment;

[0041] Fig.10 is a comprehensive performance curve diagram of the present invention at different compression ratios in this embodiment;

[0042] Fig.11is a physical layout diagram of the digital acquisition system in this embodiment;

[0043] Fig.12 3 is a comparison diagram of the original image and the reconstructed image with a compression ratio of 16 in this embodiment. DETAILED DESCRIPTION

[0044] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0045] In order to better illustrate the technical solution of the present invention, the principle on which the present invention is based is briefly described first.

[0046] Figure 1 Schematic diagram of the system of the high-resolution remote sensing image compression method based on block modulation imaging in the present invention. Figure 1 All signal matrices in are based on grayscale images. Figure 1 As shown, the present invention proposes a new remote sensing image compression paradigm, called block modulation imaging (BMI). The encoding scheme only needs to modulate the image signal using a photomask, and then the modulated image is divided into blocks and summed. The impact of the entire process on computing resources is negligible. The decoding scheme is based on the theory of compressed sensing, and high-quality image restoration is achieved by designing a suitable deep neural network for decoding. The present invention not only inherits the advantage of ultra-low coding complexity of compressed sensing (CS) compared to traditional methods, but also is superior to single pixel imaging (SPI) in terms of practicality. Compared with SPI, the present invention only requires one exposure and does not require static imaging. In addition, the present invention no longer requires a digital micromirror device (DMD), and uses a pre-configured optical mask device, which can adapt to high-resolution images.

[0047] Based on the above process, the present invention proposes a high-resolution remote sensing image compression method based on block modulation imaging. Figure 2 FIG. 1 is a flowchart of a specific implementation of the high-resolution remote sensing image compression method based on block modulation imaging of the present invention. Figure 2 As shown, the specific steps of the high-resolution remote sensing image compression method by block modulation imaging of the present invention include:

[0048] S201: Generate mask matrix:

[0049] Generate a binary mask matrix Where H and W represent the height and width of the optical signal matrix in the high-resolution remote sensing image acquisition process, respectively, and the binary mask matrix M is distributed to the acquisition end and the receiving end. In practical applications, the binary mask matrix M can be set based on experience or generated randomly. For example, the probability of each element in the binary mask matrix M taking 1 can be set as needed, and then the binary mask matrix is ​​generated by randomly taking values, and the probability is usually set to 50%.

[0050] S202: Configure optical mask device:

[0051] The acquisition end configures the optical mask device according to the binary mask matrix M. When the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes through, and when the element value is 0, the optical signal of the corresponding channel does not pass through.

[0052] S203: Acquisition of high-resolution remote sensing images:

[0053] The acquisition end uses an optical lens to collect the optical signal matrix of the high-resolution remote sensing image of the target area. Then, after optical mask processing by the optical mask device, it is modulated by the relay lens and converted into an electrical signal matrix by the photoelectric sensor.

[0054] The modulated optical signal matrix is Then the process of mask optical processing and modulation can be expressed by the following formula:

[0055] G=X⊙M

[0056] Where ⊙ represents the Hardman product.

[0057] The 0-1 mask can be viewed as a sparse representation, in line with the principles of compressed sensing, a mathematical framework that allows the reconstruction of a signal from a smaller number of measurements.

[0058] S204: High-resolution remote sensing image compression and transmission:

[0059] The acquisition end divides the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i , i = 1, 2, ..., N, the N block electrical signal matrices are superimposed and normalized to a preset value range to obtain the measurement value matrix Y and send it to the receiving end. The measurement value matrix Y can be expressed by the following formula:

[0060]

[0061] Among them, Nor() represents the normalization operation.

[0062] S205: Receive and decode high-resolution remote sensing images:

[0063] After receiving the measurement value matrix Y, the receiving end extracts the block mask matrix m corresponding to each block from the binary mask matrix M according to the block mode of the acquisition end. i , and then extract each block signal matrix from the measurement matrix Y

[0064]

[0065] N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix respectively. The pre-built and trained decoding network is input and decoded to obtain the optical signal matrix

[0066] Block Modulated Imaging (BMI) and Video Snapshot Compressed Imaging (SCI) share the same decoding principle. Compared with the decoding of video SCI, the decoding of BMI can be considered as a more challenging task because the correlation between blocks in the image cannot be guaranteed.

[0067] Since the present invention divides the electrical signal matrix into blocks, the optical signal matrix X can also be regarded as N blocks, and each block is denoted as X. i .make Φ=[d1,d2,…,d N ] T , where the superscript T indicates transposition, x i =vec(X i ), d i =diag(vec(M i )), vec() means vectorization. Let y = vec(Y), then we can get the following representation:

[0068] y=Φx

[0069] Reconstruct the remote sensing image signal from y It is to solve an ill-posed optimization problem:

[0070]

[0071] Among them, ||||2 means to obtain the second norm, λ means the prior regularization coefficient, and R() means the prior regularization term.

[0072] The above optimization problem can be solved using the deep unfolding algorithm. The basic idea of ​​the deep unfolding algorithm is to unfold the iterative steps of traditional optimization algorithms (such as ADMM and GAP) into stages in a deep neural network. Each stage contains a linear projection and a small neural network. Start, solve The kth iteration of is:

[0073]

[0074] Among them, v (k) is the introduced characteristic matrix used to reconstruct the estimated signal matrix D (k) Represents the neural network operation at the kth iteration step. η (k) represents the regularization term.

[0075] In the field of image restoration, an intuitive approach may be to design a neural network using two-dimensional convolution. However, experimental results show that when dealing with high compression rates, deep architectures built from many two-dimensional CNNs have difficulty reconstructing high-resolution images, while three-dimensional CNNs exhibit higher efficiency. Three-dimensional convolution can be interpreted as a method of modeling non-local relationships between image blocks. However, due to throughput considerations, only a limited number of three-dimensional convolutions are usually used. In this embodiment, considering the remote sensing application scenario, it can be assumed that the receiving end, that is, the decoding end, has sufficient computing resources, so the complexity of the decoding network is strategically increased to produce better performance.

[0076] Based on the above analysis, this embodiment proposes a decoding network based on three-dimensional convolution, called BMNet. Figure 3 is a structural diagram of the decoding network in this embodiment. Figure 3 As shown, the decoding network in this embodiment includes a K-level linear projection module and a 3DU-net network, a 3D-2D conversion module and a 2D U-net network. Next, each module is described in detail.

[0077] The k-th level linear projection module and 3D U-net network are used for k-th level iterative decoding, where:

[0078] The linear projection module is used to transform the signal matrix Perform linear projection and obtain the feature matrix v (k) Sent to the 3D-Unet network, where The formula for linear projection is as follows:

[0079]

[0080] Where superscript T represents transposition, Φ=[d1,d2,…,d N ] T , d i =diag(vec(M i )), y = vec(Y), vec() means vectorization, η (k) represents the regularization term at the kth level.

[0081] 3D U-net network feature matrix v (k) Perform reconstruction operation to obtain the signal matrix Output, where the signal matrix obtained by the first K-1 level 3D U-net network Output to the next linear projection module, k′=1,2,…,K-1, the signal matrix obtained by the K-th level 3D U-net network Output to the 3D-2D conversion module.

[0082] 3D-2D conversion module is used to convert the signal matrix Converted into N signal matrices, and then spliced ​​into a two-dimensional signal matrix according to the block method of the acquisition end and input into the 2D-Unet network.

[0083] The 2D U-net network is used to reconstruct the input two-dimensional signal matrix to obtain the restored optical signal matrix of the high-resolution remote sensing image. Research has found that using only 3D convolution for image decoding is not enough because it causes blocky artifacts. Therefore, this embodiment adds a 2D U-net network at the end of the decoding network to refine the results, thereby learning the residual between the output of the final stage of the decoding network BMNet and the true value.

[0084] The U-net network is a commonly used neural network that uses an encoder-decoder structure, wherein the encoder is used to perform several downsamplings using a convolution module, and the decoder is used to perform several upsamplings using a convolution module, thereby realizing input reconstruction. In order to further enhance the representation capability of the decoding network, this embodiment introduces a gating mechanism for the three-dimensional convolution module in the 3D U-net network, and proposes a gated three-dimensional convolution module. Figure 4 is a structural diagram of the gated three-dimensional convolution module in this embodiment. Figure 4 As shown, the gated three-dimensional convolution module in this embodiment includes a first three-dimensional convolution module, a LeakyReLU activation module, a second three-dimensional convolution module, a third three-dimensional convolution module, a sigmoid module, a Hadron der Taubert product module and a matrix superposition module, wherein:

[0085] The first three-dimensional convolution module is used to perform a three-dimensional convolution operation on the input feature B, and send the obtained feature to the LeakyreLU activation module.

[0086] The LeakyReLU activation module is used to activate the received features using the LeakyReLU activation function, and send the obtained features to the second three-dimensional convolution module.

[0087] The second three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received features, and send the obtained features to the Hadron der Taubert product module.

[0088] The third three-dimensional convolution module is used to perform a three-dimensional convolution operation on the input feature B, and send the obtained features to the sigmoid module.

[0089] The sigmoid module is used to process the received features using the sigmoid function and send the obtained features to the Hadron der Taubert product module.

[0090] The Haddon product module is used to calculate the Haddon product of the two received features and send the obtained feature F to the matrix superposition module.

[0091] The matrix superposition module is used to superimpose the input feature B and feature F, and output the superimposed matrix.

[0092] According to the above description, since the signal matrix for restoration extracted by the receiving end through the block mask matrix contains invalid elements, applying a convolution kernel with equal weights to all elements may not be optimal. Therefore, this embodiment sets a gated three-dimensional convolution, and by introducing an additional branch to calculate the weight, it is possible to dynamically select important features and improve the accuracy of the decoded remote sensing image.

[0093] In the deep unfolding algorithm, it is very important to promote information exchange between different stages. Optimizing these information exchange mechanisms can mitigate the inherent information loss in the unfolding process. To address this challenge, this embodiment proposes a bidirectional cross attention (TWCA) module to achieve sufficient information exchange between stages by making full use of the latent vectors generated by the encoder in the 3D U-net network. Figure 5 3D U-net network structure diagram in this embodiment. Figure 5 As shown, the 3D U-net network in this embodiment includes an encoder, a bidirectional cross attention module and a decoder, wherein:

[0094] The encoder is used to transform the feature matrix v (k) Encoding is performed to generate latent variables u (k) , and then send it to the bidirectional criss-cross attention module.

[0095] The bidirectional cross attention module is used to use the bidirectional cross attention mechanism to the hidden variable h generated in the previous level. (k -1) and the latent variable u (k) Bidirectional information exchange is carried out between them to generate the optimized latent variable u′ (k) And send it to the decoder to generate the hidden variable h (k)And sent to the bidirectional cross attention module of the next level 3D U-net network, where h (0) =u (0) , that is, the hidden variables in the first-level 3D U-net are the latent variables generated by the encoder. Please make this exchange so that the latent variable u (k) The hidden variable h generated from the previous level (k-1) Integrate information reflecting the accumulated knowledge from all previous stages.

[0096] The decoder is used to optimize the latent variables Decode and generate signal matrix Output.

[0097] Figure 6 is a structural diagram of the bidirectional cross attention module in this embodiment. Figure 6 As shown, the bidirectional cross attention module in this embodiment includes a first three-dimensional convolution module, a second three-dimensional convolution module, a first matrix product module, a first softmax module, a second matrix product module, a second softmax module, a third matrix product module, a fourth matrix product module, a third three-dimensional convolution module, a fourth three-dimensional convolution module, a first Hadrian product module and a second Hadrian product module, wherein:

[0098] The first three-dimensional convolution module is used to generate the hidden variable h (k-1) Perform a three-dimensional convolution operation to generate the query matrix q h , value matrix v h , key matrix k h , which can be expressed as follows:

[0099] q h =W1 k,h h (k-1)

[0100]

[0101] Among them, W1 k,h , They represent the hidden variables h (k-1) The weight matrix corresponding to the corresponding query, value, and key.

[0102] Then the key matrix k h Send it to the first matrix product module, and query the matrix q h Send it to the second matrix product module and convert the value matrix v h Sent to the third matrix product module.

[0103] The second 3D convolution module is used to transform the latent variable u (k) Perform a three-dimensional convolution operation to generate the query matrix q u, value matrix v u , key matrix k u , which can be expressed as follows:

[0104] q u =W1 k,u u (k)

[0105]

[0106] Among them, W1 k,u , They represent the hidden variables h (k-1) The weight matrix corresponding to the corresponding query, value, and key.

[0107] The query matrix q u Send to the first matrix product module, the key matrix k u Send it to the second matrix product module and convert the value matrix v u Sent to the fourth matrix product module.

[0108] The first matrix product module is used to calculate the key matrix k h and the query matrix q u The matrix product of is sent to the first softmax module.

[0109] The first softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f1 to the third matrix product module.

[0110] The third matrix product module is used to calculate the value matrix v h The matrix product of the feature matrix f1 is sent to the third 3D convolution module.

[0111] The third three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g1 to the second Hadrian code product module.

[0112] The second Hadid product module is used to calculate the feature matrix g1 and the latent variable u (k) The Hadamard product of (k) And output.

[0113] The second matrix product module is used to calculate the key matrix k u and the query matrix q h The matrix product of is sent to the second softmax module.

[0114] The second softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f2 to the fourth matrix product module.

[0115] The fourth matrix product module is used to calculate the value matrix v u The matrix product of the feature matrix f2 is sent to the fourth three-dimensional convolution module.

[0116] The fourth three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g2 to the first Hadrian product module.

[0117] The first Hadid product module is used to calculate the feature matrix g2 and the hidden variable h (k-1) The Hadamard product of (k) And output.

[0118] The training samples of the decoding network can be high-resolution remote sensing images obtained by actual measurement, and a conventional training method can be used. The loss function can use L1 loss, L2 loss, SSIM (Structure Similarity IndexMeasure, structural similarity) loss, MS-SSIM loss, etc.

[0119] In order to better implement the practical application of the high-resolution remote sensing image compression method based on block modulation imaging of the present invention, the present invention also proposes a high-resolution remote sensing image compression system based on block modulation imaging. Figure 7 : is a structural diagram of the high-resolution remote sensing image compression system based on block modulation imaging of the present invention. Figure 7 As shown, the high-resolution remote sensing image compression system based on block modulation imaging of the present invention includes a collection end compression module 1 and a receiving end decoding module 2, wherein:

[0120] The collection end compression module 1 includes an optical lens 11, an optical mask device 12, a relay lens 13, a photoelectric sensor 14 and a compression transmission module 15, wherein:

[0121] The optical lens 11 is used to collect the optical signal matrix of the high-resolution remote sensing image of the target area.

[0122] The optical mask device 12 generates a binary mask matrix according to the pre-generated Configuration is performed, when the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes, and when the element value is 0, the optical signal of the corresponding channel does not pass; the optical mask device 12 performs optical mask processing on the optical signal matrix X.

[0123] The relay lens 13 modulates the optical signal matrix after the optical mask processing.

[0124] The photoelectric sensor 14 converts the modulated optical signal matrix into an electrical signal matrix

[0125] The compression and transmission module 15 is used to divide the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i , i = 1, 2, ..., N, the N block electrical signal matrices are superimposed and normalized to a preset value range to obtain a measurement value matrix Y and send it to the receiving end.

[0126] The receiving end decoding module 2 includes a receiving module 21 and a decoding network 22, wherein:

[0127] The receiving module 21 is used to extract the block mask matrix m corresponding to each block from the binary mask matrix M according to the block mode of the acquisition end. i , extract each block signal matrix from the received measurement value matrix Y

[0128]

[0129] N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix respectively.

[0130] The decoding network 22 is used to decode the three-dimensional signal matrix S to obtain the optical signal matrix The specific structure of the decoding network 22 is as follows Figure 3 shown.

[0131] In order to better illustrate the technical solution of the present invention, a specific example is used to experimentally verify the present invention.

[0132] In this embodiment, the DOTA-v1.0 dataset is selected to train the decoding model BMNet in the present invention. The DOTA-v1.0 dataset is a benchmark large-scale dataset designed specifically for object detection in aerial images. It contains 2,806 high-resolution images covering 15 object categories, including aircraft, ships, storage tanks, stadiums, and bridges. In this embodiment, 1,411 training set images were randomly cropped to form non-overlapping image blocks of size 512×512. From the resulting 41,672 image blocks, 39,588 images were randomly selected as training sets, and the remaining 2,084 images were used as test sets. Peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) were selected as evaluation metrics. To be consistent with SPI, compression is only applied to the luminance channel (Y) in the YCbCr space. In addition to evaluating the quality of reconstruction, the annotations of DOTA-v1.0 can also be used to evaluate the mean average accuracy (mAP) of downstream task object detection at different compression rates.

[0133] In order to study the impact of compression on more fine-grained downstream tasks, the ISPRS Vaihingen dataset was selected in this example to test the BMNet trained on DOTA-v1.0 without fine-tuning. The ISPRS Vaihingen dataset is a benchmark for evaluating semantic segmentation algorithms in remote sensing images. It contains 33 high-resolution aerial orthophotos and corresponding digital surface models, covering an urban area in Vaihingen, Germany. We only use TOP images, without DSM and NDSM. We use IDs 34 and 37 as the test set and crop the images into 31 non-overlapping image patches of size 512×512. The images of the remaining IDs are cropped into 3120 non-overlapping image patches of size 512×512 for training the semantic segmentation model DC-SWin. The mean intersection over union (mIoU) was selected as the evaluation metric.

[0134] In order to facilitate comparison with more SPI methods, CBSD68 is selected as an additional test set in this embodiment. It consists of 68 natural color images. The experiment is conducted on the luminance channel of the YCbCr space. PSNR is selected as the evaluation indicator in this embodiment.

[0135] The BMNet model in the present invention is first evaluated on CBSD68, which is a benchmark dataset for the SPI method. For fair comparison, the model of the present invention also adopts a jointly trained measurement matrix here. The performance of the BMNet model and the SOTA SPI decoding network SAUNet was then evaluated on the DOTA-v1.0 dataset. Due to the high computational complexity of mainstream SPI decoding algorithms, especially when applied to high-resolution images, it would be extremely expensive to replicate them on the DOTA-v1.0 dataset. For the experiments on DOTA-v1.0, this embodiment adjusts the image to a resolution of 256×256. The model is first trained with a compression rate of 10, and then the pre-trained model is used as a basis to fine-tune the models with compression rates of 4 and 25.

[0136] Figure 8 It is a visual comparison chart of the images reconstructed by the present invention and the SAUNet decoding network on the DOTA-v1.0 dataset at different compression ratios in this embodiment. Table 1 is a comparison table of the average PSNR values ​​of the images reconstructed by the present invention and the SAUNet decoding network on the DOTA-v1.0 dataset at different compression ratios in this embodiment.

[0137]

[0138] Table 1

[0139] like Figure 8 As shown in Table 1, the performance of BMNet of the present invention is comparable to or better than that of SAUNet (SOTA SPI method), which indicates that BMI has the potential to replace SPI as a solution for on-orbit remote sensing image compression, with both lower encoding complexity and superior decoding performance.

[0140] For lossy compression methods, it is crucial to analyze the impact of the compression ratio on subsequent downstream tasks. Most compressed sensing methods focus primarily on minimizing the reconstruction error. However, the reconstruction quality (measured by metrics such as mean squared error (MSE)) may not always be directly consistent with the performance requirements of a specific application. In this section, a baseline will be established for object detection and semantic segmentation tasks as a benchmark for evaluating the effectiveness of future remote sensing image compression methods based on compressed sensing.

[0141] In order to obtain decoding networks with different compression ratios, this embodiment first established a baseline BMNet trained on DOTA-v1.0 with a compression ratio of 16, and then used this pre-trained model to fine-tune additional models with compression ratios of 4, 9, 25, 36, 49, 64, 81 and 100. For the object detection task, the pre-trained YOLO-v5s model was first evaluated on the uncompressed test set, achieving an mAP of 86.5%. The method of the present invention was then applied to compress and reconstruct the test set at different compression ratios. The reconstructed test set was then input into the YOLO model for detection. For the semantic segmentation task, the DC-Swin model was first trained and achieved an mIoU of 78.6% on the uncompressed test set. Then, the test set compressed and reconstructed by the BMI framework was used again to evaluate the performance of the trained DC-Swin model at different compression ratios.

[0142] Fig. 9 It is a visual comparison diagram of different tasks of the present invention at different compression ratios in this embodiment. Fig.10 4 is a comprehensive performance curve diagram of the present invention at different compression ratios in this embodiment. Fig.10 The performance indicators include PSNR and SSIM (DOTA-v1.0 dataset) for image reconstruction, mAP (DOTA-v1.0 dataset) for target detection, and mIoU (Vaihingen dataset) for semantic segmentation. Figure 7 and Figure 8 As shown, at a compression ratio of 4, the loss of semantic information is negligible and the degradation of downstream task performance is limited to less than 7.5% when the compression ratio is lower than 16.

[0143] In this embodiment, a digital acquisition system based on Metacam is designed to simulate the acquisition end, thereby realizing the simulation process in the physical system. Due to its flexibility, the DMD is retained in the prototype to accommodate experiments with different measurement matrix configurations. Please note that in actual applications, the DMD can be replaced by a photomask to achieve high-resolution imaging. Fig.11 is a physical layout diagram of the digital acquisition system in this embodiment. Fig.11As shown, the digital acquisition system includes an imaging lens assembly, an optical encoding unit, and a relay lens assembly. Through the camera lens (CHIOPT HC3505A), the scene image is placed on a virtual plane for raw data acquisition. The relay lens (Thorlabs MAP10100100-A) focuses the target scene image onto the DMD (ViALUX V-9001, 2560×1600 resolution, 7.6μm pixel pitch) to form a main image. The DMD modulates the data by adjusting the amplitude of the light to achieve instantaneous encoding. The reflected light from the DMD is focused onto the image sensor (FLIR GS3-U3-120S6M-C, 4242×2830 resolution, 3.1μm pixel pitch) through the zoom lens (Utron VTL0714V). The relay lens group is used to transfer the encoded data to the photosensitive surface of the sensor, so that one DMD mirror matches one image sensor pixel, ensuring the precise alignment of the DMD and the image sensor. It is worth noting that the system has a flexible plug-and-play feature. The hardware integration level in the application has the potential to be significantly enhanced. In addition, the system has the ability to adapt to various compression ratios because the compression ratio depends only on the electrical domain calculation.

[0144] To verify the performance of the reconstruction algorithm in a real scene, the test dataset was projected onto a surface in a natural light environment. The digital acquisition system captured the projected data to form measurements, and the encoded measurements were fed into the decoding network BMNet for reconstruction. Due to the oblique hinge design of the DMD's digital micromirrors, their rotational modulation occurs on an axis that is tilted 45° relative to the array orientation. This results in a 45° rotation of the image after processing by the digital acquisition unit. To address this issue, image scaling was used to enlarge the projected image and subsequently cropped to achieve a captured image size of 512×512, which meets the required size for reconstruction. In addition, we observed that the calibrated mask would deviate from the binary encoding used for simulation due to the use of lenses during the alignment process. To address this issue, the model was fine-tuned using masks obtained in a real environment. Specifically, a uniform illumination source provided by an integrating sphere illuminated the entrance pupil. The binary mask was then loaded onto the DMD. The resulting coded light pattern served as the actual experimental mask for real-world testing.

[0145] Fig.12 is a comparison diagram of the original image and the reconstructed image with a compression ratio of 16 in this embodiment. Fig.12 As shown in the figure, although there are certain differences between the reconstructed image and the original image, the features are still well restored and can be adapted to practical engineering applications.

[0146] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A high-resolution remote sensing image compression method based on block modulation imaging, characterized in that: The following steps are involved: S1: Generate a binary mask matrix Where H and W represent the height and width of the optical signal matrix in the process of high-resolution remote sensing image acquisition, respectively, and the binary mask matrix M is distributed to the acquisition end and the receiving end; S2: The acquisition end configures the optical mask device according to the binary mask matrix M. When the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes through, and when the element value is 0, the optical signal of the corresponding channel does not pass through; S3: The acquisition end uses an optical lens to collect the optical signal matrix of the high-resolution remote sensing image of the target area After optical mask processing by the optical mask device, it is modulated by the relay lens and then converted into an electrical signal matrix by the photoelectric sensor. S4: The acquisition end divides the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i , i = 1, 2, ..., N, superimpose the N block electrical signal matrices and normalize them to a preset value range to obtain a measurement value matrix Y and send it to the receiving end; S5: After receiving the measurement value matrix Y, the receiving end extracts the block mask matrix m corresponding to each block from the binary mask matrix M according to the block method of the acquisition end. i , and then extract each block signal matrix from the measurement matrix Y The N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix respectively. The pre-built and trained decoding network is input to decode the optical signal matrix of the high-resolution remote sensing image.

2. The high-resolution remote sensing image compression method according to claim 1, characterized in that: In step S1, the binary mask matrix M is randomly generated, wherein the probability of each element taking 1 is 50%.

3. The high-resolution remote sensing image compression method according to claim 1, characterized in that: The decoding network in step S5 includes a K-level linear projection module and a 3D U-net network, a 3D-2D conversion module and a 2D U-net network, wherein: The k-th level linear projection module and 3D U-net network are used for k-th level iterative decoding, where: The linear projection module is used to transform the signal matrix Perform linear projection and obtain the feature matrix v (k) Sent to the 3D-Unet network, where The formula for linear projection is as follows: Where superscript T represents transposition, Φ=[d1,d2,…,d N ] T , d i =diag(vec(M i )), y = vec(Y), vec() means vectorization, η (k) represents the regularization term of the kth level; 3D U-net network feature matrix v (k) Perform reconstruction operation to obtain the signal matrix Output, where the signal matrix obtained by the first K-1 level 3D U-net network Output to the next linear projection module, k′=1,2,…,K-1, the signal matrix obtained by the K-th level 3DU-net network Output to 3D-2D conversion module; 3D-2D conversion module is used to convert the signal matrix Convert into N signal matrices, then splice the N signal matrices into a two-dimensional signal matrix according to the block method of the acquisition end and input it into the 2D-Unet network; The 2D U-net network is used to reconstruct the input two-dimensional signal matrix to obtain the restored optical signal matrix of the high-resolution remote sensing image.

4. The high-resolution remote sensing image compression method according to claim 1, characterized in that: The three-dimensional convolution module in the 3D U-net network adopts a gated three-dimensional convolution module.

5. The high-resolution remote sensing image compression method according to claim 1, characterized in that: The 3D-Unet includes an encoder, a cross-attention module and a decoder, where: The encoder is used to transform the feature matrix v (k) Encoding is performed to generate latent variables u (k) , and then send it to the bidirectional cross attention module; The bidirectional cross attention module is used to use the bidirectional cross attention mechanism to the hidden variable h generated in the previous level. (k-1) and the latent variable u (k) Bidirectional information exchange is carried out between them to generate the optimized latent variable u′ (k) And send it to the decoder to generate the hidden variable h (k) And sent to the bidirectional cross attention module of the next level 3D U-net network, where h (0) =u (0) ; The decoder is used to optimize the latent variables Decode and generate signal matrix Output.

6. The high-resolution remote sensing image compression method according to claim 5, characterized in that: The bidirectional cross attention module includes a first three-dimensional convolution module, a second three-dimensional convolution module, a first matrix product module, a first softmax module, a second matrix product module, a second softmax module, a third matrix product module, a fourth matrix product module, a third three-dimensional convolution module, a fourth three-dimensional convolution module, a first Hadrian product module and a second Hadrian product module, wherein: The first three-dimensional convolution module is used to generate the hidden variable h (k-1) Perform a three-dimensional convolution operation to generate the query matrix q h , key matrix v h , value matrix k h , then the key matrix k h Send it to the first matrix product module, and query the matrix q h Send it to the second matrix product module and convert the value matrix v h Send to the third matrix product module; The second 3D convolution module is used to transform the latent variable u (k) Perform a three-dimensional convolution operation to generate the query matrix q u , key matrix v u , value matrix k u , then the query matrix q u Send it to the first matrix product module and convert the key matrix k u Send it to the second matrix product module and convert the value matrix v u Send to the fourth matrix product module; The first matrix product module is used to calculate the key matrix k h and the query matrix q u The matrix product of , and the obtained feature matrix is ​​sent to the first softmax module; The first softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f1 to the third matrix product module; The third matrix product module is used to calculate the value matrix v h The matrix product of the feature matrix f1 is sent to the third three-dimensional convolution module; The third three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g1 to the second Hadrian code product module; The second Hadid product module is used to calculate the feature matrix g1 and the latent variable u (k) The Hadamard product of (k) And output; The second matrix product module is used to calculate the key matrix k u and the query matrix q h The matrix product of , and the obtained feature matrix is ​​sent to the second softmax module; The second softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f2 to the fourth matrix product module; The fourth matrix product module is used to calculate the value matrix v u The matrix product of the feature matrix f2 is sent to the fourth three-dimensional convolution module; The fourth three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g2 to the first Hadrian product module; The first Hadid product module is used to calculate the feature matrix g2 and the hidden variable h (k-1) The Hadamard product of (k) And output.

7. A high-resolution remote sensing image compression system based on block modulation imaging, characterized in that: It includes a compression module at the acquisition end and a decoding module at the receiving end, wherein: The compression module at the acquisition end includes an optical lens, an optical mask device, a relay lens, a photoelectric sensor and a compression transmission module, wherein: Optical lens is used to collect optical signal matrix of high-resolution remote sensing images of target areas The optical mask device generates a binary mask matrix based on the pre-generated Configuration is performed, when the element value in the binary mask matrix M is 1, the optical signal of the corresponding channel passes, and when the element value is 0, the optical signal of the corresponding channel does not pass; the optical mask device performs optical mask processing on the optical signal matrix X; The relay lens modulates the optical signal matrix processed by the optical mask; Photoelectric sensors convert the modulated optical signal matrix into an electrical signal matrix The compression transmission module is used to divide the electrical signal matrix Z of the high-resolution remote sensing image into blocks to obtain N block electrical signal matrices z of the same size. i , i = 1, 2, ..., N, superimpose the N block electrical signal matrices and normalize them to a preset value range to obtain a measurement value matrix Y and send it to the receiving end; The receiving end includes a receiving module and a decoding network, where: The receiving module is used to extract the block mask matrix m corresponding to each block from the binary mask matrix M according to the block mode of the acquisition end. i , extract each block signal matrix from the received measurement value matrix Y The N block signal matrices Constructing a three-dimensional signal matrix h and w represent the height and width of the block signal matrix, respectively; The decoding network is used to decode the three-dimensional signal matrix S to obtain the optical signal matrix 8. The high-resolution remote sensing image compression system according to claim 7, characterized in that: The decoding network includes a K-level linear projection module and a 3D U-net network, a 3D-2D conversion module and a 2D U-net network, wherein: The k-th level linear projection module and 3D U-net network are used for k-th level iterative decoding, where: The linear projection module is used to transform the signal matrix Perform linear projection and obtain the feature matrix v (k) Sent to the 3D-Unet network, where The formula for linear projection is as follows: Where superscript T represents transposition, Φ=[d1,d2,…,d N ] T , d i =diag(vec(M i )), y = vec(Y), vec() means vectorization, η (k) represents the regularization term of the kth level; 3D U-net network feature matrix v (k) Perform reconstruction operation to obtain the signal matrix Output, where the signal matrix obtained by the first K-1 level 3D U-net network Output to the next linear projection module, k′=1,2,…,K-1, the signal matrix obtained by the K-th level 3DU-net network Output to 3D-2D conversion module; 3D-2D conversion module is used to convert the signal matrix Convert into N signal matrices, then splice the N signal matrices into a two-dimensional signal matrix according to the block method of the acquisition end and input it into the 2D-Unet network; The 2D U-net network is used to reconstruct the input two-dimensional signal matrix to obtain the restored optical signal matrix of the high-resolution remote sensing image.

9. The high-resolution remote sensing image compression system according to claim 7, characterized in that: The 3D-Unet includes an encoder, a cross-attention module and a decoder, where: The encoder is used to transform the feature matrix v (k) Encoding is performed to generate latent variables u (k) , and then send it to the bidirectional cross attention module; The bidirectional cross attention module is used to use the bidirectional cross attention mechanism to the hidden variable h generated in the previous level. (k-1) and the latent variable u (k) Bidirectional information exchange is carried out between them to generate the optimized latent variable u′ (k) And send it to the decoder to generate the hidden variable h (k) And sent to the bidirectional cross attention module of the next level 3D U-net network, where h (0) =u (0) ; The decoder is used to optimize the latent variables Decode and generate signal matrix Output.

10. The high-resolution remote sensing image compression system according to claim 9, characterized in that: The bidirectional cross attention module includes a first three-dimensional convolution module, a second three-dimensional convolution module, a first matrix product module, a first softmax module, a second matrix product module, a second softmax module, a third matrix product module, a fourth matrix product module, a third three-dimensional convolution module, a fourth three-dimensional convolution module, a first Hadrian product module and a second Hadrian product module, wherein: The first three-dimensional convolution module is used to generate the hidden variable h (k-1) Perform a three-dimensional convolution operation to generate the query matrix q h , key matrix v h , value matrix k h , then the key matrix k h Send it to the first matrix product module, and query the matrix q h Send it to the second matrix product module and convert the value matrix v h Send to the third matrix product module; The second 3D convolution module is used to transform the latent variable u (k) Perform a three-dimensional convolution operation to generate the query matrix q u , key matrix v u , value matrix k u , then the query matrix q u Send it to the first matrix product module and convert the key matrix k u Send it to the second matrix product module and convert the value matrix v u Send to the fourth matrix product module; The first matrix product module is used to calculate the key matrix k h and the query matrix q u The matrix product of , and the obtained feature matrix is ​​sent to the first softmax module; The first softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f1 to the third matrix product module; The third matrix product module is used to calculate the value matrix v h The matrix product of the feature matrix f1 is sent to the third three-dimensional convolution module; The third three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g1 to the second Hadrian code product module; The second Hadid product module is used to calculate the feature matrix g1 and the latent variable u (k) The Hadamard product of (k) And output; The second matrix product module is used to calculate the key matrix k u and the query matrix q h The matrix product of , and the obtained feature matrix is ​​sent to the second softmax module; The second softmax module is used to process the received feature matrix using the softmax function, and send the obtained feature matrix f2 to the fourth matrix product module; The fourth matrix product module is used to calculate the value matrix v u The matrix product of the feature matrix f2 is sent to the fourth three-dimensional convolution module; The fourth three-dimensional convolution module is used to perform a three-dimensional convolution operation on the received feature matrix, and send the obtained feature matrix g2 to the first Hadrian product module; The first Hadid product module is used to calculate the feature matrix g2 and the hidden variable h (k-1) The Hadamard product of (k) And output.

Images