Pre-sensing compression system for space-based remote sensing large-field-of-view image

Abstract

The invention relates to a pre-sensing compression system for a space-based remote sensing large-view-field image, and belongs to the technical field of image processing. Comprising n light paths, a sensor, a storage transmission module and a decoding system, a large-view-field picture reflected by a ground object is equally divided into n information blocks, and each information block is transmitted to a corresponding light path; each information block enters a sensor after being sequentially subjected to first light adjustment, coding and second light adjustment in a corresponding light path; the sensor receives the n encoded information blocks, converts the n encoded information blocks into n electric signals, and stores the n electric signals to the storage transmission module; the storage transmission module sends the n electric signals to a decoding system; the decoding system decodes and recovers the n electric signals to obtain information of each block of picture, and a large-field-of-view picture is obtained after splicing; according to the invention, the problem of large-view-field high-resolution imaging of a space-based remote sensing imaging system is solved, the timeliness of information downloading and utilization is realized, and the application value of data is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and relates to a pre-sensing compression system for space-based remote sensing large-field-of-view images. Background Art

[0002] Satellite optical remote sensing images contain rich physical information and play a vital role in various fields. Satellite optical remote sensing images can provide accurate spatial information, help decision makers formulate effective response strategies, improve resource utilization efficiency and reduce disaster risks. In recent years, with the continuous development of satellite optical remote sensing technology, optical remote sensing satellites can continuously observe the surface with high resolution and high frequency. The spatial resolution of remote sensing images has been continuously improved from meter level to sub-meter level, which has led to an explosive growth in the pixel size of optical remote sensing image data. However, due to the limitations of satellite payload space, data transmission bandwidth and energy consumption, the collected high-resolution image data will inevitably increase the storage pressure of the onboard data recorder and the transmission pressure when downlinking data. The limited transmission bandwidth is difficult to meet the transmission requirements of huge data volumes. The utilization efficiency of massive remote sensing images on the satellite is greatly limited, and the application value of the data is greatly reduced, especially in application scenarios that require real-time data support. This brings huge challenges to tasks such as onboard data storage, onboard intelligent real-time processing of data, and satellite-to-ground / inter-satellite data transmission.

[0003] The amount of data from optical remote sensing satellites is very large, and the specific value varies depending on the satellite's resolution, imaging frequency, coverage area and other factors. These satellites can continuously observe the surface at high resolution and high frequency, and the amount of data generated is huge, reaching TB or even PB levels. For example, the data acquisition volume of the US commercial earth observation satellite WorldView-3 is about 1.2TB per day. China's Gaofen-2 satellite generates more than 1000GB of data per day.

[0004] High-resolution optical remote sensing image data contains a large amount of details and information, and the file size of a single image may be as high as several gigabytes (GB). However, the current remote sensing satellite data transmission rate is generally only between a few hundred Mbps and a few thousand Mbps. Taking the United States' GOES-R series of meteorological satellites as an example, its maximum data transmission rate is only 400Mbps, and the data transmission rate of Europe's Sentinel-2 satellite is 1.2Gbps. Although the data transmission rate of China's high-resolution satellite can reach 8Gbps, its demand for high-speed and large-capacity image data transmission is still difficult to meet.

[0005] In addition, for asynchronous satellites, data can only be transmitted downward when the satellite enters the visual range of the ground receiving station. Therefore, it is necessary to store data when the satellite is out of sight of the earth (out of sight) and transmit data when it returns to the visual range of the ground receiving station. The collected high-resolution image data will inevitably increase the storage pressure of the onboard data recorder and the transmission pressure when transmitting data downward.

[0006] The above situation will lead to a series of data transmission problems, such as slow data transmission speed, increased transmission delay, and even the inability to complete the transmission in the case of network congestion. This affects the real-time and timeliness of data analysis and processing, especially in emergency situations, which may cause information lag. Whether in scientific research or commercial applications, the rapid acquisition and transmission of remote sensing image data is crucial for decision-making and resource management. The above data transmission problems will greatly reduce the application value of data, especially in areas that require real-time data support. For example, when natural disasters occur, such as earthquakes, storms, floods, etc., satellite images can provide real-time updates of disaster assessment and disaster area conditions. If the images cannot be transmitted back to the ground command center in time, rescuers and decision makers will not be able to quickly obtain disaster information, affecting the accuracy and timeliness of emergency rescue and resource allocation.

[0007] Space-based image compression algorithms can be divided into two main categories: lossless and lossy. Lossless compression can maintain a perfect reconstruction of the original image without losing any information. However, lossless compression usually achieves a low compression ratio, which makes it insufficient to handle the transmission bandwidth and storage limitations of satellites. In contrast, lossy compression provides significantly higher compression ratios by losing information within a controllable range, but this advantage comes at the cost of increased computational requirements. Traditional lossy image compression requires the image to be fully acquired before compression, which affects the overall efficiency of the system. Compared with the requirements of the Nyquist-Shannon sampling theorem, the theory based on compressed sensing (CS) provides new ideas for large-scale image compression on satellites. CS uses the inherent sparsity of the signal and can be reconstructed from fewer samples. Block compressed sensing (BCS) has become a powerful framework for simultaneous sampling and compression, eliminating the need to obtain a complete image. BCS requires the use of a sensing matrix to obtain a compressed measurement of the signal, including multiplying the signal (or its block) by the sensing matrix to obtain a compressed measurement. Although the introduction of BCS reduces the signal processing cost, its encoding process still requires additional operations, such as matrix multiplication or convolution, which consumes onboard payload computing resources. Summary of the invention

[0008] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and propose a pre-sensing compression system for space-based remote sensing large-field-of-view images, so as to solve the problem of large-field-of-view high-resolution imaging faced by space-based remote sensing imaging systems without increasing additional computing and storage consumption, realize the timeliness of information transmission and utilization, improve the application value of data, and provide a solution for effectively reducing the size of the detector and reducing the amount of transmitted data.

[0009] The solution of the present invention is:

[0010] A pre-sensing compression system for space-based remote sensing large-field-of-view images, comprising n optical paths, sensors, storage and transmission modules, and a decoding system;

[0011] The large-field-of-view image reflected by the ground object is equally divided into n information blocks, and each information block is transmitted to a corresponding optical path; each information block undergoes the first light adjustment, encoding, and the second light adjustment in the corresponding optical path in sequence before entering the sensor; the sensor receives the n encoded information blocks, converts the n encoded information blocks into n electrical signals, and stores the n electrical signals in the storage and transmission module; the storage and transmission module sends the n electrical signals to the decoding system; the decoding system decodes and recovers the n electrical signals to obtain each piece of image information, obtains a large-field-of-view image after splicing, and sends the large-field-of-view image to the downstream terminal; n is a positive integer not less than 3.

[0012] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, each optical path includes a lens, an optical encoding device and a relay lens;

[0013] Lens: Receives the information block corresponding to the image with a large field of view reflected by the ground object; performs the first light adjustment on the corresponding information block, converts the corresponding information block light into the horizontal direction, and transmits it to the corresponding optical encoding device;

[0014] Optical encoding device: receives the information block from the lens, encodes the information block, generates an encoded information block, and sends the encoded information block to the relay lens;

[0015] Relay lens: Receives the encoded information block from the optical encoding device, performs a second light adjustment on the encoded information block; converts the horizontal direction of the encoded information block to point to the sensor direction.

[0016] In the above-mentioned pre-sensing compression system for space-based remote sensing images with a large field of view, n is the compression rate of the large field of view image; it is set according to downstream task requirements.

[0017] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, the optical encoding device encodes the information block according to a preset encoding pattern.

[0018] In the above-mentioned pre-sensing compression system of space-based remote sensing large-field-of-view images, the information block is the light field information obtained by dividing the large-field-of-view image reflected by the ground object into n equal parts; the sensor converts the n encoded light field information into electrical signals and performs a readout operation.

[0019] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, the space occupied by the sensor is reduced to the original size according to the compression rate. This effectively reduces the space requirements for sensors and the information processing and storage pressure at the back end.

[0020] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, the storage and transmission module realizes the storage and distribution of n electrical signals. When the downstream terminal sends a request to obtain a large-field-of-view image, the storage and transmission module sends the n electrical signals to the decoding system.

[0021] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, the lens converts the information block corresponding to the incident angle into the horizontal direction; the relay lens is a lens structure, which deflects the horizontal direction and converges n encoded information blocks to the sensor.

[0022] In the above-mentioned pre-sensing compression system for space-based remote sensing images with a large field of view, the decoding algorithm of the decoding system is obtained by training with a preset coding pattern.

[0023] In the above-mentioned pre-sensing compression system for space-based remote sensing large-field-of-view images, the training process of the decoding system is as follows:

[0024] Perform directional task-driven network training according to downstream tasks, that is, select appropriate training data and network structure according to different task scenarios and subsequent task requirements, and realize adjustable parameter training of encoding graph generation algorithm and decoding algorithm model.

[0025] The beneficial effects of the present invention compared with the prior art are:

[0026] (1) This invention proposes a new pre-sensing computing paradigm, called the pre-sensing optical coding computing paradigm, which can realize encoding and computing in the optical domain before the space-based remote sensor captures the image, thus realizing pre-sensing processing of remote sensing images;

[0027] (2) The pre-sensing optical coding computing paradigm proposed in the present invention performs coding and computing in the optical domain. It realizes block coding operation in the optical domain by designing the arrangement of optical coding devices, realizes binary coding adjustment through optical coding elements on the optical path, and the encoded light field information is calibrated through optical path adjustment and imaging on the same optical sensor, thereby realizing compression operation on the space-based remote sensing large field image space.

[0028] (3) The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view. The sensor converts the encoded light field information into an electrical signal and performs a readout operation. The size of the sensor used can be reduced accordingly according to the compression ratio, and the space size requirement occupied by the sensor and the information processing and storage pressure of the back-end are reduced. The readout image contains multiple blocks of image information of the large field of view image, thus realizing a spatial compression operation.

[0029] (4) The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view. The compressed small-width (compared to the original space-based remote sensing images with a large field of view) images are decoded and restored using a preset decoding system to obtain each piece of image information, and then spliced ​​together to obtain images with a large field of view.

[0030] (5) The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view, which moves part of the previous electrical domain operations to the optical domain before imaging, and can achieve data compression on the optical path, saving on-board resources and energy consumption, and providing a new technical idea for the compression of space-based remote sensing images with a large field of view. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a diagram of a pre-sensing compression system for space-based remote sensing large-field-of-view images of the present invention;

[0032] Figure 2 It is a schematic diagram of the prior art front compression system;

[0033] Figure 3 This is a comparison diagram of the original image and the reconstructed image of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the embodiments.

[0035] The present invention proposes a pre-sensing compression system for space-based remote sensing images with a large field of view, and proposes a new pre-sensing computing paradigm, called Pre-Sensor Optical Encoding (PreSOE), which can realize encoding and computing in the optical domain before the sensor captures the image. The technical solution of the present invention is: the space-based optical system collects ground object information, and by designing the arrangement of optical encoding devices, block encoding operations are implemented in the optical domain, and binary encoding adjustment is implemented on the optical path through the optical encoding element. The encoded light field information is calibrated through the optical path adjustment, and imaging is performed on the same optical sensor to realize compression operations on the space-based remote sensing image with a large field of view.

[0036] Pre-sensing compression system for space-based remote sensing large field of view images, such as Figure 1 As shown, it specifically includes n optical paths, sensors, storage and transmission modules and a decoding system.

[0037] The large field of view image reflected by the ground object is divided into n information blocks, and each information block is transmitted to a corresponding optical path; each information block is adjusted for the first time, encoded, and adjusted for the second time in the corresponding optical path, and then enters the sensor; the sensor receives the n encoded information blocks, converts the n encoded information blocks into n electrical signals, and stores the n electrical signals in the storage and transmission module; the storage and transmission module sends the n electrical signals to the decoding system; the decoding system decodes and recovers the n electrical signals, obtains each piece of image information, and obtains the large field of view image after splicing, and sends the large field of view image to the downstream terminal; n is a positive integer not less than 3. n is the compression rate of the large field of view image; it is set according to the downstream task requirements.

[0038] Each optical path includes a lens, an optical encoding device and a relay lens.

[0039] Lens: Receives the information block corresponding to the large-field image reflected by the ground object; performs the first light adjustment on the corresponding information block, converts the corresponding information block light into the horizontal direction, and transmits it to the corresponding optical encoding device.

[0040] Optical encoding device: receives the information block from the lens, encodes the information block, generates an encoded information block, and sends the encoded information block to the relay lens.

[0041] Relay lens: Receives the encoded information block from the optical encoding device, performs a second light adjustment on the encoded information block; converts the horizontal direction of the encoded information block to point to the sensor direction.

[0042] The optical encoding device encodes the information block according to a preset encoding pattern.

[0043] The information block is the light field information obtained by dividing the large field of view picture reflected by the ground object into n equal parts; the sensor converts the n encoded light field information into electrical signals and performs a readout operation.

[0044] like Figure 2 As shown, the space occupied by the sensor designed by the present invention is reduced to the original size according to the compression ratio. This effectively reduces the space requirements for sensors and the information processing and storage pressure on the back end.

[0045] The storage and transmission module realizes the storage and distribution of n electrical signals. When the downstream terminal sends a request to obtain a large field of view image, the storage and transmission module sends the n electrical signals to the decoding system.

[0046] The lens converts the information block corresponding to the incident angle into the horizontal direction; the relay lens is a lens structure, which deflects the horizontal direction and converges n encoded information blocks to the sensor.

[0047] The decoding algorithm of the decoding system is obtained by training with preset coding patterns.

[0048] The training process of the decoding system is:

[0049] Perform directional task-driven network training according to downstream tasks, that is, select appropriate training data and network structure according to different task scenarios and subsequent task requirements, and realize adjustable parameter training of encoding graph generation algorithm and decoding algorithm model.

[0050] The present invention depicts the physical layout of the entire framework. The light field information reflected by the ground object enters the optical system, and the lens is used to perform spectroscopic processing on the entrance pupil light beam, and each beam is focused on the optical coding element. The number of correspondingly increased coding elements corresponds to the compression rate of the large-field-of-view remote sensing image, which is the number of blocks here. According to the preset coding pattern, the coding device encodes each piece of light field information received in the optical path, and the lens group and the like transmit the encoded and calculated data to the photosensitive surface of the sensor, and converge on the same sensor through the adjustment of the optical path; the sensor converts the encoded light field information into an electrical signal and performs a readout operation. Here, a large-scale spliced ​​imaging surface is no longer used, and the size of the sensor used can be reduced accordingly according to the compression ratio, and the size requirement of the sensor is reduced. At the same time, the on-board storage pressure and the information processing pressure after imaging are effectively reduced. The readout image contains multiple blocks of image information of the large-field-of-view image, realizing the spatial compression operation, and the compressed image data and encoding-related information are effectively stored in the storage and transmission module. Assuming the compression ratio is n, then Figure 2 As shown in the figure, compared with the on-board electrical domain compression of traditional space-based remote sensing imaging, the pre-sensing compression method of space-based remote sensing large field of view images can fully save energy consumption and computing resources, realize compression operations in the optical domain, and reduce the imaging surface size to the original The high difficulty requirements for the imaging surface are greatly reduced, and the impact of the entire compression operation on computing resources during the optical process is negligible. By using the optical block encoding and superimposed compression method, the sensor size can be effectively reduced, providing a solution to reduce the difficulty of stitching wide imaging surfaces.

[0051] When the optical path information enters the entire system, the calculation is performed immediately, highlighting the real-time processing capability of the system framework. The entire process of PreSOE is isolated from other systems of the satellite, so no processor resources need to be allocated. After conversion to electrical data, the block division and summation of the modulated image constitute the main source of computational complexity in the encoding process. The compressed small-width (compared to the original space-based remote sensing large-field-of-view) image uses a preset decoding system to decode and restore the compressed image information to obtain each piece of image information, and after splicing, a large-field-of-view image is obtained. The algorithm of the decoding system is trained by the preset coding pattern, and the directional task-driven network training can be performed according to the downstream tasks, that is, the appropriate training data and network structure can be selected according to different task scenarios and subsequent task requirements, and the adjustable parameter training of the coding image generation algorithm and the decoding algorithm model can be realized. The reconstruction network adopts the Deep Unfolding Network (DUN) framework using gated three-dimensional convolution. Each stage contains a 3D U-net architecture. In order to further enhance the performance of the network, a gating mechanism is introduced for the 3D convolution. Since the initialization measurement used for restoration contains invalid pixels, the convolution kernel that applies the same weight to all pixels may not be the optimal parameter. Therefore, by introducing additional branches to calculate the weights, the network can dynamically select important features. At the same time, in the reconstruction algorithm, a new two-way cross attention (TWCA) module is proposed to achieve full information interaction between stages by making full use of the latent vectors generated by the 3D U-net encoder. Block artifacts will appear when the restored image is spliced ​​with a large width original image. A subsequent 2D U-net is merged after DUN to optimize the results.

[0052] Before being put into use, simulation can be performed according to the usage scenario, the encoding used can be calibrated, and the intensity of the input light field can be adjusted through the full black and full white encoding normalization operations, so as to adaptively adjust the range of image data, avoid the problem of local overexposure of stored data and loss of information under high compression ratios, and avoid poor performance of the recovery algorithm. Finally, the received compressed image is decompressed through a decoding system based on a preset decoding algorithm and encoding feature map, and the original light field information is retained as much as possible during the decompression process to achieve high-fidelity image restoration.

[0053] In order to verify the feasibility of the reconstruction algorithm in the experiment, the dataset used in the simulation was actually photographed in a natural light scene, and the encoded remote sensing image was used as the input of the spatial compression system to generate compression and reconstruction results. In addition, since the hinge of the micromirror is along the diagonal of the mirror, the mirror rotates around an axis that is 45° to the array size, and the collected image will produce a 45° rotation result. We pre-processed the images used in the experiment and cropped the collected images to meet the image size required for reconstruction (512×512). Figure 3 As shown, the original image and the compressed reconstructed image obtained in the actual test.

[0054] The present invention proposes a pre-sensing compression method for space-based remote sensing large-field-of-view images, which is a new pre-sensing computing paradigm, called the pre-sensing optical coding computing paradigm, which can realize encoding and calculation in the optical domain before the space-based remote sensor captures the image, thereby realizing pre-sensing processing of the remote sensing image.

[0055] The pre-sensing optical coding computing paradigm proposed in the present invention performs coding and computing in the optical domain. It realizes block coding operations in the optical domain by designing the arrangement of optical coding devices, and realizes binary coding adjustment through optical coding elements on the optical path. The encoded light field information is calibrated through optical path adjustment and imaging is performed on the same optical sensor, thereby realizing compression operations on large-field-of-view images of space-based remote sensing.

[0056] The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view. The sensor converts the encoded light field information into an electrical signal and performs a read operation. The size of the sensor used can be reduced accordingly according to the compression ratio, and the size requirement of the sensor and the information processing and storage pressure of the back-end are reduced. The read image contains multiple blocks of image information of the large field of view image, thereby realizing a spatial compression operation.

[0057] The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view. The compressed small-width (compared with the original space-based remote sensing images with a large field of view) image can be decoded and restored using a preset decoding system to obtain each piece of image information, and a large-field-of-view image can be obtained after splicing.

[0058] The present invention proposes a pre-sensing compression method for space-based remote sensing images with a large field of view, which places part of the previous electrical domain operations on the optical domain before imaging, so that data compression can be achieved on the optical path, saving on-board resource and energy consumption, and providing a new technical idea for space-based remote sensing image compression with a large field of view.

[0059] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A pre-sensing compression system for space-based remote sensing large-field-of-view images, characterized in that: It includes n optical paths, sensors, storage and transmission modules and decoding systems; The large field of view image reflected by the ground object is equally divided into n information blocks, and each information block is transmitted to a corresponding optical path; each information block is sequentially subjected to the first light adjustment, encoding, and second light adjustment in the corresponding optical path before entering the sensor; the sensor receives the n encoded information blocks, converts the n encoded information blocks into n electrical signals, and stores the n electrical signals in the storage and transmission module; The storage transmission module sends n electrical signals to the decoding system; The decoding system decodes and recovers n electrical signal rows, obtains each piece of image information, splices them together to obtain a large field of view image, and sends the large field of view image to the downstream terminal; n is a positive integer not less than 3.

2. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 1, characterized in that: Each optical path includes a lens, an optical encoding device and a relay lens; Lens: Receives the information block corresponding to the image with a large field of view reflected by the ground object; performs the first light adjustment on the corresponding information block, converts the corresponding information block light into the horizontal direction, and transmits it to the corresponding optical encoding device; Optical encoding device: receives the information block from the lens, encodes the information block, generates an encoded information block, and sends the encoded information block to the relay lens; Relay lens: Receives the encoded information block from the optical encoding device, performs a second light adjustment on the encoded information block; converts the horizontal direction of the encoded information block to point to the sensor direction.

3. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 2, characterized in that: The n is the compression ratio of the large field of view image, which is set according to the downstream task requirements.

4. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 2, characterized in that: The optical encoding device encodes the information block according to a preset encoding pattern.

5. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 3 is characterized in that: The information block is light field information obtained by dividing a large field of view picture reflected by a ground object into n equal parts; the sensor converts the n encoded light field information into electrical signals and performs a readout operation.

6. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 5, characterized in that: The space occupied by the sensor is reduced to its original size according to the compression ratio. This effectively reduces the space requirements for sensors and the information processing and storage pressure at the back end.

7. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 2, characterized in that: The storage and transmission module realizes the storage and distribution of n electrical signals. When the downstream terminal sends a request for obtaining a large field of view image, the storage and transmission module sends the n electrical signals to the decoding system.

8. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 2, characterized in that: The lens converts the information block corresponding to the incident angle into the horizontal direction; the relay lens is a lens structure, which deflects the horizontal direction and converges n encoded information blocks to the sensor.

9. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 4, characterized in that: The decoding algorithm of the decoding system is obtained by training with a preset coding pattern.

10. The pre-sensing compression system for space-based remote sensing large-field-of-view images according to claim 9, characterized in that: The training process of the decoding system is: Perform directional task-driven network training according to downstream tasks, that is, select appropriate training data and network structure according to different task scenarios and subsequent task requirements, and realize adjustable parameter training of encoding graph generation algorithm and decoding algorithm model.

Images