Spatial situation awareness image compression method and system based on aggregation sparse coding
Through aggregation of sparse coding and overlap-perceived adaptive binary arithmetic coding, the multi-view redundancy problem in the prior art is solved, the image compression rate and reconstruction quality are improved, and the computing resource consumption is reduced, and it is suitable for low-orbit satellite environments.
Patent Information
- Application Number
- CN202411263803.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-09-10
AI Technical Summary
The existing image compression technology based on sparse encoding has failed to effectively solve the problem of multi-view redundancy. There is no further processing of pixels in overlapping areas taken by adjacent low-orbit satellites, resulting in insufficient compression rate, high computational complexity, and failure to be effective. Take advantage of space-time correlation.
The spatial situation-aware image compression method based on aggregate sparse encoding is adopted, and adjacent satellites are allocated for distributed calculations through background segmentation and overlapping area detection. The overlapping perception adaptive binary arithmetic coding algorithm is used to further compress image data and reduce computing resource consumption.
A space situation-aware image compression rate of 93.78% is achieved, the reconstructed image quality is better than the existing methods, and can be effectively deployed in resource-constrained low-orbit satellite environments, significantly reducing CPU and RAM usage.
Smart Images

Figure CN119211567A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of image processing, and specifically to a spatial situation awareness image compression method and system based on aggregated sparse coding. Background Art
[0002] Satellite images are both sparse and temporally correlated, that is, the information entropy contained in space situational awareness images is low. Specifically, the foreground of such images contains near-orbit targets, such as space debris, missiles, abnormal aircraft, and other fast-moving objects, which appear as sparse white dots or lines, and the pixel positions change in continuous images. Most of the background is deep space and static white dots, such as distant stars and the Milky Way. Since the shooting parameters (such as field of view and shooting direction) remain consistent, low-orbit satellites perform periodic motion according to the ephemeris, resulting in periodic repetition of the background of space situational awareness images. In the images taken at the same time by two adjacent low-orbit satellites with spatial correlation, there are areas for overlapping perspectives. These overlapping areas are the same areas observed by adjacent satellites from different angles, and many elements such as stars, planets, and aircraft are consistent. Existing image compression technologies based on sparse coding use the inherent sparsity of space situational awareness images to represent pixels by a reduced set of basic elements, including decomposition-based and convolution-based methods: the decomposition-based method uses a separable dictionary for each image dimension, and the convolution-based method uses a multidimensional dictionary to effectively handle high-order correlations. However, these methods have failed to effectively solve the multi-view redundancy problem, and there is no further processing of the overlapping area pixels captured by adjacent low-orbit satellites at the same time. Summary of the invention
[0003] In view of the shortcomings of the prior art, such as the low compression rate, failure to consider the spatiotemporal correlation of images taken by adjacent low-orbit satellites, inability to solve the spatiotemporal sparse redundancy of space situational awareness images, and high computational complexity, the present invention proposes a space situational awareness image compression method and system based on aggregated sparse coding, which improves the compression rate of the corresponding images and ensures the reconstruction quality of the images. At the same time, the distributed computing method is used to reduce the resource consumption of a single satellite during calculation, ensuring that the system can still be effectively deployed in a low-orbit satellite constellation environment with limited computing and transmission resources.
[0004] The present invention is achieved through the following technical solutions:
[0005] The present invention relates to a space situation awareness image compression method based on aggregate sparse coding. After background segmentation is performed on the collected space situation awareness image, overlapping areas are detected on the foreground and allocated to adjacent satellites. Each satellite obtains a dictionary tensor and a coefficient tensor of allocated overlapping or non-overlapping areas through aggregate sparse coding in turn according to images from adjacent satellites. Then, the coefficient tensor is further compressed into a transfer tensor and a position tensor expressing the pixel relationship of the overlapping area through an overlap-aware adaptive binary arithmetic coding algorithm, and then output to a ground station for image reconstruction.
[0006] The background segmentation refers to: combining the advantages of sparse coding and local binary pattern, and using the SCS-LBP operator to perform background and foreground segmentation.
[0007] The background segmentation described above outputs the obtained background area to a ground station for image reconstruction.
[0008] The allocation refers to: pre-allocating serial numbers within the range of [0, N] for N low-orbit satellites in the same layer, satellites with odd serial numbers are allocated to overlapping area pixels transmitted by satellites within an adjacent one-hop range, and satellites with even serial numbers are allocated to non-overlapping area pixels transmitted by satellites within an adjacent one-hop range, so that the same space situation awareness image is divided into overlapping and non-overlapping areas for distributed compression calculation.
[0009] Preferably, the inter-satellite transmission is performed by establishing a link through capture, tracking and aiming by an on-board laser communication payload, and then 1550nm wavelength communication is performed between adjacent satellites in the same orbital plane.
[0010] The overlapping area detection described above is relatively fixed because the distance between two adjacent low-orbit satellites on the same layer is relatively fixed, and the imaging parameters on each satellite (such as focal length, sensor size, aperture and shooting direction) are fixed, so that the overlapping area width of the space situation awareness images taken by adjacent satellites is relatively fixed, and the overlapping area length is consistent with the length of the taken photos. Therefore, it is only necessary to perform overlapping area detection after the space situation awareness image is captured at the first moment, and the obtained overlapping width is applicable to image detection at subsequent moments, specifically including:
[0011] Step 1: Each satellite uses the SIFT operator to extract feature points and descriptors from the captured images, and exchanges the extracted feature points and descriptor results and the corresponding image shooting timestamps with adjacent satellites. Adjacent satellites use the BFMatcher operator to match the feature points in two images taken at the same time.
[0012] Step 2: Adjacent satellites use the RANSAC algorithm to calculate the homography matrix by matching feature points, which is used to transform the first image into the coordinate system of the second image to identify the overlapping area.
[0013] Step 3: Each satellite calculates the area of the overlapping region using the area equivalence method, and then divides the area by the image length to determine the overlapping width, thereby obtaining the boundary coordinates of the overlapping region.
[0014] Step 4: The overlapping width of two adjacent captured images is finally determined as a larger value of two values, and then the larger width value is used to update the boundary coordinates of the overlapping area in the two images, and finally the range of the overlapping area is determined.
[0015] The group sparsity means that for a tensor of order 3 Its group sparsity ||X|| 1,1,2 For X ij The number of non-zero entries of , and relaxes to a convex version where:
[0016] The aggregated sparse coding specifically includes:
[0017] Step a: Initialize dictionary tensor D mt and auxiliary tensor A.
[0018] Step b: Update the auxiliary tensor, specifically: in: D' mt is the dictionary tensor D mt The transpose of ||D' mt *D mt || is the binary norm result after convolution of two tensors.
[0019] Step c, repeat step b until the number of iterations reaches i, by Perform Fourier transform to obtain dictionary tensor D mt ,in: is the frequency domain representation obtained by Fourier transform of the auxiliary tensor A in the third dimension, For the conjugate transform of this representation, diag() produces a diagonal matrix with all diagonal elements equal to λ.
[0020] Step d: Dictionary tensor D obtained from the overlapping areas captured by two adjacent satellites it and D jt Perform dictionary aggregation to obtain the aggregated dictionary
[0021] Step e: The aggregated dictionary tensor obtained from step d And the dictionary tensor D obtained in step c mt , with the goal of minimizing the loss between the reconstructed image obtained by performing tensor convolution calculation on the dictionary tensor and the original image, the dictionary tensor Dmt As the auxiliary tensor C mt , update using the method of step bc to obtain the dictionary tensor and coefficient tensor of the overlapping or non-overlapping area, specifically: The restriction condition is that the square root of the sum of the squares of all elements in the dictionary vector matrix of each channel does not exceed 1, which is used to limit the complexity of the dictionary solution, that is: Where: m is the satellite serial number, t is the shooting time, n3 is the number of channels of each space situational awareness image, is the tensor of the image captured by the mth satellite at the tth time on the jth channel, D mt is the dictionary tensor learned by sparse coding for the image, is the coefficient tensor on the corresponding channel, β is the balance coefficient, which is generally taken as 0.1. ||·|| 1,1,2 is the group sparsity.
[0022] The overlap-aware adaptive binary arithmetic coding algorithm specifically includes:
[0023] Step i: transform the coefficient tensor C mt Each floating point value in is quantized to an integer value The coefficient value c i Round to the nearest integer a i ∈[0,9], the quantization interval is Δ.
[0024] Step ii: rearrange the quantized coefficient tensor into an importance map p that records the position of the non-zero coefficients in the tensor t and the corresponding level map used to record its value
[0025] Step iii. For level mapping, first use fixed-length binarization to encode it into a sequence of 0 and 1. For non-overlapping area pictures, directly use context-aware binary arithmetic coding (CABAC). For two overlapping area pictures at the same time, they are represented as the current view CurrView(x,y) and the reference view RefView(x',y'), respectively. Based on the pixel values and context features in the two views, the context index ctxIdx = f(CurrView(x,y), RefView(x',y'), D(x,y,x',y')) is obtained, where: D(x,y,x',y') is the absolute difference of pixels at corresponding positions of the two pictures, and f(·) is a quantization function used to map continuous values to discrete context index space.
[0026] Step iv: Update the binary coded probability model P(b=0|ctx)=εP(b=0|ctx) according to the context index CurrView )+(1-ε)P(b=0|ctxRefView ), where: P(b=1|ctx)=1-P(b=1|ctx), P(b=0|ctx CurrView ) is the probability based on the current view context, P(b=0|ctx RefView ) is the probability of the reference view context after the transformation. ε is a weight factor that can be adjusted according to the size response of D(x,y,x',y'). The smaller D is, the larger the ε value is.
[0027] Step v: Perform arithmetic coding on the binary sequence of the level mapping according to the updated probability model.
[0028] The image reconstruction means that the ground station retrieves the corresponding background pixels according to the satellite sequence number m and the shooting time t And get the coefficient tensor C for decoding mt After that, the image is reconstructed using tensor convolution calculation Where: t = t mod T. Technical Effects
[0029] The present invention realizes direct on-board calculation of low-orbit satellite constellations based on the aggregated sparse coding algorithm and the overlap-aware adaptive binary arithmetic coding algorithm based on overlapping area detection, and achieves a space situational awareness image compression rate of 93.78%. In addition, under the conditions of 300Mbps satellite-to-ground transmission downlink bandwidth, 10-minute satellite-to-ground communication connection, and 20 satellite-to-ground communication connections per day, the space situational awareness image captured at a sampling rate of 30Hz within 24 hours can be successfully transmitted to the ground station within one day at a data transmission rate of 93.04%. The above effect is significantly better than existing benchmark methods including ZIP, JPEG, multidimensional tensor sparse coding (MDTSC) and GAACNN based on deep learning. In addition, the image reconstructed by the method obtains an SSIM value of 0.73 and a PSNR value of 27.56dB, both of which are better than multidimensional tensor sparse coding (MDTSC). Compared with centralized image sparse coding, the present invention can reduce CPU and RAM usage and processing time by about 20%, so it is feasible to deploy the method in a satellite network with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of the overall structure of the present invention;
[0031] Figure 2 It is the flow chart of the aggregation sparse coding algorithm;
[0032] FIG3 is a schematic diagram of the embodiment architecture;
[0033] In the figure: Figure 3a To simulate the communication between ground station and satellite; Figure 3bA schematic diagram of the satellite computer and computing payload simulating a single satellite.
[0034] FIG4 is an effect analysis diagram of an image compression algorithm;
[0035] In the figure: Figure 4a This is a comparison chart of image compression ratio. Figure 4b For the data transfer rate diagram, Figure 4c Compute delay graph for single image compression;
[0036] FIG5 is a visual diagram of a sparse coding reconstructed image;
[0037] FIG6 is a diagram showing the computing resource utilization analysis. DETAILED DESCRIPTION
[0038] like Figure 1 As shown, a space situation awareness image compression system based on aggregated sparse coding involved in this embodiment includes: an overlapped area segmentation and allocation module, a background segmentation module, a sparse coding module and an image reconstruction module, wherein: the background segmentation module performs foreground and background segmentation according to the captured space situation awareness image and outputs the background area to the ground station; the overlapped area segmentation and allocation module calculates the pixel area belonging to the same field of view according to the pixel relationship of the foreground area in the image captured by two adjacent satellites at the same time after segmentation, and allocates the corresponding area to the corresponding satellite for image compression; the sparse coding module performs aggregated sparse coding and overlap-aware adaptive binary arithmetic coefficient coding according to the received image area, further reducing the dictionary and coefficient size of the sparse coding output; the image reconstruction module performs the final image reconstruction in conjunction with the background part according to the encoded dictionary and coefficient.
[0039] The allocation means that the overlapping pixel areas under the same field of view are allocated to the odd-numbered satellites within the adjacent one-hop range for subsequent image compression, and the remaining non-overlapping pixel areas are allocated to the even-numbered satellites within the adjacent one-hop range for subsequent image compression. The corresponding areas are determined based on whether the serial number of the local satellite is odd or even, and are directly involved in subsequent calculations on the local satellite without additional allocation.
[0040] The sparse coding module includes: an aggregated sparse coding unit and an overlap-aware adaptive binary coefficient arithmetic coding unit, wherein: the dictionary aggregation unit aggregates dictionary tensors in overlapping areas to represent common basic elements in the image, thereby further reducing the size of the dictionary obtained by sparse coding; the overlap-aware adaptive binary coefficient arithmetic coding unit utilizes the sparsity of the coefficient matrix obtained by sparse coding, applies geometric transformation on the overlapping area to predict the probability model, encodes the coefficient tensor into a more compact string, thereby further reducing the size of the coefficient obtained by sparse coding.
[0041] The background segmentation module includes: a background segmentation unit and a background uploading unit, wherein: the background segmentation unit uses the SCS-LBP operator to perform background and foreground segmentation. The background uploading unit outputs the obtained background area to the ground station for image reconstruction.
[0042] The image reconstruction module includes: a background query unit and a reconstruction calculation unit, wherein: the background query unit retrieves the corresponding background pixels according to the satellite sequence number and the shooting time and obtains the coefficient tensor for decoding, and the reconstruction calculation unit uses the background image, the dictionary tensor and the decoded coefficient tensor to perform tensor convolution calculation to reconstruct the image.
[0043] like Figure 2 As shown, a spatial situational awareness image compression method based on aggregated sparse coding based on the above-mentioned system involved in this embodiment, the dictionary tensor and the auxiliary tensor are initialized according to the image to be encoded, and the auxiliary tensor is iteratively updated according to the designed algorithm. The number of iterations will affect the computing resource consumption and computing delay, and 3 times are selected in the embodiment. After the auxiliary tensor is updated, the frequency domain representation of the dictionary tensor will be calculated at the same time, and the corresponding time domain representation dictionary tensor will be obtained by performing Fourier transform, and the dictionary tensor will be aggregated for the overlapping area. The aggregated dictionary will be used as input to enter the step of iterative updating of the auxiliary tensor, and the updated auxiliary tensor is the coefficient tensor finally solved.
[0044] After specific practical experiments, in the hardware simulation platform based on low-orbit satellites shown in Figure 3, Raspberry Pi is used as the satellite's on-orbit computer, which is used to control the satellite and manage information exchange, respectively, but the embodiment can be applied to other embedded development platforms, such as Arduino. Each Raspberry Pi is connected to a Jetson Orin OX, which is simulated as a computing load device. The Jetson device is equipped with an 8-core NVIDIAA78 CPU and 16GB LPDDR5 memory, which can provide 100INT8 TOPS of computing power. However, the embodiment can be applied to other types of lightweight embedded computing devices, such as Jetson Orin Nano. In addition, the ground station is simulated by a Linux server running the Ubuntu 16.04LTS system, and the server is configured with a 12-core 3.2GHz Intel Core i7-8700 CPU and a GeForce GTX 10606GB GPU. However, the embodiment can be applied to other high-performance computing devices, such as HP DL20.
[0045] The space situational awareness images used in this embodiment are pre-divided into overlapping and non-overlapping areas and stored on two Raspberry Pis, simulating the input data received by odd-numbered and even-numbered satellites, respectively. After receiving the image, the Jetson computing device will run the sparse coding module to compress the image. The compressed result is sent to the server via the Raspberry Pi's WiFi module, which has a maximum transmission bandwidth of 300Mbps, which is equivalent to the satellite downlink bandwidth. The space situational awareness images used in this embodiment are taken by two astronomical telescopes. The shooting equipment has a resolution of 1080×1920, an optical aperture of 50mm, and a focal length of 250mm, and can capture space images at a sampling rate of 30Hz.
[0046] like Figure 4a As shown, it is a comparison between the present invention and a variety of existing image compression algorithms. The compression ratio of the dictionary aggregation sparse coding algorithm involved in the present invention is about 93.78%;
[0047] like Figure 4b As shown, it is the data transmission ratio of the satellite downlink to the ground station, which is represented by the transmission ratio of the original captured image data. It can be seen that the amount of original image data that can be transmitted by using the aggregated sparse coding algorithm for image compression is the largest;
[0048] like Figure 4c As shown, the delay of different compression algorithms running on orbit is shown for the compression delay of a single image. It can be seen that the delay of the dictionary aggregation sparse coding algorithm involved in the present invention is about 13.68s, which is much smaller than the interval between two adjacent satellite downlinks. At the same time, the delay is shorter than that of MDTSC, which verifies the feasibility of its on-orbit image compression operation.
[0049] like Figure 5a As shown in the figure, it is a visual comparison of the MDTSC reconstructed image. The left side is the reconstruction result of the whole image, and the right side is the enlarged detail image of the image with a 100*100 pixel block in the upper left corner. Figure 5b As shown, the visualization of the image reconstructed by the dictionary aggregation sparse coding algorithm involved in the present invention. Similarly, the left side is the reconstruction result of the entire image, and the right side is an enlarged detail image of the image with a 100*100 pixel block in the upper left corner. It can be seen from observation that the effect of the dictionary aggregation sparse coding algorithm involved in the present invention is due to the use of a dictionary learned from multiple perspectives, not a single perspective, and the final reconstructed image provides more details than MDTSC. Accordingly, the PSNR value of the image reconstructed by the dictionary aggregation sparse coding algorithm involved in the present invention is 27.56dB, and the SSIM is 0.73, both higher than the PSNR value and SSIM value of the image reconstructed by MDTSC, which are 25.03dB and 0.65 respectively.
[0050] As shown in Figure 6, the computing resource occupancy rate of the space situational awareness image compression system during operation, where: Figure 6a The CPU and RAM computing resource usage on a single computing device over time when using centralized image compression computing is simulated. Figure 6b The CPU and RAM computing resource usage over time when the overlapping area image compression processing is performed on a single odd-numbered satellite using distributed image compression computing is simulated. Figure 6c The CPU and RAM computing resource usage over time for overlapping region image compression processing on a single even-numbered satellite using distributed image compression computing is simulated. It is observed that, with distributed computing, CPU and RAM utilization and processing time are significantly reduced compared to centralized coding on a single satellite, especially when processing non-overlapping regions. It is also observed that the proposed sparse coding method uses about 40% of the CPU and 20%-30% of the RAM, which indicates that the invention is feasible on satellites with limited computing power, storage and energy, and its moderate resource consumption enables it to run in parallel with other satellite missions (such as attitude and orbit control and communication).
[0051] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A spatial situational awareness image compression method based on aggregated sparse coding, characterized in that: After background segmentation of the collected space situational awareness images, the overlapping areas of the foreground are detected and allocated to adjacent satellites. Each satellite obtains the dictionary tensor and coefficient tensor of the allocated overlapping or non-overlapping areas through aggregate sparse coding according to the images from adjacent satellites. The coefficient tensor is then further compressed into a transfer tensor and a position tensor that express the pixel relationship of the overlapping area through the overlap-aware adaptive binary arithmetic coding algorithm, and then output to the ground station for image reconstruction.
2. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The background segmentation refers to: combining the advantages of sparse coding and local binary pattern, using SCS-LBP operator to perform background and foreground segmentation; outputting the obtained background area to the ground station for image reconstruction.
3. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The allocation refers to: pre-allocating serial numbers within the range of [0, N] for N low-orbit satellites in the same layer, satellites with odd serial numbers are allocated to overlapping area pixels transmitted by satellites within an adjacent one-hop range, and satellites with even serial numbers are allocated to non-overlapping area pixels transmitted by satellites within an adjacent one-hop range, so that the same space situation awareness image is divided into overlapping and non-overlapping areas for distributed compression calculation.
4. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The overlapping area detection described above is relatively fixed because the distance between two adjacent low-orbit satellites on the same layer is relatively fixed, and the imaging parameters on each satellite are fixed, so that the overlapping area width of the space situation awareness images taken by adjacent satellites is relatively fixed, and the overlapping area length is consistent with the length of the taken photos. Therefore, it is only necessary to perform overlapping area detection after the space situation awareness image is captured at the first moment, and the obtained overlapping width is applicable to image detection at subsequent moments.
5. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 or 4, characterized in that: The overlapping area detection specifically includes: Step 1: Each satellite uses the SIFT operator to extract feature points and descriptors from the captured images, and exchanges the extracted feature points and descriptor results and the corresponding image shooting timestamps with adjacent satellites. Adjacent satellites use the BFMatcher operator to match the feature points in two images taken at the same time. Step 2: Adjacent satellites use the RANSAC algorithm to calculate the homography matrix by matching feature points, which is used to transform the first image into the coordinate system of the second image to identify the overlapping area; Step 3: Each satellite calculates the area of the overlapping region using the area equivalence method, and then divides the area by the image length to determine the overlapping width, thereby obtaining the boundary coordinates of the overlapping region; Step 4: The overlapping width of two adjacent captured images is finally determined as a larger value of two values, and then the larger width value is used to update the boundary coordinates of the overlapping area in the two images, and finally the range of the overlapping area is determined.
6. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The aggregated sparse coding specifically includes: Step a: Initialize dictionary tensor D mt and auxiliary tensor A; Step b: Update the auxiliary tensor, specifically: in: D' mt is the dictionary tensor D mt The transpose of ||D' mt *D mt || is the binary norm result after convolution of two tensors; Step c, repeat step b until the number of iterations reaches i, by Perform Fourier transform to obtain dictionary tensor D mt ,in: is the frequency domain representation obtained by Fourier transform of the auxiliary tensor A in the third dimension, For the conjugate transformation of this representation, diag() generates a diagonal matrix with all diagonal elements being λ; Step d: Dictionary tensor D obtained from the overlapping areas captured by two adjacent satellites it and D jt Perform dictionary aggregation to obtain the aggregated dictionary Step e: The aggregated dictionary tensor obtained from step d And the dictionary tensor D obtained in step c mt , with the goal of minimizing the loss between the reconstructed image obtained by performing tensor convolution calculation on the dictionary tensor and the original image, by mt As the auxiliary tensor C mt , update using the method of step bc to obtain the dictionary tensor and coefficient tensor of the overlapping or non-overlapping area, specifically: The restriction condition is that the square root of the sum of the squares of all elements in the dictionary vector matrix of each channel does not exceed 1, which is used to limit the complexity of the dictionary solution, that is: Where: m is the satellite serial number, t is the shooting time, n3 is the number of channels of each space situational awareness image, is the tensor of the image captured by the mth satellite at the tth time on the jth channel, D mt is the dictionary tensor learned by sparse coding for the image, is the coefficient tensor on the corresponding channel, β is the balance coefficient, which is generally taken as 0.1, ||·|| 1,1,2 is the group sparsity; The group sparsity means that for a tensor of order 3 Its group sparsity ||X|| 1,1,2 For X ij The number of non-zero entries of , and relaxes to a convex version, where: i∈[n1],j∈[n2]:
7. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The overlap-aware adaptive binary arithmetic coding algorithm specifically includes: Step i: transform the coefficient tensor C mt Each floating point value in is quantized to an integer value The coefficient value c i Round to the nearest integer a i ∈[0,9], the quantization interval is Δ; Step ii: rearrange the quantized coefficient tensor into an importance map p that records the position of the non-zero coefficients in the tensor t and the corresponding level map used to record its value Step iii, for level mapping, first use fixed-length binarization to encode it into a sequence of 0 and 1. For non-overlapping area pictures, directly use context-aware binary arithmetic coding (CABAC). For two overlapping area pictures at the same time, they are represented as the current view CurrView(x, y) and the reference view RefView(x', y'), respectively. Based on the pixel values and context features in the two views, the context index ctxIdx = f(CurrView(x, y), RefView(x', y'), D(x, y, x', y')) is obtained, where: D(x, y, x', y') is the absolute difference of pixels at corresponding positions of the two pictures, and f(·) is a quantization function used to map continuous values to a discrete context index space; Step iv: Update the binary coded probability model P(b=0|ctx)=εP(b=0|ctx) according to the context index CurrView )+(1-ε)P(b=0|ctx RefView ), where: P(b=1|ctx)=1-P(b=1|ctx), P(b=0|ctx CurrView ) is the probability based on the current view context, P(b=0|ctx RefView ) is the probability based on the transformed reference view context, and ε is a weighting factor that is adjusted according to the size response of D(x,y,x',y'); Step v: Perform arithmetic coding on the binary sequence of the level mapping according to the updated probability model.
8. The spatial situation awareness image compression method based on aggregated sparse coding according to claim 1 is characterized in that: The image reconstruction means that the ground station retrieves the corresponding background pixels according to the satellite sequence number m and the shooting time t And get the coefficient tensor C for decoding mt After that, the image is reconstructed using tensor convolution calculation Where: t = t mod T.
9. A space situational awareness image compression system implementing any of the methods described in claims 1-8, characterized in that: include: The overlapping area segmentation and allocation module, the background segmentation module, the sparse coding module and the image reconstruction module are as follows: the background segmentation module performs foreground and background segmentation based on the space situation awareness image captured and outputs the background area to the ground station; the overlapping area segmentation and allocation module calculates the pixel area belonging to the same field of view based on the pixel relationship of the foreground area in the image captured by two adjacent satellites at the same time after segmentation and allocates the corresponding area to the corresponding satellite for image compression; The sparse coding module performs aggregate sparse coding and overlap-aware adaptive binary arithmetic coefficient coding according to the received image area, further reducing the dictionary and coefficient size of the sparse coding output; the image reconstruction module performs the final image reconstruction based on the encoded dictionary and coefficients in conjunction with the background part; The allocation means that the overlapping pixel areas under the same field of view are allocated to the odd-numbered satellites within the adjacent one-hop range for subsequent image compression, and the remaining non-overlapping pixel areas are allocated to the even-numbered satellites within the adjacent one-hop range for subsequent image compression. The corresponding areas are determined based on whether the serial number of the local satellite is odd or even, and are directly involved in subsequent calculations on the local satellite without additional allocation.
10. The space situational awareness image compression system according to claim 9, characterized in that: The sparse coding module includes: an aggregated sparse coding unit and an overlap-aware adaptive binary coefficient arithmetic coding unit, wherein: the dictionary aggregation unit aggregates dictionary tensors in overlapping areas to represent common basic elements in the image, thereby further reducing the size of the dictionary obtained by sparse coding; the overlap-aware adaptive binary coefficient arithmetic coding unit uses the sparsity of the coefficient matrix obtained by sparse coding to apply geometric transformation on the overlapping area to predict the probability model, encode the coefficient tensor into a more compact string, thereby further reducing the size of the coefficient obtained by sparse coding; The background segmentation module includes: a background segmentation unit and a background uploading unit, wherein: the background segmentation unit uses the SCS-LBP operator to perform background and foreground segmentation, and the background uploading unit outputs the obtained background area to the ground station for image reconstruction; The image reconstruction module includes: a background query unit and a reconstruction calculation unit, wherein: the background query unit retrieves the corresponding background pixels according to the satellite sequence number and the shooting time and obtains the coefficient tensor for decoding, and the reconstruction calculation unit uses the background image, the dictionary tensor and the decoded coefficient tensor to perform tensor convolution calculation to reconstruct the image.
Citation Information
Patent Citations
Video compressed sensing and reconstruction method and device based on deep neural network
CN110933429A
Satellite Internet of Things image compression method and device
CN115348455A
Medical image fusion method based on shared multi-dimensional component tensor dictionary learning
CN117934305A