A method, device and remote sensing satellite for compressing continuous frames of remote sensing images

Abstract

This invention discloses a method, apparatus, and remote sensing satellite for compressing consecutive frames of remote sensing images, belonging to the field of image processing technology. The compression method includes: dividing the remote sensing image to be compressed into non-overlapping image partitions; performing multi-threshold adaptive block division on each image partition; using the minimum absolute error criterion and a low-complexity two-layer search method to perform motion estimation on two consecutive frames of remote sensing images; finding the minimum absolute error sum corresponding to the optimal matching block; adaptively selecting intra-frame and inter-frame prediction based on the relationship between the minimum absolute error sum and a preset threshold; and finally performing compression according to standard subsequent steps, traversing all image partitions until the entire image is compressed. This invention partitions and blocks the remote sensing image, and designs a low-complexity inter-frame prediction method, which can ensure low-latency compression while also having good error resistance and high prediction performance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and more particularly relates to a continuous frame remote sensing image compression method and device and remote sensing satellite. BACKGROUND

[0002] In recent years, with the continuous breakthroughs in satellite remote sensing technology, earth observation satellites have become one of the important strategic resources of the country, and play an important role in both civilian and military fields. With the continuous development of scientific information technology, in order to obtain higher resolution and faster frame rate satellite observation information, satellite remote sensing image data volume presents an exponential explosive growth. Under the extreme on-board resource environment and limited space-ground transmission bandwidth, the unprocessed remote sensing image data will bring great challenges to the on-board memory and processor. Image compression is a key technology to solve this problem.

[0003] The on-board system computing resources and storage resources are scarce, and under the constraint of limited resources, it is difficult to use high-complexity lossless compression algorithms. At the same time, the working environment of the on-board image compression system is harsh, and in some harsh electromagnetic and radiation environments, high-energy particles in space can cause the logic state of the device to flip, which is prone to cause errors. The block compression method can prevent error propagation to some extent, but will cause the compression performance to decrease. The traditional compression method widely used in the current on-board system, such as intra-frame compression algorithm JPEG_LS, and the LOCO-3D inter-frame compression algorithm extended on this basis, all use fixed block method, which does not combine with image features and cannot have good error resistance and high compression performance. At the same time, JPEG-LS does not consider inter-frame redundancy, and the compression performance is low for high frame rate images, which is difficult to meet the high-fidelity and high-multiplying compression requirements of the coding and decoding task. LOCO-3D uses multiple reference frames for motion estimation and uses bilinear interpolation for pixel prediction, which contains multiple floating point operations, and is time-consuming and resource-consuming, which is difficult to meet the real-time requirements of the coding and decoding task. In summary, the existing on-board system is difficult to be compatible with low error rate and low prediction complexity when performing image compression. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the present application provides a continuous frame remote sensing image compression method, device and remote sensing satellite, which aims to partition and block the remote sensing image, and design a low-complexity inter-frame prediction method, which can ensure low-latency compression while having good error resistance and low prediction complexity. Thus, the technical problems of high error rate and high inter-frame prediction complexity when compressing the existing remote sensing image are solved.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a continuous frame remote sensing image compression method is provided, comprising:

[0006] S1: Divide the current frame of remote sensing image into multiple non-overlapping image partitions;

[0007] S2: Divide each of the image partitions into multiple non-overlapping image blocks;

[0008] S3: Using the absolute error and minimum criterion, the motion estimation is performed on the current image block in the current frame remote sensing image as a unit, taking the previous frame as the reference frame, to find the optimal matching block. The absolute error and SAD of each pixel in the adjacent area of ​​the optimal matching block are calculated, and the minimum SAD is found.

[0009] S4: Based on the relationship between the minimum SAD and the threshold, select one of inter-frame prediction, intra-frame prediction, and inter-intra-frame prediction to calculate the pixel prediction value of each point in the current image block;

[0010] S5: Compress the current image block using the pixel prediction values ​​of each point in the current image block;

[0011] S6: Sequentially traverse to the next image block, and execute steps S3 to S5 until the compression of the image partition corresponding to the current image block is completed;

[0012] S7: Sequentially traverse to the next image partition and execute steps S2-S6 until the compression of the current frame remote sensing image is completed.

[0013] In one embodiment, S2 includes performing the following operations for each image partition:

[0014] S21: Divide the current image partition Y into multiple non-overlapping initial blocks X. i ;

[0015] S22: Calculate each of the initial blocks X i The gradient of each pixel;

[0016] S23: Set each of the initial blocks X i The sum of the gradients of all pixels in the initial block X is used as the initial block X. i Image complexity K i ;

[0017] S24: Utilize all initial blocks X within the current image partition Y. i Image complexity K i The mean μ and standard deviation σ are used to set the first block threshold T1 and the second block threshold T2, and the minimum size threshold is set to M. min ×N min ;

[0018] S25: According to each of the initial blocks X i Image complexity Ki and the relationship between the size MxN of each of the initial blocks X i and M min xN min is partitioned, and the partition principle is as follows:

[0019] A1: if M≤M min or N≤N min , stop partitioning and take the initial block X i as the final image block, otherwise execute A2;

[0020] A2: if K i ≤T1, stop partitioning and take the initial block X i as the final image block; if T1 i ≤T2, divide the initial block X i again into a plurality of non-overlapping blocks as the final image block; if T2 i , take the initial block as the current image partition and execute S21-S25.

[0021] In one embodiment, the S22 comprises:

[0022] calculating the gradient of the pixel point with coordinates (x, y) by the formula K(x, y) = |G1(x, y)| + |G2(x, y)| + |G3(x, y)| + |G4(x, y)|; wherein I(x, y) is the pixel value of the pixel point with coordinates (x, y), G1(x, y), G2(x, y), G3(x, y), and G4(x, y) are the gradients of the pixel point (x, y) in the horizontal, vertical, 45°, and 135° directions;

[0023] G1(x, y) = I(x+1, y-1) + 2xI(x+1, y) + I(x+1, y+1)

[0024] - I(x-1, y-1) - 2xI(x-1, y) - I(x-1, y+1);

[0025] G2(x, y) = I(x-1, y+1) + 2xI(x, y+1) + I(x+1, y+1)

[0026] - I(x-1, y-1) - 2xI(x, y-1) - I(x+1, y-1);

[0027] G3(x, y) = I(x+1, y) + 2xI(x+1, y+1) + I(x, y+1)

[0028] - I(x - 1, y) - 2 x I(x - 1, y - 1) - I(x, y - 1);

[0029] G4(x, y) = I(x - 1, y) + 2 x I(x - 1, y + 1) + I(x, y + 1)

[0030] - I(x, y - 1) - 2 x I(x + 1, y - 1) - I(x + 1, y).

[0031] In one of the embodiments, the S24 comprises:

[0032] calculating the mean value μ and the standard deviation σ of the image complexity of all initial blocks X i in the current image partition Y; P is the number of all initial blocks in the current image partition Y; K i is the gradient of the i-th initial block X i with the size of M x N,

[0033] taking the mean value μ of the image complexity of all initial blocks X i in the current image partition Y as the first partition threshold T1, and taking μ + σ as the second partition threshold T2.

[0034] In one of the embodiments, the S3 comprises: for the current image block corresponding to the same position of the current frame remote sensing image and the previous frame remote sensing image:

[0035] S31: performing motion estimation in the current image block from the search starting point with a preset search step size; calculating the SAD of the block matching starting point (x, y) corresponding to each motion estimation by using the formula M and N represent the row and column values of the image block corresponding to the pixel point (x, y), I z (x, y) is the pixel value of the coordinate (x, y) in the current frame, I z-1 (x + i, y + j) is the pixel value of the coordinate (x, y) in the previous frame, and i and j are the horizontal and vertical direction offsets, respectively;

[0036] S32: selecting the minimum SAD as the optimal matching point, taking the neighborhood of the optimal matching point as the center, calculating the SAD of multiple points in the neighborhood as the matching points, and finding the minimum SAD and the corresponding point.

[0037] In one of the embodiments, the S4 comprises:

[0038] if the minimum SAD is greater than the maximum threshold of SAD, selecting the intra-frame prediction to calculate the pixel prediction value of each point in the current image block;

[0039] If the minimum SAD is smaller than the SAD minimum threshold, then inter prediction is selected to calculate the pixel prediction value of each point in the current image block;

[0040] If the minimum SAD is between the SAD maximum threshold and the SAD minimum threshold, then intra-inter prediction is selected to calculate the pixel prediction value of each point in the current image block.

[0041] In one embodiment, if intra prediction is selected, then the pixel prediction value of each point in the current image block is calculated using

[0042] If inter prediction is selected, then the pixel prediction value of each point in the current image block is calculated using P x x z-1 If intra-inter prediction is selected, then the pixel prediction value of each point in the current image block is calculated using the formula

[0043]

[0044] wherein R a_z , R b_z and R c_z are the pixels at the left, top and top-left of the pixel point P x_z in the z-th frame, R a_z-1 , R b_z-1 and R c_z-1 are the pixels at the corresponding positions of P x_z-1 in the (z-1)-th frame after motion estimation, and their mean value is G1 z-1 , G2 z-1 , G3 z-1 , G4 z-1 are the gradients of P x_z-1 in the horizontal, vertical, 45° and 135° directions, and G min = min(|G1 z-1 |, |G2 z-1 |, |G3 z-1 |, |G4 z-1 |).

[0045] In one embodiment, the S5 comprises:

[0046] S51: adaptively adjust the context parameters according to the size of the image block, use the context parameters to compensate the pixel prediction value of each point in the current image block, calculate the residual quantization value Errval_q and the pixel reconstruction value R x of the pixel corresponding to the compensated pixel prediction value; and calculate the residual quantization value Errval_q and the pixel reconstruction value R​​​​​x Entropy encoding is performed on each pixel in the current image block;

[0047] S52: Traverse each pixel sequentially according to the Z scan until the compression of the current image block is completed to obtain the bitstream of the current image block;

[0048] S53: Output the first pixel FP, motion vector MV, block size, prediction method, and bitstream of the current image block.

[0049] According to another aspect of the present invention, a compression apparatus for continuous frame remote sensing images is provided, comprising:

[0050] The partitioning module is used to divide the current frame of remote sensing image into multiple non-overlapping image partitions;

[0051] The segmentation module is used to divide each of the image partitions into multiple non-overlapping image blocks;

[0052] The matching module is used to perform motion estimation on the current image patch in the current frame remote sensing image using the previous frame as a reference frame and the absolute error and minimum criterion to find the optimal matching patch. It calculates the absolute error and SAD of each pixel in the adjacent area of ​​the optimal matching patch and finds the minimum SAD.

[0053] The prediction module is used to select one of inter-frame prediction, intra-frame prediction and inter-intra-frame prediction to calculate the pixel prediction value of each point in the current image block based on the relationship between the minimum SAD and the threshold.

[0054] The compression module is used to compress the current image block using the pixel prediction values ​​of each point in the current image block; sequentially traverse to the next image block and transfer to the matching module until the compression of the image partition corresponding to the current image block is completed; sequentially traverse to the next image partition and transfer to the block segmentation module until the compression of the current frame remote sensing image is completed.

[0055] According to another aspect of the present invention, a remote sensing satellite is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the compression method described above.

[0056] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the compression method described above.

[0057] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0058] (1) The application provides a compression method for continuous frame remote sensing images, wherein the remote sensing image to be compressed is divided into image partitions which do not overlap with each other; each image partition is divided into blocks, motion estimation is performed on two continuous frames of remote sensing images by using the absolute error sum minimum criterion and a low complexity two-layer search method, the minimum absolute error sum corresponding to the optimal matching block pair is found out, and intra-frame and inter-frame prediction is adaptively selected according to the size relationship between the minimum absolute error sum and a preset threshold value; finally, compression is performed according to the standard subsequent link, and all the image partitions are traversed until the compression of the whole image is completed; the remote sensing image is divided into partitions and blocks, and a low complexity inter-frame prediction method is designed, so that low delay compression can be ensured, and good error code resistance and low prediction complexity are achieved.

[0059] (2) The scheme calculates the gradient of a pixel point with coordinates (x, y) by using K(x, y) = |G1(x, y)| + |G2(x, y)| + |G3(x, y)| + |G4(x, y)|; based on the property that the local gray value at the image edge contour is the most drastic change in gradient, the pixel gradient within a block is used as an index for measuring the texture complexity of an image, so that most of the image edge contours are contained in smaller size blocks, and the error code resistance of the key information at the image edge can be improved.

[0060] (3) The scheme considers that the traditional Sobel operator only has a good response to the vertical and horizontal edges, and the edge detection effect in other directions is not ideal. Considering that the remote sensing image is rich in details, a and template are added on this basis, and joint detection in four directions makes the image edge more sensitive.

[0061] (4) The scheme takes the mean value μ of the image complexity of all initial blocks X i in the current image partition Y as the first block threshold T1, and takes μ+σ as the second block threshold T2, which is a fixed threshold setting method, and the method fully combines the image features, so that the block accuracy is higher.

[0062] (5) The scheme takes the optimal matching point corresponding to the minimum SAD as the center, calculates the SAD of multiple points in the neighborhood of the center as matching points, finds out the minimum SAD and the corresponding point, and compared with the full search algorithm, the two-layer search method learns from the idea of the three-step search method, that is, coarse search first and fine search later, which greatly reduces the calculation amount; compared with the fast search algorithm, the two-layer search method considers the global starting position in the search area during coarse search, so it will not fall into a local optimal solution, and the matching accuracy is high. In short, a good compromise between matching accuracy and calculation amount is achieved, which is suitable for hardware implementation.

[0063] (6) The block matching criterion SAD in the motion estimation process represents the sum of absolute values of errors between pixels of the to-be-encoded block and the candidate block, which well measures the similarity of the two, that is, the block correlation. Considering the subsequent compression in units of blocks, SAD can be selected as the measurement index of the block inter-frame correlation, and the proportion of intra-frame and inter-frame information in prediction is weighed according to the size of SAD. The advantage is that the SAD value has been calculated in the motion estimation link, and can be directly used in the prediction link without introducing additional indexes to increase the complexity and delay. In addition, using the SAD value as the criterion for selecting the prediction calculation mode can effectively improve the prediction performance of the gaze remote sensing image sequence with different inter-frame correlations in different working modes.

[0064] (7) Compared with the JPEG-LS standard, the present scheme only has intra-frame prediction, that is, the first calculation mode, while the last two calculation modes introduce inter-frame information to improve the prediction performance and thus improve the compression ratio. Moreover, the two calculation modes have low complexity and are simple to implement in hardware.

[0065] (8) The present scheme adaptively adjusts the context parameter according to the size of the image block, compensates the pixel prediction value of each point in the current image block by using the context parameter, and can adaptively compress according to the size of the image block, so as to guarantee the compression precision. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 A schematic diagram of the compression method of the continuous frame remote sensing image provided by the embodiment 1 of the present application.

[0067] Figure 2a A flowchart of the compression method of the continuous frame remote sensing image provided by the embodiment 1 of the present application.

[0068] Figure 2b An adaptive blocking flowchart based on the Sobel operator provided by the embodiment 1 of the present application.

[0069] Figure 3 A four-direction Sobel operator gradient calculation template diagram provided by the embodiment 1 of the present application.

[0070] Figure 4 A low-complexity two-layer search method schematic diagram provided by the embodiment 1 of the present application.

[0071] Figure 5 A reference frame and a current frame context template diagram provided by the embodiment 1 of the present application.

[0072] Figure 6 A reference frame gradient detection template diagram provided by the embodiment 1 of the present application.

[0073] Figure 7 An adaptive blocking schematic diagram based on the Sobel operator provided by the embodiment 1 of the present application.

[0074] Figure 8 Fig. 1 shows the anti-error capability comparison analysis result chart of embodiment 1 of the present application and JPEG-LS and LOCO-3D. DETAILED DESCRIPTION

[0075] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0076] Embodiment 1

[0077] As shown in Figure 1 and Figure 2a , the present embodiment provides a compression method for continuous frame remote sensing images, comprising: S1: dividing a current frame remote sensing image into a plurality of non-overlapping image partitions. S2: performing block division on each image partition to obtain a plurality of non-overlapping image blocks. S3: using absolute error sum minimum criterion, taking the previous frame as a reference frame, performing motion estimation on the current image block in the current frame remote sensing image as a unit to find the optimal matching block, calculating the absolute error sum SAD of each pixel in the adjacent region of the optimal matching block, and finding the minimum SAD therefrom. S4: according to the relationship between the minimum SAD and the threshold, selecting one from inter-frame prediction, intra-frame prediction and inter-frame-intra-frame prediction to calculate the pixel prediction value of each point in the current image block. S5: using the pixel prediction value of each point in the current image block to compress the current image block. S6: sequentially traversing to the next image block, and performing steps S3-S5 until the compression of the image partition corresponding to the current image block is completed. S7: sequentially traversing to the next image partition, and performing steps S2-S6 until the compression of the current frame remote sensing image is completed.

[0078] Step S1. Dividing the remote sensing image to be compressed into non-overlapping image partitions. Dividing the remote sensing image Img with a size of DxH into non-overlapping image partitions ICR i , i = 1, 2,.. q, where q is the total number of image partitions, D and H are the row and column values of the image Img respectively, and L and W are the row and column values of the image partition ICR i , which are set as needed. The specific implementation process is as follows: according to the image partition row value L (0 < L < D) and the column value W (0 < W < H), the image Img is divided into a plurality of non-overlapping image partitions. When the image Img row value D cannot be divided by the image partition row value L, the last row of pixel values of the current image partition is used for padding. When the image Img column value H cannot be divided by the image partition column value W, the last column of pixel values of the current image partition is used for padding.

[0079] Preferred, such as Figure 2b As shown, S2 includes: performing the following operation for each image partition: S21: dividing the current image partition Y into multiple non-overlapping initial blocks X. i For example, the input image partition size Y = B L ×B W B L B W Let the row and column values ​​of block Y be the values ​​of the image block Y. This block can be divided into four non-overlapping image blocks X of size M×N. i Where M = B L / 2, N=B W / 2. S22: Calculate each initial block X i The gradient of each pixel in the initial block X. S23: Calculate the gradient of each pixel in the initial block X. i The sum of the gradients of all pixels in the initial block X is used as the initial block X. i Image complexity K i S24: Utilize all initial blocks X within the current image partition Y. i The mean μ and standard deviation σ of the image complexity are used to set the first block threshold T1 and the second block threshold T2, and the minimum size threshold is set to M. min ×N min S25: Based on each initial block X i Image complexity K i With the first block threshold T1, the second block threshold T2, and each initial block X i The dimensions M×N and M min ×N min The relationships between elements are used to partition the data, and the partitioning principles are as follows:

[0080] A1: If M≤M min or N≤N min Then stop block splitting and reset the initial block X. i As the final image block, A2 is executed instead.

[0081] A2: If K is satisfied i If the value is less than or equal to T1, then stop the block division and reset the initial block X. i This is the final image block. If T1 is satisfied... <K i If ≤T2, then the initial block X will be... i The image is then further divided into multiple non-overlapping blocks as the final image blocks. If T2 is satisfied... <K i If so, the initial block is treated as the current image partition and S21 to S25 are executed.

[0082] Based on the local gray value change of image edge is often the most intense, the fastest nature, with the block pixel gradient and as the index of measuring image texture complexity, the method makes the image edge profile contained in smaller size block, targeted to improve the key information error code resistance ability at image edge.

[0083] Preferably, S22 comprises: using the formula:

[0084] K(x, y) = |G1(x, y)| + |G2(x, y)| + |G3(x, y)| + |G4(x, y)|

[0085] The gradient of the pixel point with coordinates (x, y) is calculated. Wherein, I(x, y) is the pixel value of the pixel point with coordinates (x, y), the four-direction Sobel operator gradient calculation template is as shown in Figure 3 , G1(x, y), G2(x, y), G3(x, y), G4(x, y) are the horizontal, vertical, 45° and 135° direction gradients of the pixel point (x, y).

[0086] G1(x, y) = I(x+1, y-1) + 2xI(x+1, y) + I(x+1, y+1)

[0087] -I(x-1, y-1)-2xI(x-1, y)-I(x-1, y+1).

[0088] G2(x, y) = I(x-1, y+1) + 2xI(x, y+1) + I(x+1, y+1)

[0089] -I(x-1, y-1)-2xI(x, y-1)-I(x+1, y-1).

[0090] G3(x, y) = I(x+1, y) + 2xI(x+1, y+1) + I(x, y+1)

[0091] -I(x-1, y)-2xI(x-1, y-1)-I(x, y-1).

[0092] G4(x, y) = I(x-1, y) + 2xI(x-1, y+1) + I(x, y+1)

[0093] -I(x, y-1)-2xI(x+1, y-1)-I(x+1, y).

[0094] Preferably, S24 comprises: calculating the mean value μ and the standard deviation σ of the image complexity of all initial blocks X i in the current image partition Y. P is the number of all initial blocks in the current image partition Y. K iLet X be the i-th initial block of size M×N. i gradient, All initial blocks X within the current image partition Y. i The mean value of the image complexity μ is used as the first block threshold T1, and μ+σ is used as the second block threshold T2.

[0095] Preferably, the image patch obtained in step S2 is processed using the following method: Figure 4 The two-layer search method shown is used for the search. S3 includes: for the current image patch corresponding to the same position in the current frame remote sensing image and the previous frame remote sensing image: S31: within the current image patch, motion estimation is performed starting from the search starting point with a preset search step size. Using the formula... Calculate the SAD of the block matching starting point (x, y) corresponding to each motion estimate. M and N represent the row and column values ​​of the image block corresponding to pixel (x, y), and I... z (x,y) represents the pixel value at coordinates (x,y) in the current frame. z-1 (x+i,y+j) represents the pixel value at coordinates (x,y) in the previous frame, where i and j are the horizontal and vertical offsets, respectively. S32: Select the point with the smallest SAD as the optimal matching point. Using the optimal matching point as the center, calculate the SAD of multiple points in its neighborhood as matching points, and find the point with the smallest SAD and its corresponding point.

[0096] Specifically, step S3 includes the following sub-steps:

[0097] S31: Coarse-grained search. The search starting point is set to (1,1). Within the search area, with a search step size of 2, the SAD (Solution Aspect Ratio) is calculated, with every other point serving as the starting point for block matching. The point with the minimum SAD value is selected as the optimal matching point for the coarse-grained search. M and N represent the row and column values ​​of the block, I... z (x,y) represents the pixel value at coordinates (x,y) in the z-th frame. z-1 (x+i, y+j) represents the pixel value at the corresponding coordinates of the reference frame, where i and j are the horizontal and vertical offsets, respectively. The SAD calculation formula is as follows:

[0098] S32: Fine-grained search. Take the optimal matching point in step S31 as the center, and take 8 points in its neighborhood as matching points to calculate SAD, and take the point with the minimum value of the 9 SADs as the final optimal matching point to obtain the motion vector MV and the SAD value. For example, if the image block size is 16x16 and the search range is [-4, +4], the search area size is 24x24, the full search method needs to calculate 81 SAD values, while the coarse-grained search link of the two-layer search method needs to calculate 16 SAD values, and the fine-grained search needs to calculate 8 SAD values, a total of only 24 SAD values, the calculation amount is reduced by 70.37%, which is suitable for acceleration on an embedded platform with limited on-board resources. Moreover, this method considers the global starting position in the search area, rather than only selecting 8 points around the center for matching calculation, so it will not fall into a local optimal solution.

[0099] Preferably, the reference frame and the current frame context template are as shown in the figure. Figure 5 The best matching block obtained in step 3 and the current block are subjected to pixel prediction based on the reference frame and the current frame context template as shown in the figure.

[0100] S4 includes: if the minimum SAD is greater than the maximum SAD threshold, selecting intra-frame prediction to calculate the pixel prediction value of each point in the current image block; if the minimum SAD is less than the minimum SAD threshold, selecting inter-frame prediction to calculate the pixel prediction value of each point in the current image block; and if the minimum SAD is between the maximum SAD threshold and the minimum SAD threshold, selecting intra-frame-inter-frame prediction to calculate the pixel prediction value of each point in the current image block.

[0101] Step S2 divides smaller blocks at the edge contour to improve the error code resistance of the edge contour, but at the cost of the compression performance. In order to make up for the loss of compression performance, an adaptive context number reduction block compression method is proposed.

[0102] Specifically, when the image block size is 64x64, the quantization and mapping calculation formula is as follows:

[0103]

[0104] Q=|81xQ1+9xQ2+Q3|;

[0105]

[0106] When the image block size is 32x32, the gradient quantization and mapping formula is as follows, wherein the gradient quantization branch is reduced from 9 to 7, and the context number is reduced from 365 to 172.

[0107]

[0108] Q=|49xQ1+7xQ2+Q3|;

[0109]

[0110] When the image patch size is 16×16, the gradient quantization and mapping formulas are as follows, where the gradient quantization branches are reduced from 9 to 5, and the number of contexts is reduced from 365 to 63.

[0111]

[0112]

[0113] The JPEG-LS standard provides 365 context addresses, suitable for large-format image compression. However, when image blocks are small, such as 16×16 blocks containing only 256 pixels but with 365 addresses, the large number of context addresses (i.e., more classifications) inevitably leads to fewer observed pixels with the same context address. This results in insufficient samples to support adaptive context updates, leading to poor prediction correction and compensation, and decreased prediction performance. Therefore, adaptive adjustments are needed for gradient quantization and gradient mapping. In principle, for smaller blocks, the number of context addresses should be appropriately reduced.

[0114] S43: Based on the current frame pixel P x The left, top, and top-left three pixels R a_z R b_z and R c_z , and P x_z The pixel P at the corresponding position after motion estimation in frame z-1 x_z-1 The field pixel R a_z-1 R b_z-1 and R c_z-1 , and P x_z-1 Gradient G1 in four directions z-1 G2 z-1 G3 z-1 G4 z-1 Three prediction methods are designed. Based on the magnitude of SAD, prediction method X is selected, and adaptive prediction value calculation is performed.

[0115]

[0116] Preferably, if intra-frame prediction is selected, then utilize Calculate the predicted pixel values ​​for each point in the current image patch. If inter-frame prediction is selected, then use P... x =P x_z-1 Calculate the predicted pixel values ​​for each point in the current image patch. If intra-frame / inter-frame prediction is selected, use the formula... Calculate the predicted pixel value for each point in the current image patch. Where R... a_z R b_z and Rc_z P is the pixel in the z-th frame. x_z The pixels at the left, top, and top-left, R a_z-1 R b_z-1 and R c_z-1 For frame z-1, P x_z-1 The mean value of the pixel at the corresponding position after motion estimation. intermediate quantity G1 z-1 G2 z-1 G3 z-1 G4 z-1 For P x_z-1 Gradients G in the horizontal, vertical, 45° and 135° directions min =min(|G1 z-1 |、|G2 z-1 |、|G3 z-1 |、|G4 z-1 |).

[0117] In motion estimation, the block matching criterion SAD represents the sum of the absolute values ​​of errors between pixels in the block to be encoded and those in candidate blocks, effectively measuring their similarity, i.e., block correlation. Considering that subsequent compression is performed on a block-by-block basis, SAD can be chosen as a measure of inter-frame correlation, using its magnitude to balance the proportion of intra-frame and inter-frame information in prediction. The advantage of this is that the SAD value has already been calculated in the motion estimation stage and can be directly used in the prediction stage, eliminating the need to introduce additional metrics that would increase complexity and latency. Furthermore, since the gradients of each pixel in four directions have already been calculated using the Sobel operator during block segmentation, these can be used as gradient detection results during prediction, again avoiding the need to introduce additional metrics that would increase complexity and latency. Additionally, considering that the neighborhood of the pixel to be reconstructed in the current frame cannot be predicted in advance during decoding, the P value in the reference frame is chosen as the reference value. x_z-1 The gradient is used as the basis for predictor branch selection. This is based on the premise that the inter-frame correlation of the research object is strong, and P can be approximated as... x and P x_z-1 The gradient directions are consistent.

[0118] For example, when the block size is 16×16, max_SAD is 25000, and min_SAD is 3000, if the SAD value of a certain block is 18000, then Prediction3 is used for prediction. Current frame pixel P x Pixel P in the corresponding reference frame x_z-1 ,like Figure 6 As shown, the values ​​of the 8 pixels in the surrounding area are: R a_z-1 =0, R b_z-1 =10, R c_z-1= 10, R d_z-1 = 10, R e_z-1 = 0, R f_z-1 = 0, R g_z-1 = 0, R h_z-1 = 0, G1 z-1 = 0, G2 z-1 = -40, G3 z-1 = -30, G4 z-1 = -30, i.e. at the pixel point P x_z-1 has a gradient in the vertical direction, the vertical direction changes rapidly, so it has a strong boundary in the horizontal direction, and the pixel similarity along the boundary direction is higher. At this time, G min = |G1 z-1 |, predictor 1 is selected, and the pixel value R a_z , R a_z-1 on the left side in the horizontal direction is used for prediction.

[0119] The pixel that performs the optimal prediction on the pixel P x should be the pixel corresponding to the minimum gradient direction. Gradient detection is performed on the corresponding pixel field in the reference frame based on the Sobel operator, and the optimal prediction branch of the pixel to be predicted is obtained by comparing the gradient size of the mode gradient of the point gradient in different prediction directions.

[0120] Preferably, S5 comprises: S51: calculating the residual quantization value Errval_q and the pixel reconstruction value R x of the corresponding pixel by using the pixel prediction value of each point in the current image block. According to the residual quantization value Errval_q and the pixel reconstruction value R x , entropy encoding is performed on each pixel in the current image block. S52: each pixel is sequentially traversed according to Z scanning until the compression of the current image block is completed to obtain the code stream of the current image block. S53: the first pixel FP, the motion vector MV, the block size, the prediction mode and the code stream of the current image block are jointly output.

[0121] Step S5 comprises the following sub-steps:

[0122] S51: calculating the residual quantization value Errval_q and the pixel reconstruction value R x of the corresponding pixel according to the prediction value P x .

[0123] S511: compensating the prediction value P x according to the context parameter C[Q] and Sign to obtain the compensated prediction value P’ x , the pixel bit width is bp, and the maximum value that can be reached by the pixel is MaxVal, and the calculation formula is as follows:

[0124] P’ x = min((Px + Sign x C[Q], MaxVal);

[0125] MaxVal = 2 bp - 1;

[0126] S512: According to the pixel actual value I x , and the compensated prediction value P' x , combined with the sign Sign, the prediction residual value Errval is calculated, and the calculation formula is as follows:

[0127] Errval = Sign x (I x - P' x );

[0128] S513: According to the obtained prediction residual value Errval and the micro-loss degree parameter Near, combined with the sign Sign, the residual quantization value Errval_q and the pixel reconstruction value R x are calculated, and the calculation formula is as follows:

[0129]

[0130] R x = P x + Sign x Errval_q x (2 x Near + 1);

[0131] If R x < 0, then R x = 0; if R x > MaxVal, then R x = MaxVal.

[0132] S514: According to the residual quantization value Errval_q, the context parameters A[Q], B[Q], C[Q], N[Q] are updated, A[Q] is the sum of residual absolute values, B[Q] is the sum of residuals, C[Q] is the adaptive correction value, and N[Q] is the number of context occurrences.

[0133] Specifically, step S514 is specifically:

[0134] (1) A[Q], B[Q], N[Q] parameter update, the calculation formula is as follows:

[0135] A[Q] = A[Q] + abs(Errval_q);

[0136] B[Q] = B[Q] + Errval_q x (2 x Near + 1);

[0137] If N[Q] = RESET, where RESET is the threshold value when A[Q], B[Q], N[Q] will be half of the original, the calculation formula is as follows:

[0138]

[0139]

[0140]

[0141] N[Q] = N[Q] + 1.

[0142] (2) C[Q] parameter update: B[Q] will trigger the update of C[Q], and limit C[Q] in [MIN_C, MAX_C], the calculation formula is as follows:

[0143]

[0144] The update process of C[Q] updates B[Q] at the same time, the calculation formula is as follows:

[0145]

[0146] When B[Q] ≤ -N[Q], if the condition still holds after updating, then B[Q] = -N[Q] + 1; when B[Q] > 0, if the condition still holds after updating, then B[Q] = 0.

[0147] S52: According to the residual quantization value Errval_q and the pixel reconstruction value R x , entropy encoding is performed for each pixel of the block:

[0148] S521: According to the maximum value MaxVal and the micro-loss parameter Near, the residual modulo parameter RANGE is obtained, and further, the residual modulo value Errval_mod is obtained, the calculation formula is as follows:

[0149]

[0150]

[0151] S522: According to the sum of residual absolute values A[Q] and the adaptive correction value C[Q], the Golomb coding variable k is calculated, the calculation formula is as follows:

[0152] S523: According to the modulo residual value Errval_mod, non-negative mapping is performed to obtain the non-negative integer residual mapping value MErrval, the calculation formula is as follows:

[0153] If Near = 0, k = 0, and 2B[Q] ≤ -N[Q], then:

[0154]

[0155] If Near = 0, k = 0, and 2B[Q] ≤ -N[Q] then:

[0156]

[0157] S524: According to the coding variable k, the residual modulo parameter RANGE and the residual mapping value MErrval, the parameters VAL, n and the pixel quantization bit width qbpp and the coding limit length LIMIT required by Golomb limited length coding are calculated, and the final length limit is LMAX, and the calculation formula is as follows:

[0158]

[0159]

[0160] S525: According to VAL, n, the current pixel is Golomb limited length coded to obtain the code stream.

[0161] S53: According to Z scanning, each pixel is sequentially traversed, and steps S51-S52 are repeated until the compression of the image block is completed, and the first pixel FP, the motion vector MV, the block size MxN and the prediction mode number X of the current block are output together with the code stream of the current block.

[0162] Figure 7 The gaze remote sensing image with a size of 256x256 is divided into blocks according to the above method, and the results are shown in the figure. It can be seen that this method can effectively separate the complex region and the flat region, and realize the effect that the block of the complex region is smaller and the block of the flat region is larger. In the image partition, due to the large gradient difference between the pixel points located in the edge contour region and the pixel points located in the flat region, the block containing the edge region with a relatively larger K value is divided into smaller blocks. It can be seen from the observation of the image that most of the edge contours of the image are contained in smaller size blocks, such as the middle part of data1, the upper part of data2, and the upper left corner of data3. The smaller the block size is, the stronger the error code resistance is. The adaptive block splitting method based on Sobel gradient detection can improve the error code resistance of the edge contour region of the image.

[0163] In the compressed code stream, some part of the error code is injected, and after reconstruction at the decoding end, the anti-error diffusion results of the three algorithms of this method, JPEG-LS, LOCO-3D are as follows Figure 8The results show that the JPEG-LS does not have error resilience ability and large area error diffusion occurs; the LOCO-3D limits error diffusion within the 64x64 sub-block; and the AVBIC limits error diffusion within the smaller sub-block by dividing the edge into smaller sub-blocks. That is, under the premise of equivalent compression performance, the error resilience ability of the AVBIC at the edge profile is stronger than that of the LOCO-3D, which provides a reliable guarantee for subsequent scene recognition and image segmentation tasks on remote sensing images.

[0164] Taking eight groups of different test image sequences as experimental objects, setting the image partition size to be 128x128, the minimum sub-block size to be 16x16, and the micro-loss parameter Near to be 0, lossless compression is performed, and the compression performance of the three algorithms is analyzed, wherein the LOCO-3D selects three sub-block sizes of 128x128, 64x64 and 32x32. The results show that, compared with the JPEG-LS, the compression ratio of the AVBIC is increased by 37.48%; under the premise of equivalent compression performance, the AVBIC has stronger error resilience ability, lower computational complexity and lower processing delay, and has the potential to complete real-time encoding and decoding tasks of high-frame-rate remote sensing images.

[0165] Table 1: Compression performance comparison

[0166]

[0167]

[0168] Embodiment 2

[0169] The embodiment provides a compression device for continuous frame remote sensing images, which comprises a division module, a sub-block module, a matching module, a prediction module and a compression module. The division module is used for dividing a current frame remote sensing image into a plurality of non-overlapping image partitions. The sub-block module is used for dividing each image partition to obtain a plurality of non-overlapping image blocks. The matching module is used for using the absolute error sum criterion to perform motion estimation on a current image block in the current frame remote sensing image by taking a previous frame as a reference frame to find an optimal matching block, calculating the absolute error of each pixel in the adjacent region of the optimal matching block, and finding a minimum SAD from the SAD. The prediction module is used for calculating the pixel prediction value of each point in the current image block from the inter-frame prediction, the intra-frame prediction and the inter-frame-intra-frame prediction according to the relationship between the minimum SAD and a threshold. The compression module is used for compressing the current image block by using the pixel prediction value of each point in the current image block. Sequentially traversing to the next image block and transferring to the matching module until the compression of the image partition corresponding to the current image block is completed. Sequentially traversing to the next image partition and transferring to the sub-block module until the compression of the current frame remote sensing image is completed.

[0170] Embodiment 3

[0171] The embodiment provides a remote sensing satellite, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the compression method when executing the computer program.

[0172] Embodiment 4

[0173] The embodiment provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the compression method when executed by a processor.

[0174] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as object-oriented programming languages Java and direct script language JavaScript.

[0175] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0176] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0177] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable devices provide the function for implementing the processes specified in the flowchart Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or steps of the functions specified in the flowchart

[0178] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the spirit and scope of the application. Accordingly, it is intended that the present application embrace all such modifications and variations as fall within the scope of the application.

[0179] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the application can be practiced otherwise than as specifically described herein.

Claims

1. A method of compressing a continuous frame remote sensing image, characterized in that, Comprise: S1: the current frame remote sensing image is divided into a plurality of non-overlapping image partitions; S2: each of the image partitions is divided into a plurality of non-overlapping image blocks; S3: using the absolute error sum of squares minimum criterion, the last frame is used as a reference frame to perform motion estimation on the current image block of the current frame remote sensing image, and the optimal matching block is found out, the absolute error sum of squares of each pixel in the adjacent region of the optimal matching block is calculated, and the minimum SAD is found out from the minimum SAD; S4: according to the relationship between the minimum SAD and the threshold, the pixel prediction value of each point in the current image block is calculated from inter-frame prediction, intra-frame prediction and inter-frame-intra-frame prediction; S5: using the pixel prediction value of each point in the current image block to compress the current image block; S6: sequentially traversing to the next image block, performing step S3 S5 until the compression of the image partition corresponding to the current image block is completed; S7: sequentially traversing to the next image partition, and performing steps S2-S6 until the compression of the current frame remote sensing image is completed; S2 includes performing the following operation for each image partition: S21: Dividing the current image partition Y into multiple non-overlapping initial blocks. S22: Calculate each of the initial blocks S23: Gradient of each pixel in the initial block; The sum of the gradients of all pixels in the block is used as the initial block. Image complexity S24: Utilize all initial blocks within the current image partition Y Image complexity mean and standard deviation Set the first block threshold. Second block threshold And set the minimum size threshold to S25: According to each of the initial blocks Image complexity With the first block threshold Second block threshold and each of the aforementioned initial blocks The dimensions M×N and The relationships between the data points are used to partition the data into groups, with the following partitioning principles: A1: If the conditions are met... or Then stop dividing the blocks and put the initial block into place. As the final image block, execute A2; A2: if satisfied Then stop dividing the blocks and put the initial block into place. As the final image block; if it satisfies Then the initial block The image is further divided into multiple non-overlapping sub-blocks, and these sub-blocks are used as the final image blocks; if the following conditions are met... Then the initial block is regarded as the current image partition and S21~S25 are executed; The S3 comprises: for a current image block corresponding to a same position of the current frame remote sensing image and a previous frame remote sensing image: S31: performing motion estimation in a preset search step length from a search starting point in the current image block; using a formula to calculate a SAD of each motion estimation corresponding block matching starting point and representing row and column values of a pixel point corresponding to an image block, being a pixel value of a pixel with a coordinate of in the current frame, being a pixel value of a pixel with a coordinate of in the previous frame, and respectively being horizontal and vertical direction offset amounts; S32: selecting a minimum SAD as an optimal matching point, taking the optimal matching point as a center, calculating SADs of multiple points in a neighborhood of the optimal matching point as matching points, and finding a minimum SAD and a point corresponding to the minimum SAD.​ If the minimum SAD is greater than the maximum SAD threshold, the pixel prediction value of each point in the current image block is calculated by selecting intra-frame prediction; if the minimum SAD is less than the minimum SAD threshold, the pixel prediction value of each point in the current image block is calculated by selecting inter-frame prediction; if the minimum SAD is between the maximum SAD threshold and the minimum SAD threshold, the pixel prediction value of each point in the current image block is calculated by selecting inter-frame-intra-frame prediction; If intra-frame prediction is selected, then utilize... Calculate the midpoint of the current image patch The pixel prediction value; if inter-frame prediction is selected, then the pixel prediction value is used. Calculate the midpoint of the current image patch The pixel prediction value; if intra-frame / inter-frame prediction is selected, the formula is used. Calculate the midpoint of the current image patch Pixel prediction values; in, , and These are the pixels in the z-th frame. The pixels at the left, top, and top left. , and For the first frame The mean value of the pixel at the corresponding position after motion estimation. intermediate quantity , for In horizontal, vertical, and Gradient of direction, .

2. The compression method of the continuous frame remote sensing image according to claim 1, characterized in that, The S22 comprises: The gradient of a pixel point with coordinates (x, y) is calculated by the formula + + + wherein, x and y are integer values, and x, y ∈ [0, W-1] and [0, H-1] respectively, W and H are the width and height of the image respectively, and wherein, I(x, y) is the pixel value of the pixel point with coordinates (x, y), Gx(x, y) and Gy(x, y) are the gradients of the pixel point with coordinates (x, y) in horizontal and vertical directions respectively, and wherein, I(x, y) is the pixel value of ； ； ； 。 3. The compression method of the continuous frame remote sensing image according to claim 1, characterized in that, The S24 comprises: Calculate all initial blocks within the current image partition Y. The average image complexity and standard deviation ; , P represents the number of all initial blocks within the current image partition Y. For the i-th initial block of size M×N gradient, ; partitioning the current image into a plurality of blocks a mean value of image complexity of the initial blocks as the first block partitioning threshold as the second block partitioning threshold as the second block partitioning threshold .​ 4. The compression method of the continuous frame remote sensing image according to claim 1, wherein the S5 comprises: S51: adaptively adjusting a context parameter according to a size of the image block, using the context parameter to compensate a pixel prediction value of each point in the current image block, and calculating a residual quantization value of a pixel corresponding to the compensated pixel prediction value and a pixel reconstruction value ; entropy encoding each pixel in the current image block according to the residual quantization value and the pixel reconstruction value ; S52: traversing each pixel in turn according to Z scanning until the compression of the current image block is completed to obtain the code stream of the current image block; S53: the first pixel FP, motion vector MV, block size, prediction mode and code stream of the current image block are output together.

5. An apparatus for compressing a continuous frame remote sensing image, characterized in that, For executing the compression method of any one of claims 1-4, comprising: A division module is configured to divide the current frame remote sensing image into a plurality of non-overlapping image partitions; A block division module is configured to divide each of the image partitions into a plurality of non-overlapping image blocks; A matching module is configured to use the absolute error sum of squares minimum criterion, use the last frame as a reference frame to perform motion estimation on the current image block of the current frame remote sensing image, find out the optimal matching block, calculate the absolute error sum of squares of each pixel in the adjacent region of the optimal matching block, and find out the minimum SAD from the minimum SAD; A prediction module is configured to calculate the pixel prediction value of each point in the current image block from inter-frame prediction, intra-frame prediction and inter-frame-intra-frame prediction according to the relationship between the minimum SAD and the threshold; A compression module is configured to compress the current image block using the pixel prediction value of each point in the current image block; sequentially traverse to the next image block and enter the matching module until the compression of the image partition corresponding to the current image block is completed; sequentially traverse to the next image partition and enter the block division module until the compression of the current frame remote sensing image is completed.

6. A remote sensing satellite comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 4.

Images