A preprocessing algorithm for remote sensing image coding

Abstract

The application discloses a kind of pre-processing algorithms for remote sensing image coding.The algorithm is mainly to extract image texture feature parameters, extract image texture feature parameters, according to the texture feature parameter bypasses the prediction mode and the division mode with smaller probability.The algorithm mainly includes image texture feature extraction;Fast rate distortion cost estimation;Intra-frame division prediction;Determine candidate prediction mode and division set.The present application, especially for hardware difficult to realize the H.265 encoding algorithm of remote sensing image.The algorithm saves hardware resources, reduces hardware implementation complexity and difficulty under the premise of trying to guarantee image compression quality, effectively utilizes hardware storage resources, and also reduces the time delay of compression and decoding.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a preprocessing algorithm for remote sensing image coding and decoding. BACKGROUND

[0002] With the rapid development of computers and the Internet, multimedia data communication technology with image and video as important forms has rapidly risen, and simple text and voice communication forms no longer meet people's daily needs. Multimedia communication is loved by all walks of life and is widely used in remote education, remote conference, video phone, security monitoring and other fields, and changes our way of life, study and work. The latest digital video compression coding standard HEVC adopts many higher performance compression coding technologies, which not only improves the coding performance, but also leads to a sharp increase in operation complexity and a significant increase in coding time. In terms of software, the size of the HEVC coding block supports 64*64, 32*32, 16*16, 8*8 and 4*4, and each coding block has 35 intra-prediction directions. The method of HEVC to determine the optimal partition block and prediction mode is to traverse all combinations and use rate-distortion optimization to obtain the best combination method. The combination of HEVC to determine the optimal block partition mode and prediction mode can indeed obtain high compression ratio and image quality, but the complexity and resource requirements are also very high, and these operations also directly lead to an increase in time delay. In terms of hardware, it has very high requirements for hardware structure and resources, and it is difficult to implement in practical applications.

[0003] Obviously, for high-resolution remote sensing images, directly using the reference software of the H.265 video coding standard to realize compression coding means extremely high computational complexity and large storage resource consumption, which further limits its application on hardware platforms with limited computing and storage resources.

[0004] The H.265 video coding standard brings significant compression efficiency improvement of image / video data, but also means higher computational complexity and longer processing time delay. SUMMARY

[0005] The preprocessing algorithm for remote sensing image coding and decoding of the present application uses the characteristics of remote sensing images and hyperspectral data, and simplifies the traversal combination of the coding intra-prediction mode and the block partition mode of H.265 by extracting image feature parameters through the algorithm; the method reduces the hardware implementation resources, reduces the difficulty of hardware implementation, and shortens the coding time delay.

[0006] To achieve the above purpose, the preprocessing algorithm for remote sensing image coding proposed by the present application is characterized by processing based on the tree structure unit CTU of H.265, including the following steps:

[0007] S1, image texture feature extraction;

[0008] S2, fast rate-distortion cost estimation;

[0009] S3, intra-frame partition prediction;

[0010] S4, determining candidate prediction modes and partition sets, and simplifying intra-frame encoding;

[0011] wherein the size of the CTU is NxN, each CTU is divided into coding units (CU) in the form of a quadtree, each CU is divided into prediction units (PU) in the form of a quadtree, and each PU is divided into transform units (TU) in the form of a quadtree.

[0012] Preferably, step S1 comprises the following specific steps: analyzing the directionality of the image texture and the strength of the image texture to extract a feature parameter reflecting the image texture characteristics; and extracting K prediction modes of the image block, K <= 4. The K prediction modes include the vertical direction (horizontal mode), the horizontal direction (horizontal mode), and the difference between the absolute values of the DC mode and the Planar mode (SAD), which are used as parameters reflecting the local texture characteristics of the image block.

[0013] Preferably, the absolute difference of the vertical direction is defined as follows:

[0014]

[0015] wherein n represents the side length of the current PU, P j,i represents the pixel value of the jth row and the ith column in the current PU; the SAD of the four n*n size PUs contained in the 2n*2n size PU Ver is superimposed, i.e.:

[0016]

[0017] wherein N represents the side length of the 2n*2n size PU;

[0018] The SAD of the horizontal direction is defined as follows:

[0019]

[0020] The SAD of the flow is used to reflect the texture flatness of the current block, and the calculation method is as follows:

[0021]

[0022] wherein Avg block represents the average of the first row and the first column of pixels of the current block, SAD DC represents the SAD of the difference between the pixel value and the average of the current block.

[0023] The calculation of SAD of the Planar mode is as follows:

[0024]

[0025] SAD = ∑ (x - x')2 plan SADn represents the SAD value of the n-th row of the combined prediction mode in the Planar mode, wherein SADn = ∑ (x - x')2 plan_m SADm represents the calculation result of the SAD value of the m-th row in the Planar mode.

[0026] Preferably, the specific step of step S2 is to establish a simplified rate-distortion cost function, and preliminarily obtain the division mode of the current prediction block according to the rate-distortion cost function; wherein the rate-distortion cost function is as follows:

[0027] Cost = coeff1 * min_SAD + coeff2

[0028] Wherein, min_SAD represents the distortion, and coeff1 and coeff2 represent the sizes of the two corresponding coefficients.

[0029] Preferably, the calculation method of min_SAD is to compare the sizes of the SADs calculated in K modes; the sizes of the SADs calculated in the vertical mode, the horizontal mode, the direct current mode and the Planar mode are calculated, that is, SAD ver , SAD Hor , SAD DC , and SAD Plan The minimum one is assigned to min_SAD, and the Best_mode is preliminarily obtained.

[0030] min_SAD = min (SAD ver , SAD Hor , SAD DC , SAD Plan )

[0031]

[0032] Wherein, Best_mode represents the optimal prediction mode selected, and Best_mode is one of the four prediction modes of ver, Hor, Dc and Plan, that is, the prediction mode with the minimum SAD.

[0033] Preferably, the selection of the CU, PU, TU and prediction mode is optimized for each CTU; each CTU is processed in the zig-zag scanning order.

[0034] Compared with the prior art, the present application has the beneficial effects that: an image preprocessing algorithm is performed on a remote sensing image, and then the coding and decoding complexity is reduced, the algorithm mainly utilizes the characteristics of the remote sensing image and the hyperspectral image, first performs block-based image texture feature parameter extraction, according to the extracted texture features and three prediction modes of vertical, horizontal and DC, determines the prediction candidate mode according to the rate-distortion cost function proposed in the present application; the present application greatly reduces the coding complexity of H.265 and reduces the video coding traversal combination under the condition of basic remote sensing image coding and decoding quality. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to the structures shown in the drawings without creative labor for those skilled in the art.

[0036] Figure 1 The algorithm step flowchart of the present application;

[0037] Figure 2 The improved H.265 coding framework of the algorithm of the present application;

[0038] Figure 3 The implementation block diagram of the algorithm of the present application on FPGA;

[0039] Figure 4 The intra prediction coding CU unit processing example;

[0040] Figure 5 The prediction mode for image texture feature extraction;

[0041] Figure 6 The selection schematic diagram of the optimal mode neighboring mode;

[0042] Figure 7 The flowchart of constructing the candidate prediction mode list in the preprocessing module.

[0043] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0044] The preprocessing algorithm for remote sensing image coding proposed in the present embodiment is characterized by processing based on the tree structure unit CTU of H.265, including the following steps as shown in Figure 1

[0045] S1, image texture feature extraction; ​

[0046] S2, fast rate-distortion cost estimation;

[0047] S3, intra-frame partition prediction;

[0048] S4. Determine candidate prediction modes and partition sets, and simplify intra-frame coding;

[0049] The size of a CTU is NxN. Each CTU is divided into coding units (CUs) using a quadtree. Each CU is divided into prediction units (PUs) using a quadtree, and each PU is divided into transform units (TUs) using a quadtree. The image texture feature extraction module calculates the texture complexity of the image on a block-by-block basis. Fast rate-distortion cost estimation uses a simplified rate-distortion cost function to estimate the rate-distortion cost under different partitions. Intra-frame partition prediction uses the rate-distortion cost to derive intra-frame partition predictions, ultimately determining candidate modes and partition sets for simplified intra-frame coding.

[0050] Furthermore, during intra-frame coding, the optimal coding mode often has a strong correlation with the texture distribution of the image, that is, the optimal mode is mainly affected by the flatness and directionality of the image texture. By analyzing the directionality and strength of the image texture, most candidate modes that are unlikely to be the optimal mode can be skipped during preprocessing. The basic idea is to extract feature parameters that reflect the texture characteristics of the image. K prediction modes are extracted for the image block, where K <= 4. The K prediction modes include the difference in absolute values ​​(SAD) of the vertical direction (horizontal mode), the horizontal direction (horizontal mode), and the direct current (DC mode) planar mode, and use them as parameters to reflect the local texture characteristics of the image block.

[0051] Furthermore, the difference in absolute values ​​in the vertical direction is defined as follows:

[0052]

[0053] Among them, n represents the side length of the current PU, P j,i Represents the pixel value of the jth row and ith column in the current PU; a 2n*2n PU can have the SAD of the four n*n PUs it contains Ver Superposition gives:

[0054]

[0055] Where N represents the side length of the PU of size 2n*2n;

[0056] The SAD in the horizontal direction is defined as follows:

[0057]

[0058] DC SAD is used to reflect the texture flatness of the current block, and its calculation method is as follows:

[0059]

[0060] Among them, Avg block Represents the mean of the first row and first column pixels of the current block, SAD DC SAD represents the difference between the pixel value of the current block and the mean;

[0061] The SAD of the planar mode is calculated as follows:

[0062]

[0063] Among them, SAD plan Indicates the SAD value when the prediction mode after n rows are merged is Plannar, where SAD plan_m represents the calculated SAD value for the mth row in Plannar mode. Pij represents the reference pixel in the i-th row and j-th column. However, this reference pixel is different from the reconstructed reference pixel used in the actual intra prediction process. In the preprocessing stage, to improve parallelization of module processing, we use the uncoded original pixel as the reference pixel. The rate-distortion cost function is obtained by table lookup based on the QP and block size.

[0064] Among them, for PUs of the same size, the rate-distortion values ​​under different modes are compared, and the prediction mode corresponding to the minimum rate-distortion value is selected as the optimal prediction mode under the current size of the PU.

[0065] Among them, for block division, a tree-shaped iterative method is used, that is, the sum of the minimum rate-distortion cost functions under the optimal mode of four 8x8 blocks is compared with the rate-distortion cost function under the optimal mode of a 16x16 block, and the smaller one is selected as the current optimal division mode. By analogy again, the optimal division selection of four 32x32 blocks and one 64x64 block is finally obtained.

[0066] Among them, the candidate modes and partitions are determined. The optimal partition is determined by using the best partition mode obtained above as the optimal partition mode for intra-frame prediction. The candidate prediction modes are determined by selecting the best prediction mode Best_mode obtained based on the simplified rate distortion and selecting the M nearest prediction modes, where M=3 in this embodiment.

[0067] Among them, the relationship and size between CU, PU and TU are determined as follows: the maximum size of CU is 64x64 and the minimum size is 8x8; the size of PU is maximum 32x32 and minimum 8*8; TU and PU are equal in size.

[0068] In the embodiment, the intra entropy coding adopts a CABAC coding mode.

[0069] Further, the specific step of step S2 is to establish a simplified rate-distortion cost function, and preliminarily obtain the partition mode of the current prediction block according to the rate-distortion cost function; wherein the rate-distortion cost function is as follows:

[0070] Cost = coeff1 * min_SAD + coeff2

[0071] Wherein, min_SAD represents the distortion, and coeff1 and coeff2 represent the sizes of the two corresponding coefficients.

[0072] Wherein, coeff1, coeff2 and the quantization parameter QP are related to the size of the current prediction block. They are obtained by table lookup, as shown in the following table:

[0073]

[0074]

[0075] Further, the calculation method of min_SAD is to compare the sizes of SADs calculated under K modes; the sizes of SADs calculated under vertical mode, horizontal mode, direct current mode and Planar mode, i.e. SAD ver , SAD Hor , SAD DC , SAD Plan The smallest one is assigned to min_SAD, and the Best_mode is preliminarily obtained.

[0076] min_SAD = min (SAD ver , SAD Hor , SAD DC , SAD Plan )

[0077]

[0078] Wherein, Best_mode represents the optimal prediction mode selected, and Best_mode is one of the four prediction modes of ver, Hor, Dc and Plan, i.e. the prediction mode with the smallest SAD.

[0079] Further, the selection of CU, PU, TU and prediction mode is optimized for each CTU; each CTU is processed in zig-zag scanning order.

[0080] Among them, the embodiment of the present application can be applied to remote sensing image data compression, mainly aiming at the compression and coding of high-resolution remote sensing images, remote sensing videos and hyperspectral data, and the optimization design of hardware implementation based on the existing H.265 video coding standard. The main idea is to analyze the content characteristics of high-resolution remote sensing images, remote sensing videos and hyperspectral data, and to redesign and optimize the parts with limited performance impact but high computational complexity, or not conducive to parallel implementation in the original H.265 video encoder framework, and then meet the application requirements and expected characteristics on the target FPGA platform.

[0081] As shown in Figure 2 , the algorithm of the present application is generally placed before the intra-frame coding in the H.265 framework, that is, in the pre-processing part before coding. Since this module is located before coding, the reference pixels used for extracting image texture feature parameters are the original pixels.

[0082] As shown in Figure 3 , it is the FPGA implementation block diagram of the present application. Among them, the peripheral part mainly includes the input of image / video source, in this embodiment, the video resolution is 2080x1980, including DDR unit and CPU unit, mainly for storage and coding control and parameter configuration. Among them, the encoding layer, namely encoder_top, mainly includes the video interface module, namely the DMA unit, mainly to complete the read-write control arbitration of DDR. Among them, the Encode module is the core module, which is also the module where the algorithm is located. This module includes all basic processes of intra-frame coding, that is, the intra-frame prediction module (invention algorithm) transform / quantization module, inverse quantization / inverse transform unit, entropy coding module, and bitstream output interface unit, as well as the image preprocessing module of the present application, that is, the algorithm module of the present application.

[0083] Among them, in the image preprocessing, the texture feature parameters are extracted, and the division method is determined according to the rate-distortion cost function. As shown in Figure 4 , taking CU=32x32 as an example. That is, first calculate the SAD of the four 8x8 blocks with serial numbers 1, 2, 5 and 6 in parallel, and then calculate the SAD of the four 8x8 blocks with serial numbers 3, 4, 7 and 8. At this time, by using the SAD of the four 8x8 blocks with serial numbers 1, 2, 3 and 4, the SAD of the corresponding 16x16 block can be obtained. With the help of QP and the obtained SAD, the rate-distortion cost of dividing the 16x16 block into four 8x8 blocks and the rate-distortion cost of not dividing the 16x16 block can be estimated, and then it is determined whether the 16x16 block needs to be divided. By using a similar method, the best division scheme of the current 32x32 size CU in the sense of rate-distortion optimization can be determined.

[0084] Among them, as shown in Figure 5As shown in the figure, image preprocessing extracts feature parameters. The HEVC hardware model only uses four intra-frame prediction directions in its implementation: DC mode, Planar mode, Ver mode, and Hor mode. These four modes are traversed and the optimal mode is selected for predictive coding based on the rate-distortion criterion.

[0085] The determination of the prediction candidate mode for the frame is to select the M nearest neighbor modes according to the optimal prediction mode selected in the preprocessing. Figure 6 As shown in the figure, the optimal mode of the example is the horizontal prediction mode, and the nearest neighbor mode is expanded to the left and right respectively.

[0086] like Figure 7 As shown in the figure, a flowchart for constructing a candidate set of prediction modes based on four metrics is provided. As can be seen from the above, before performing rate-distortion optimization for intra prediction, the CTU is first partitioned into blocks ranging from 8x8 to 64x64. Sad_ver, Sad_hor, Sad_dc, and Sad_plan are calculated for each partitioned region, from the 8x8 to the 64x64 layer. By comparing the numerical relationships between these three parameters, the texture characteristics of the local block can be quickly determined, thereby establishing a candidate set of optimal prediction modes for the current block under different partitioning conditions.

[0087] Finally, encoding is performed according to the intra-frame coding process, that is, quantization, transformation, and entropy coding are performed.

[0088] This embodiment utilizes video preprocessing statistics and video texture characteristics to simplify the HEVC intra-frame coding quadtree optimal block partitioning module and the intra-frame optimal prediction direction module. While ensuring high compression ratios and video quality, this significantly reduces computational complexity, latency, and implementation resources. This specifically addresses the difficulty of implementing the highly complex and resource-intensive HEVC on hardware platforms.

[0089] While the H.265 video coding standard significantly improves image and video data compression efficiency, it also implies higher computational complexity and longer processing latency. Considering the processing capabilities of the target hardware platform and the application requirements of this project, this paper primarily leverages the characteristics of remote sensing imagery and hyperspectral data, and proposes a preprocessing algorithm for remote sensing imagery. This algorithm extracts image feature parameters that simplify the traversal and combination of H.265 encoding intra-frame prediction modes and block partitioning methods. This approach reduces hardware implementation resources and eases hardware implementation difficulty, while also shortening encoding latency.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM) or the like.

[0091] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. A pre-processing algorithm for remote sensing image coding, characterized in that, The tree structure based on H.265 processes a CTU as a unit, including the following steps: S1, image texture feature extraction; S2, fast rate-distortion cost estimation; S3, intra-frame division prediction; S4, determining candidate prediction modes and division sets, and simplifying intra-frame encoding; Wherein the size of the CTU is NxN, each CTU is divided into coding units (CU) in the form of a quadtree, each CU is divided into prediction units (PU) in the form of a quadtree, and each PU is divided into transform units (TU) in the form of a quadtree; The specific steps of step S1 are: by analyzing the directionality of image texture and the strength of image texture, the characteristic parameters reflecting the image texture characteristics are extracted; K prediction modes of the image block are extracted, K<=4; the K prediction modes include vertical mode, horizontal mode, direct current mode and Planar mode; the absolute difference SAD under the K prediction modes is calculated and taken as a parameter reflecting the local texture characteristics of the image block; Wherein the absolute difference of the vertical direction is defined as follows: where n represents the side length of the current PU, P j,i represents the pixel value of the i-th column in the j-th row in the current PU; the absolute difference of the 2n*2n size PU can be obtained by the SAD of the four n*n size PUs contained in it Ver superposition, that is: Wherein, N represents the side length of the PU of 2n*2n size; The SAD of the horizontal direction is defined as follows: The direct current SAD is used to reflect the texture flatness of the current block, and the calculation method is as follows: wherein, Avg block represents the mean value of the first row and first column pixels of the current block, SAD DC represents the SAD of the difference between the pixel values and the mean value of the current block; The calculation of the SAD of the planar mode is as follows: where SAD plan represents the SAD value of n rows of merged prediction modes in the case of a planar mode, where SAD plan_m represents the calculation result of the SAD value of the mth row in the planar mode. The specific steps of step S2 are to establish a simplified rate-distortion cost function, and preliminarily obtain the division mode of the current prediction block according to the rate-distortion cost function; wherein the rate-distortion cost function is as follows: Cost=coeff1*min_SAD+coeff2 Wherein, min_SAD represents distortion, and coeff1 and coeff2 represent corresponding encoding prediction coefficients.

2. The pre-processing algorithm for remote sensing image coding according to claim 1, characterized in that, The calculation method of min_SAD is to compare the size of SAD calculated in K modes; calculate the size of SAD calculated in vertical mode, horizontal mode, DC mode and Planar mode, that is, SAD ver , SAD Hor , SAD DC and SAD Plan , the smallest one is assigned to min_SAD, and the preliminarily selected mode Best_mode is obtained; min_SAD = min(SAD ver ,SAD Hor ,SAD DC ,SAD Plan ) Best_mode=best_mode, and best_mode∈{Ver,Hor,DC,Plan}&&(SADver==min_SAD) Wherein, Best_mode represents the selected optimal prediction mode, and Best_mode is one of the four prediction modes of Ver, Hor, DC and Plan, that is, the prediction mode with the minimum SAD.

3. The pre-processing algorithm for remote sensing image coding according to claim 2, characterized in that, Optimize the selection of CU, PU, TU and prediction mode for each CTU; each CTU is processed in zigzag scanning order.

Images