Digital image encoders, decoders, processors and electronic devices

Abstract

This invention discloses a digital image encoder, decoder, processor, and electronic device, belonging to the field of image transmission technology. The encoder includes: a primary color calculation module, used to receive RGB888 image patches and divide them into two equal rectangular sub-blocks, and calculate the primary color calculation value β of channel k in sub-block i. ik The difference value calculation module is used to iterate through the original color value γ of channel k of pixel j in sub-block i. ijk And calculate the corresponding primary color value β ik By subtracting, we obtain the 9-bit color difference value Δ. ijk Brightness modulation calculation module, used to calculate brightness modulation value. The exponent calculation module is used to determine the shared exponent for all luminance modulation values, obtaining the shared index shift; the luminance modulation quantization module is used to determine each luminance modulation value. 4-bit mantissa δ ij The compressed encoding carries a 5-bit primary color storage value α. ik 4-bit mantissa δ ij And shared index shift. This invention improves the compression ratio of compressed coding with minimal image loss through a simple pipeline design.

Description

Digital image encoders, decoders, processors, and electronic devices Technical Field

[0001] This invention belongs to the field of image transmission technology, and more specifically, relates to a digital image encoder, decoder, processor, and electronic device. Background Technology

[0002] In recent years, digital imaging technology has developed rapidly, and the resolution of image sensors in embedded devices, handheld mobile devices, and specialized medical instruments has become increasingly higher. This has led to the exhaustion of image signal processor bus bandwidth and memory resources by the massive amounts of image data. To address these issues, image compression algorithms can be used to compress high-resolution image data.

[0003] Currently, there are many algorithms and system designs for compressing raw Bayer data acquired by image sensors. Most of these algorithms first convert the data to the YUV domain, then use discrete cosine transform to convert it to the frequency domain for low-pass filtering. Some algorithms also employ entropy coding for compression. However, these algorithms are still too complex and unsuitable for embedded devices or other devices with limited processing power. Other algorithms use texture compression to compress patches. While these algorithms are simple, their compression accuracy is low, resulting in poor image quality after patch compression. High-frequency details in the compressed image often exhibit jagged edges and blurred color blocks, leading to severe ghosting problems. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides a digital image encoder, decoder, processor and electronic device, the purpose of which is to compress the original image while ensuring the quality of image restoration through a simple compression algorithm.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a digital image encoder is provided for compressing and encoding digital images, comprising:

[0006] The primary color calculation module receives an RGB888 image patch and divides it into two equal sub-blocks. Each sub-block has an equal number of rows and columns of pixels. It calculates the average value of all pixels in sub-block i across channel k, and extracts the high 5 bits of the binary representation of the average value of channel k in sub-block i to obtain the 5-bit primary color storage value α. ik The 5-bit primary color value α is stored by padding the low bits. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik k = R, G, B, i = 0, 1;

[0007] The difference value calculation module is used to iterate through the original color value γ of channel k of pixel j in sub-block i. ijk And calculate the corresponding primary color value β ik The difference is calculated to obtain the corresponding signed 9-bit color difference value Δ. ijk j = 1, 2, 3, ..., N, where N is the number of pixels in each sub-block;

[0008] The luminance modulation calculation module is used to calculate the 9-bit luminance modulation value of pixel j in sub-block i.

[0009]

[0010] The exponent calculation module is used to determine the exponent of the luminance modulation value with the largest absolute value, which is converted into a floating-point number with the smallest loss and has a 4-bit mantissa, and to use it as the shared exponent for all luminance modulation values. The corresponding shared index is determined according to the mapping relationship between the exponent and the shared index shift; different shared indices shift are mapped one-to-one with different exponents.

[0011] The brightness modulation quantization module is used to quantize each brightness modulation value. The last 4 bits after right shifting the shared exponent are used as the 4-bit mantissa δ of the corresponding luminance modulation value. ij ;

[0012] The compression code carries a 5-bit primary color storage value α. ik 4-bit mantissa δ ij And shared index shift.

[0013] In one embodiment, it further includes;

[0014] The input preprocessing module is used to identify the format of the input tile: when the input tile is in Bayer format, it is converted to RGB888 format before being input into the primary color calculation module; when the input tile is in RGB888 format, it is directly input into the primary color calculation module.

[0015] The process of converting Bayer format to RGB888 format includes merging every two vertically adjacent Bayer pixels in every two adjacent rows of the Bayer tile into RGB888 standard pixels. The color values ​​of channels R and B remain unchanged before and after merging, and the color value of channel G after merging is the average of the two channels G in the Bayer pixels before merging.

[0016] In one embodiment, the RGB888 image blocks received by the primary color calculation module are H*2H or 2H*H images blocks, which are divided into H*H sub-blocks, where H=2.

[0017] In one embodiment, in the primary color calculation module, the 5-bit primary color storage value α is padded with low-order bits. ik Expanding to 8 bits includes: storing the 5-bit primary color value α ik The high 3 bits of data are appended to the 5-bit primary color storage value α. ik At the end, we obtain the 8-bit primary color calculation value β. ik .

[0018] In one embodiment, in the luminance modulation quantization module, the shared index shift is 2 bits. The exponent calculation module is used to determine the shared exponent that meets the conditions from four different exponents: 0, 2, 3, and 5. The exponent is an exponent with a base of 2. The four different values ​​of the 2-bit shared index shift are mapped one-to-one with the four different exponents.

[0019] The method by which the exponent calculation module determines the shared exponent includes:

[0020] Determine if the brightness modulation value with the largest absolute value is within the range of [-8, +7]. If so, select an exponent of 0.

[0021] If not, determine whether the brightness modulation value with the largest absolute value is within the range of [-32, +31]. If so, select an exponent of 2.

[0022] If not, determine whether the brightness modulation value with the largest absolute value is within the range of [-64, +63]. If so, select an exponent of 3.

[0023] If not, it means that the brightness modulation value with the largest absolute value is in the range of [-256, +255], and the exponent is selected as 5.

[0024] According to a second aspect of the present invention, a digital image decoder is provided for decoding the compressed code output by any of the above-mentioned claims to output a tile, the digital image decoder comprising:

[0025] The primary color bit extension module is used to extract the 5-bit primary color storage value α from the compressed encoding. ik The 5-bit primary color value α is stored using the same low-bit padding method as the primary color calculation module. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik ;

[0026] The luminance modulation module is used to extract the shared index shift in the compressed code and find the corresponding shared exponent through the mapping relationship, and to extract the 4-bit mantissa δ in the compressed code. ij And construct a floating-point number with the shared exponent as the signed 9-bit luminance modulation value of pixel j in sub-block i. Brightness modulation value Calculated value β of primary color ik Adding them together, we obtain the color restoration value γ' of channel k for pixel j in sub-block i. ijk The RGB888 tile was recovered.

[0027] In one embodiment, it further includes:

[0028] Output clamping module, used when color recovery value γ' ijk When the value exceeds the range of [0, 255], if the detected color recovery value is less than 0, its value is forcibly set to 0; if the detected color recovery value is greater than 255, its value is forcibly set to 255.

[0029] In one embodiment, it further includes:

[0030] The output format adjustment module is used to convert the recovered RGB888 tiles into Bayer tiles when Bayer format tiles are required, and to directly output the decoded RGB888 tiles when RGB888 format tiles are required.

[0031] The conversion process of the decoded RGB888 patch to Bayer patch includes: splitting each RGB888 standard pixel in the RGB888 patch into two Bayer pixels, wherein the color values ​​of channels R and B remain unchanged before and after the split, and channel G in the RGB888 standard pixel is split into two channels G in the Bayer pixel with the color values ​​remaining unchanged.

[0032] According to a third aspect of the present invention, a digital image processor is provided, comprising:

[0033] The aforementioned digital image encoder; and

[0034] The aforementioned digital image decoder.

[0035] According to a fourth aspect of the present invention, an electronic device is provided, comprising the digital image processor described above.

[0036] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0037] The digital image encoder proposed in this invention has a pipelined primary color calculation module, a difference value calculation module, a luminance modulation calculation module, an exponent calculation module, and a luminance modulation quantization module. Specifically, the primary color calculation module, the difference value calculation module, and the luminance modulation calculation module calculate the primary color and obtain the color difference value between each pixel and the primary color of the corresponding sub-block. Then, the difference values ​​of the three channels of the same pixel are averaged to obtain a 9-bit luminance modulation value for each pixel. Since the RGB color values ​​of image pixels have a certain degree of local similarity (i.e., the RGB color values ​​of adjacent pixels in the image are similar), the absolute value of the 9-bit luminance modulation value obtained through the above processing is relatively small. After the exponent calculation module determines the shared exponent, the floating-point number constructed using the shared exponent for all pixels has less loss compared to the original luminance modulation value. Furthermore, using a shared exponent for all pixels, compared to each pixel having its own exponent, significantly improves the compression ratio of the final compressed code and saves a large amount of data storage overhead. Using the digital image decoder proposed in this invention, the original image block can be recovered by decoding through the reverse process of encoding, resulting in a high-quality image.

[0038] Furthermore, by setting a preprocessing module, the device proposed in this invention can be applied to Bayer blocks and RGB888 blocks.

[0039] Furthermore, during bit extension, the high 3 bits of data are appended to the 5-bit primary color storage value α. ik The way to end the image can reduce image loss.

[0040] Furthermore, fixing the optional exponent to four values ​​of 0, 2, 3, and 5, and forming a mapping relationship with the 2-bit shared index shift, can reduce quantization loss while reducing memory consumption. Attached Figure Description

[0041] Figure 1 is a structural block diagram of a digital image encoder according to an embodiment;

[0042] Figure 2 is a schematic diagram of converting a Bayer tile into an RGB888 standard pixel tile according to an embodiment;

[0043] Figure 3 is a schematic diagram of a 2×4 RGB888 image block being divided into 2×2 sub-blocks according to an embodiment;

[0044] Figure 4 is a schematic diagram of the information distribution carried by the compression encoding in an embodiment;

[0045] Figure 5 is a structural block diagram of a digital image decoder according to an embodiment;

[0046] Figure 6 is a schematic diagram of converting an RGB888 standard pixel patch into a Bayer patch according to an embodiment;

[0047] Figure 7 is a schematic diagram of the structure of a digital image processor according to an embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Figure 1 shows a block diagram of a digital image encoder (PicBC encoder) in one embodiment, which mainly includes a primary color calculation module, a difference value calculation module, a luminance modulation calculation module, an index calculation module, and a luminance modulation quantization module.

[0050] The primary color calculation module receives the RGB888 image patch and divides it into two equal rectangular sub-blocks. It calculates the average value of all pixels in sub-block i in channel k, and extracts the high 5 bits of the binary representation of the average value of channel k in sub-block i to obtain the 5-bit primary color storage value α. ik The 5-bit primary color value α is stored by padding the low bits. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik k = R, G, B, i = 0, 1.

[0051] Specifically, the primary color calculation module receives an H*2H or 2H*H RGB888 image patch and divides it into two equal H*H rectangular sub-blocks, with i as the sub-block number (i = 0, 1), meaning the two rectangular sub-blocks are sub-block 0 and sub-block 1. Each sub-block has N pixels, N = H*H, with j as the pixel number (j = 1, 2, 3, ..., N). Each pixel has three RGB channels, with k as the channel number (k = R, G, B). The original color value of each channel is 8 bits.

[0052] Understandably, the smaller the size of the compressed image tile, the higher the compression quality, but the lower the compression efficiency. Therefore, the size of the image tile to be compressed each time can be selected according to the actual situation. In this embodiment, the RGB888 image tile compressed each time is a 192-bit 2×4 or 4×2 image tile, which is divided into 2×2 sub-blocks. This image tile size can be well adapted to the memory bus width of the mobile phone GPU module and is well applied to image processing of mobile devices.

[0053] In one embodiment, an input preprocessing module is provided before the primary color calculation module to identify the format of the input tile: when the input tile is in Bayer format, the Bayer format is converted to RGB888 format before being input to the primary color calculation module; when the input tile is in RGB888 format, it is directly input to the primary color calculation module. Specifically, the blockType_isBayer signal, which is input along with the tile data to be compressed, indicates the input data format. This signal serves as the control signal for the 2-to-1 data selector in the input preprocessing module: when blockType_isBayer = 0, it indicates that the input data is in RGB888 format. At this time, the data is sent to the 2-to-1 data selector through the bypass buffer and directly output to the next stage pipeline register; when blockType_isBayer = 1, it indicates that the input data is in Bayer format. At this time, the data passes through the input preprocessing module, converts the Bayer block to an RGB tile, and then outputs it to the next stage pipeline register.

[0054] The process of converting Bayer format to RGB888 format includes merging each vertical Bayer pixel formed by every two adjacent rows in the Bayer tile into an RGB888 standard pixel. The color values ​​of channels R and B remain unchanged before and after merging, and the color value of channel G after merging is the average of the two channels G in the Bayer pixel before merging.

[0055] As shown in Figure 2, in a 256-bit 4×4 Bayer tile, two vertically adjacent Bayer pixels in the first and second rows are merged into one group, and two vertically adjacent Bayer pixels in the third and fourth rows are merged into another group. The red and blue channels directly take their original values ​​(color values), while the green channel takes the average of the two values ​​as the new sub-pixel's green channel value. After this processing, a Bayer format RGBG or BGRG tile is converted into a 2x4 RGB888 standard pixel tile.

[0056] After dividing the data into sub-blocks, each sub-block is traversed, and the average value of all pixels in sub-block i in channel k is calculated. Since there are three channels (RGB), the average values ​​of the R, G, and B channels are obtained in sub-blocks 0 and 1 respectively. This average value is an 8-bit number. For each channel of each sub-block, the high 5 bits of the binary representation of the average value of channel k in each sub-block are extracted to obtain the 5-bit primary color storage value α. ik A total of 6 primary color storage values ​​α were obtained. ik α represents the primary color storage values ​​α for the R, G, and B channels of sub-block 0, respectively. 0R α 0G and α 0BAnd the primary color storage values ​​α of the R, G, and B channels of sub-block 1 1R α 1G and α 1B .

[0057] Figure 3 shows that a 2×4 RGB888 patch is divided into 2×2 sub-blocks, namely sub-block 0 and (sub-block 0) sub-block 1. All channels of pixels 0, 1, 4, and 5 in sub-block 0 (left) are traversed (one pixel corresponds to three RGB channels), and the average value of the three RGB channels in sub-block 0 is obtained. and Similarly, iterate through all channels of pixels 2, 3, 6, and 7 of sub-block 1 (right) to obtain the average value of the RGB values ​​of sub-block 1. and average value and and average and Each value is 8 bits. The high 5 bits of each average value are extracted to obtain the primary color storage value of each channel in each sub-block, which is the RGB three-component primary color storage value α in sub-block 0. 0R α 0G and α 0B And the RGB three-component primary color storage value α in sub-block 1 1R α 1G and α 1B .

[0058] Since the subsequent calculations involve 8-bit binary operations, it is also necessary to store the 5-bit primary color value α. ik The value β is calculated by expanding the low-bit primary color to 8-bit primary color through low-bit filling. ik That is, padding the lower 3 bits. Specifically, this can be done by randomly padding with 0s or 1s, thus filling the lower 3 bits to store the 5-bit primary color value α. ik Extended to 8 bits, each representing the RGB three-component primary color storage value β in sub-block 0. 0R β 0G and β 0B And the RGB three-component primary color storage value β in sub-block 1 1R β 1G and β 1BIn this embodiment, the high 3 bits of binary data are appended to the end to pad to 8 bits. For example, if the decimal values ​​of the primary color storage values ​​for the RGB three channels are (9, 2, 20), the corresponding 5-bit binary data are (01001, 00010, 10100). After the bit expansion described above, the resulting 8-bit binary data are (01001010, 00010000, 10100101), which correspond to the decimal values ​​(74, 16, 165). Experiments show that appending the high 3 bits of binary data to the end to pad to 8 bits can better reduce image loss.

[0059] The difference calculation module is used to iterate through the original color value γ of channel k of pixel j in sub-block i. ijk And calculate the corresponding primary color value β ik The difference is calculated to obtain the corresponding signed 9-bit color difference value Δ. ijk Specifically, each sub-block has N pixels, and each pixel has three channels. Therefore, each sub-block has 3N raw color values, and the two sub-blocks have a total of 6N raw color values. Each raw color value γ ijk The primary color value β is calculated for the corresponding type channel in the corresponding sub-block. ik The subtraction operation is performed 6N times to obtain 6N color difference values ​​Δ. ijk Since the result of subtraction can be positive or negative, the resulting color difference value Δ ijk For a 9-bit signed number, Δ ijk This represents the color difference value of channel k for pixel j in sub-block i.

[0060] In one embodiment, when each sub-block has 4 pixels (N=4), and two sub-blocks have 8 pixels, with each pixel having three RGB channels, the difference value calculation module needs to perform a total of 8*3=24 subtraction operations. Specifically, the difference value calculation module uses 24 parallel subtractors to subtract the original color value of each RGB channel of each pixel in the sub-block from the "bit-extended" primary color calculation value of the corresponding channel of that sub-block, obtaining 24 9-bit signed pixel color difference values.

[0061] The luminance modulation calculation module is used to calculate the 9-bit luminance modulation value of pixel j in sub-block i. After processing by the aforementioned difference value calculation module, three color difference values ​​for each pixel position in each sub-block are obtained. The average of the three color difference values ​​is then calculated to obtain the 9-bit luminance modulation value for each pixel in the corresponding sub-block.

[0062] Considering a well-known characteristic of human vision: the human retina contains approximately 6 to 8 million cone cells (for perceiving the color of light) and 120 million rod cells (for perceiving the intensity of light), the human eye is more sensitive to the intensity of light than to its color. Therefore, a three-channel differential averaging process is performed. When reconstructing pixel tiles, brightness adjustment is only applied to the reconstructed pixels based on the primary color (by adding or subtracting the same value from each of the pixel's RGB channels), rather than adjusting each color channel independently. This compression method minimizes the presence of noise caused by "color shift" in the reconstructed tiles.

[0063] In one embodiment, a relatively common integer division quantization method is used: Note that mathematically, the following equivalent expression is clearly present: and Converting it to a fixed-point fractional quantization of 8 bits: 0.33333333… D ≈0.33203125 D =0.01010101 B Right now: After the above transformation, the mathematical operation of "dividing by 3" can be regarded as "first multiplying the dividend by 85, and then arithmetically shifting the resulting binary result 8 bits to the right".

[0064] The exponent calculation module is used to determine the exponent of the luminance modulation value with the largest absolute value, which is converted into a floating-point number with the smallest loss and has a 4-bit mantissa, and this exponent is used as the shared exponent for all luminance modulation values. The corresponding shared index is determined according to the mapping relationship between the shared exponent and the shared index shift; different shared indices shift are mapped one-to-one with different shared exponents.

[0065] After obtaining the 9-bit luminance modulation value of each pixel in each sub-block through the luminance modulation calculation module, since each luminance modulation value is a 9-bit signed number, which occupies too many bits of data, it needs to be quantized to reduce the number of bits it occupies.

[0066] Here, each 9-bit luminance modulation value is quantized into a truncated floating-point number for storage to achieve image encoding compression. As is well known, any binary number F can be expressed as a floating-point number: F = S * 2. PWhere S is the mantissa and P is the exponent. In this invention, the mantissa is determined to be 4 bits. For the quantization of each luminance modulation value, the luminance modulation of each pixel is represented by both the 4-bit mantissa and a globally shared exponent. Therefore, it is necessary to find a suitable exponent as the shared exponent for all luminance modulation values, which can minimize the quantization error after scaling the above 9-bit luminance modulation value to a 4-bit signed number and using the shared exponent. The idea of ​​determining the shared exponent in this invention is: to obtain the exponent corresponding to each 9-bit luminance modulation value when converted to a 4-bit mantissa, and select the largest exponent among them as the shared exponent. Therefore, this invention only needs to select the luminance modulation value with the largest absolute value, convert it to a floating-point number with a 4-bit mantissa, and determine the exponent that minimizes the quantization loss as the shared exponent.

[0067] The luminance modulation quantization module is used to quantize each luminance modulation value. The last 4 bits after right shifting the shared exponent are used as the 4-bit mantissa δ of the corresponding luminance modulation value. ij In fact, dividing the luminance modulation value by the shared exponent P is equivalent to shifting the luminance modulation value to the right by P bits, and then using the last 4 bits of the shifted value as the 4-bit mantissa δ of the corresponding luminance modulation value. ij .

[0068] In one embodiment, the exponent calculation module can pre-set selectable exponents. When determining the shared exponent, only the exponent with the smallest quantization loss can be selected from the pre-set exponents. Since the luminance modulation value to be quantized is 9 bits, when the mantissa is fixed at 4 bits, the exponent can be a maximum of 5 and a minimum of 0. Specifically, a mapping relationship between the exponent and the shared index can be set, with different shared index shifts mapped one-to-one to different exponents. The number of selectable exponents can also be flexibly set as needed. For example, six exponents can be set (0, 1, 2, 3, 4, 5); or five or four exponents can be set, with no limitation on the number of exponents. The more exponents, the more bits are allocated to the shared index. For example, when six exponents are set, 3 bits are needed to represent the shared index; when four exponents are set, only 2 bits are needed to represent the shared index, further saving storage space.

[0069] In this embodiment, four optional exponents are preferably set: 0, 2, 3, and 5, each corresponding to a 2-bit shared index. The mapping relationship between the 2-bit shared index and the exponent can be represented as shown in Table 1:

[0070] Table 1

[0071] 2-bit shared index exponent 000012103115 surface

[0072] Based on the above four optional exponents, in this embodiment, the shared exponent can be determined from the individual exponents in the following manner:

[0073] Determine if the luminance modulation value with the largest absolute value is within the range of [-8, +7]. If so, select an exponent of 0 and set the corresponding 2-bit shared index shift to 00. At this point, extract each luminance modulation value. The [3:0] bits are used as its 4-bit mantissa δ ij ;

[0074] If not, determine if the luminance modulation value with the largest absolute value is within the range of [-32, +31]. If so, select an exponent of 2, and set the corresponding 2-bit shared index shift to 0 or 1. At this point, extract each luminance modulation value. The [5:2] bits are used as its 4-bit mantissa δ ij ;

[0075] If not, determine if the luminance modulation value with the largest absolute value is within the range of [-64, +63]. If so, select an exponent of 3 and the corresponding 2-bit shared index shift of 10. At this point, extract each luminance modulation value. The [6:3] bits are used as its 4-bit mantissa δ ij ;

[0076] If not, it means that the luminance modulation value with the largest absolute value is in the range of [-256, +255]. In this case, the exponent is chosen as 5, and the corresponding 2-bit shared index shift is 11. Then, each luminance modulation value is truncated. The [8:5] bits are used as its 4-bit mantissa δ ij .

[0077] After quantization, the 9-bit luminance modulation value is transformed into a 4-bit quantization mantissa and a shared 2-bit exponent index, thus achieving image encoding compression.

[0078] At this point, the compressed encoding of the image patch is complete. The compressed encoding carries a 5-bit primary color storage value α. ik 4-bit mantissa δ ij And the shared index shift. Taking the 2-bit shared index shift as an example, Figure 4 shows the information distribution carried by the compressed encoding. The encoding is 64 bits in total, and its information distribution is shown in Table 2 below:

[0079] Table 2

[0080] The meaning of the position field [3,0]δ 00 The 4-bit mantissa [7,4]δ of the 0th pixel's luminance modulation value in sub-block 0 01The 4-bit mantissa [11,8] of the first pixel's luminance modulation value in sub-block 0 10 The 4-bit mantissa [15,12] of the luminance modulation value of the 0th pixel in sub-block 1 11 The 4-bit mantissa [19,16]δ of the first pixel's luminance modulation value in sub-block 1 02 The 4-bit mantissa [23,20] of the luminance modulation value of the second pixel in sub-block 0 03 The 4-bit mantissa [27,24] of the 3rd pixel luminance modulation value in sub-block 0 12 The 4-bit mantissa [31,28] of the luminance modulation value of the second pixel in sub-block 1 13 The 4-bit mantissa [33,32] of the luminance modulation value of the 3rd pixel in sub-block 1 corresponds to the shared index [38,34] of the shared exponent. 0R The 5-bit primary color storage value [43,39]α of the R channel in sub-block 0 0G The 5-bit primary color storage value [48,44]α of channel G in sub-block 0 0B The 5-bit primary color storage value [53,49]α of channel B in sub-block 0 1R The 5-bit primary color storage value [58,54]α of the R channel in sub-block 1 1G The 5-bit primary color storage value [63,59]α of channel G in sub-block 1 1B The 5-bit primary color storage value of channel B in sub-block 1 surface

[0081] It should be noted that the specific positions of the above fields in the compression encoding are only examples, and are not limited to these.

[0082] When the original input is a 256-bit 4×4 Bayer tile, it is finally compressed to 64 bits, with a compression rate of 25%; when the original input is a 192-bit 2×4 RGB888 tile, it is finally compressed to 64 bits, with a compression rate of 33.3%. Good compression is achieved for both Bayer tiles and RGB888 tiles.

[0083] Understandably, the decoder employs an explicitly designed multi-stage pipelined processing structure. The computation modules between each pair of adjacent pipeline register groups are purely combinational logic. Data flows to the next stage of the pipeline only when the rising edge of the clock is present and the valid-ready handshake signals are valid; otherwise, the pipeline will stop.

[0084] Accordingly, the present invention also relates to a digital image decoder (PicBC decoder) for decoding the compressed code obtained above and recovering the corresponding image block. Figure 5 shows a structural block diagram of the digital image decoder in one embodiment, which mainly includes a primary color bit extension module and a brightness modulation module.

[0085] The primary color bit extension module is used to extract the 5-bit primary color storage value α from the compressed encoding. ik The 5-bit primary color value α is stored using the same low-bit padding method as the primary color calculation module. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik This process is the same as the bit expansion method in the primary color calculation module. After bit expansion, each sub-block obtains the primary color calculation values ​​for the three RGB channels.

[0086] The luminance modulation module is used to extract the shared index shift in the compressed code and find the corresponding shared exponent P through the mapping relationship, and to extract the 4-bit mantissa δ in the compressed code. ij And construct a floating-point number with the shared exponent P as the signed 9-bit luminance modulation value of pixel j in sub-block i. This process is equivalent to changing the 4-bit mantissa δ ij Shift left by P bits. Then adjust the brightness modulation value. Calculated value β of primary color ik Adding them together, we obtain the color restoration value γ' of channel k for pixel j in sub-block i. ijk The RGB values ​​of the pixels within the block are restored to obtain an RGB888 image block.

[0087] This process demonstrates that, due to the shared exponent used in encoding and the use of truncated mantissa representation, the 9-bit luminance modulation value recovered during decoding... Compared with the original 9-bit luminance modulation value during encoding There is some deviation, but because the quantization loss has been minimized during encoding, the loss of the recovered luminance modulation value during decoding is small, and the quality of the finally recovered tile is also better.

[0088] In one embodiment, the digital image decoder further includes an output clamping module for when the color recovery value γ' ijk When the value exceeds the range of [0, 255], if the detected color recovery value is less than 0, its value is forcibly set to 0; if the detected color recovery value is greater than 255, its value is forcibly set to 255.

[0089] Specifically, the pixel color values ​​after the above modulation may not conform to the range [0, 255]. For example, a pixel block has a primary color of (10, 86, 130), but its brightness is modulated to -20. After the above modulation, its reconstructed pixel color value is (-10, 66, 110). If no processing is performed, the output color will obviously show a "negative overflow," which will manifest as "color shift" noise in the final reconstructed image, which has a significant negative impact on the quality of compressed images. To avoid the above problem, clamping is required before the reconstructed pixel output. The specific implementation method is very simple, but effective: use a two-stage cascaded data comparator. When a pixel color value is detected to be less than 0, its value is forcibly set to 0; when a pixel color value is detected to be greater than 255, its value is forcibly set to 255.

[0090] In one embodiment, the digital image decoder further includes an output format adjustment module, used to convert the recovered RGB888 blocks into Bayer blocks when Bayer format blocks need to be output. The conversion process includes splitting each RGB888 standard pixel in the RGB888 block into two Bayer pixels, where the color values ​​of channels R and B remain unchanged before and after the split, and channel G in the RGB888 standard pixel is split into two channels G in the Bayer pixel with unchanged color values. As shown in Figure 6, the image format reconstructed after the preceding steps is a 2×4 RGB888 pixel block. When blockType_isBayer is "1", the RGB format blocks cannot be directly output; therefore, RGB888 to Bayer format conversion is required. The conversion process here is the reverse of the Bayer-RGB process mentioned earlier: by splitting the green pixel into two parts, an RGB pixel is transformed into two sub-pixels "RG-BG" in the Bayer mode. Furthermore, a 2×4 RGB tile is transformed into a 4×4 Bayer tile.

[0091] Accordingly, this invention also relates to a digital image processor, as shown in Figure 7, which includes the digital image encoder and digital image decoder described above. This digital image processor can compress an image block by block at a fixed compression rate. The encoding and decoding of each block only depends on the local data within the block, and the value of any specified pixel can be obtained without decompressing the entire image, making it very suitable for streaming image processing and data transmission. Moreover, its encoding and decoding complexity is greatly reduced, the required resource overhead is reduced, and the hardware acceleration efficiency is significantly improved. Verified on an FPGA system, the digital image processor designed in this paper is compatible with the AXI-Stream interface standard and supports AXI-DMA data transmission. In pipelined operation mode, the PicBC IP operates at a 100MHz clock frequency, the encoder calculation delay is 6 clock cycles, and the decoder calculation delay is 4 clock cycles. Each clock cycle can perform compression / decompression of one block of data, and the image data throughput is 3.2GB / s.

[0092] Accordingly, the present invention also relates to an electronic device comprising the aforementioned digital image processor, capable of encoding and decoding images.

[0093] The advantages of the PicBC encoder and PicBC decoder designed in this invention will be explained below with specific data.

[0094] The hardware resource overhead of the PicBC encoder is shown in Table 3:

[0095] Table 3

[0096]

[0097] The hardware resource overhead of the PicBC decoder is shown in Table 4:

[0098] Table 4

[0099]

[0100] Referring to Figure 7, an application scenario of the present invention is shown. The digital image processor PicBC IP is placed between the data source that needs to be compressed and the system memory or data path. After simple compression, it can save 66% to 75% of the resource consumption of system memory and data path bandwidth (the compression rate is 25% for Bayer format input images and 33% for RGB888 format input images).

[0101] Specifically, a host computer connected to the FPGA development board via a UART serial port sent a segment of test image data to the test SoC. After compression by the PicBC encoder, the compressed data was returned sequentially via the same serial port and output to the serial communication monitor. The compressed data was then decompressed, decompressed by the PicBC decoder, and the decompressed reconstructed image data was output to the serial communication monitor via the serial port. Finally, computer software comparison showed that the results calculated by the PicBC IP digital image processor largely matched the theoretically correct results, verifying the reliability of the invention. In addition, this embodiment also compared the compressed and reconstructed image data with the original image data pixel by pixel, calculating the mean square error (MSE) and peak signal-to-noise ratio (PSNR) caused by compression and decompression. The test results are shown in Table 5.

[0102] Table 5

[0103]

[0104] As shown in Table 5, after the digital image compression and decompression proposed in this invention, the mean square error (MSE) between the recoded image and the original image is 6.3762, and the peak signal-to-noise ratio (PSNR) reaches 40dB, meeting the "quasi-lossless" accuracy requirement for human visual observation. More specifically, the "weighted average" mentioned in the "Total" column of Table 5 means that the final total MSE weighted average is calculated according to the number of pixels in each test image.

[0105] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A digital image encoder for compressing and encoding digital images, characterized in that, include: The primary color calculation module receives an RGB888 image patch and divides it into two equal sub-blocks. Each sub-block has an equal number of rows and columns of pixels. It calculates the average value of all pixels in sub-block i across channel k, and extracts the high 5 bits of the binary representation of the average value of channel k in sub-block i to obtain the 5-bit primary color storage value α. ik The 5-bit primary color value α is stored by padding the low bits. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik k = R, G, B, i = 0, 1; The difference value calculation module is used to iterate through the original color value γ of channel k of pixel j in sub-block i. ijk And calculate the corresponding primary color value β ik The difference is calculated to obtain the corresponding signed 9-bit color difference value Δ. ijk j = 1, 2, 3, ..., N, where N is the number of pixels in each sub-block; The luminance modulation calculation module is used to calculate the 9-bit luminance modulation value of pixel j in sub-block i. The exponent calculation module is used to determine the exponent of the luminance modulation value with the largest absolute value, which is converted into a floating-point number with the smallest loss and has a 4-bit mantissa, and to use it as the shared exponent for all luminance modulation values. The corresponding shared index is determined according to the mapping relationship between the exponent and the shared index shift; different shared indices shift are mapped one-to-one with different exponents. The brightness modulation quantization module is used to quantize each brightness modulation value. The last 4 bits after right shifting the shared exponent are used as the 4-bit mantissa δ of the corresponding luminance modulation value. ij The compressed code carries a 5-bit primary color storage value α. ik 4-bit mantissa δ ij And shared index shift.

2. The digital image encoder as described in claim 1, characterized in that, It also includes an input preprocessing module for recognizing the format of the input tile: when the input tile is in Bayer format, the Bayer format is converted to RGB888 format and then input into the primary color calculation module; When the input image is in RGB888 format, it can be directly input into the primary color calculation module; The process of converting Bayer format to RGB888 format includes merging every two vertically adjacent Bayer pixels in every two adjacent rows of the Bayer tile into RGB888 standard pixels. The color values ​​of channels R and B remain unchanged before and after merging, and the color value of channel G after merging is the average of the two channels G in the Bayer pixels before merging.

3. The digital image encoder as described in claim 1, characterized in that, The RGB888 image blocks received by the primary color calculation module are H*2H or 2H*H images, which are divided into H*H sub-blocks, where H=2.

4. The digital image encoder as described in claim 1, characterized in that, In the primary color calculation module, the 5-bit primary color value α is stored by padding the low bits. ik Expanding to 8 bits includes: storing the 5-bit primary color value α ik The high 3 bits of data are appended to the 5-bit primary color storage value α. ik At the end, we obtain the 8-bit primary color calculation value β. ik .

5. The digital image encoder as described in claim 1, characterized in that, In the luminance modulation quantization module, the shared index shift is 2 bits. The exponent calculation module is used to determine the shared exponent that meets the conditions from four different exponents: 0, 2, 3, and 5. The exponent is an exponent with a base of 2. The four different values ​​of the 2-bit shared index shift are mapped one-to-one with the four different exponents. The method for determining the shared exponent by the exponent calculation module includes: determining whether the absolute value of the largest luminance modulation value is in the range of [-8, +7]; if yes, selecting exponent 0; if no, determining whether the absolute value of the largest luminance modulation value is in the range of [-32, +31]; if yes, selecting exponent 2; if no, determining whether the absolute value of the largest luminance modulation value is in the range of [-64, +63]; if yes, selecting exponent 3; if no, indicating that the absolute value of the largest luminance modulation value is in the range of [-256, +255], selecting exponent 5.

6. A digital image decoder, characterized in that, The digital image decoder is used to decode the compressed code output by the digital image encoder according to any one of claims 1 to 5 to output image tiles. The digital image decoder includes: a primary color bit extension module, used to extract the 5-bit primary color storage value α from the compressed code. ik The 5-bit primary color value α is stored using the same low-bit padding method as the primary color calculation module. ik Extending to 8 bits, we obtain the primary color calculation value β of channel k in sub-block i. ik The luminance modulation module is used to extract the shared index shift in the compressed code and find the corresponding shared exponent through the mapping relationship, and to extract the 4-bit mantissa δ in the compressed code. ij And construct a floating-point number with the shared exponent as the signed 9-bit luminance modulation value of pixel j in sub-block i. Brightness modulation value Calculated value β of primary color ik Adding them together, we obtain the color restoration value γ' of channel k for pixel j in sub-block i. ijk The RGB888 tile was recovered.

7. The digital image decoder as described in claim 6, characterized in that, Also includes: Output clamping module, used when color recovery value γ' ijk When the value exceeds the range of [0, 255], if the detected color recovery value is less than 0, its value is forcibly set to 0; if the detected color recovery value is greater than 255, its value is forcibly set to 255.

8. The digital image decoder as described in claim 6, characterized in that, Also includes: The output format adjustment module is used to convert the recovered RGB888 blocks into Bayer blocks when Bayer format blocks need to be output, and to directly output the decoded RGB888 blocks when RGB888 blocks need to be output. The conversion process of converting the decoded RGB888 blocks into Bayer blocks includes: splitting each RGB888 standard pixel in the RGB888 block into two Bayer pixels, wherein the color values ​​of channels R and B remain unchanged before and after the split, and splitting channel G in the RGB888 standard pixel into two channels G in the Bayer pixel with the color values ​​unchanged.

9. A digital image processor, characterized in that, include: The digital image encoder according to any one of claims 1 to 5; and the digital image decoder according to any one of claims 6 to 8.

10. An electronic device, characterized in that, Includes the digital image processor as described in claim 9.

Images