A Bayer data compression method and related device

Abstract

The present application discloses a method for compressing Bayer data and a related device, the method comprising obtaining a reference image block corresponding to a pixel to be predicted, interpolating pixels of different color channels in the reference image block according to the color difference continuity of the Bayer image, so as to obtain an interpolated continuous reference image block whose color channels are the same as the color channels of the pixel to be predicted, and then encoding based on the interpolated reference image block. In the present application, continuous reference image blocks with the same color channels are obtained by an interpolation method, which improves the spatial correlation of adjacent pixels in the continuous reference image block, thereby improving the compression rate of Bayer data.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a Bayer data compression method and related devices. Background Art

[0002] At present, most cameras and digital cameras use CMOS sensors for light sensing. In order to save area and power consumption, CMOS sensors generally use Bayer arrays to collect image data, so as to use fewer acquisition units to represent large-size image resolution. Figure 1 As shown, the same component data in the image data collected by the Bayer array is discontinuous, which makes the spatial correlation of adjacent pixels weak, resulting in the prediction performance of the Bayer data, and further leads to low compression efficiency of the Bayer data.

[0003] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a bayer data compression method and related device in view of the deficiencies of the prior art.

[0005] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for compressing bayer data, the method comprising:

[0006] Obtain a reference image block corresponding to a pixel to be predicted, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e, and pix_b, the reference pixels pix_c, pix_a, pix_d, and pix_e are continuous and located in a preceding row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted;

[0007] Obtaining a pixel to be interpolated in the reference image block, and determining an interpolated pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolated pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted;

[0008] The prediction residual of the pixel to be predicted is determined based on the interpolated reference image block, and a code stream is output based on the prediction residual.

[0009] The method for compressing bayer data, wherein obtaining a reference image block corresponding to a pixel to be predicted specifically comprises:

[0010] Obtain the preceding row corresponding to the pixel to be predicted, wherein when the pixel to be predicted corresponds to the green channel, the preceding row of the current row where the pixel to be predicted is located is the preceding row; when the pixel to be predicted corresponds to the red channel or the blue channel, the preceding two rows of the current row where the pixel to be predicted is located are the preceding rows;

[0011] Reference pixel points pix_c, pix_a, pix_d and pix_e are selected in the preceding row, and a reference pixel point pix_b is selected at the left end of the pixel to be predicted to obtain a reference image block.

[0012] The bayer data compression method, wherein obtaining the pixel points to be interpolated in the reference image block specifically comprises:

[0013] When the pixel to be predicted corresponds to the green channel, the reference pixel pix_a and the reference pixel pix_e are used as the pixel to be interpolated;

[0014] When the pixel to be predicted corresponds to the red channel or the blue channel, the reference pixel pix_c and the reference pixel pix_b are used as the pixel to be interpolated.

[0015] The method for compressing bayer data, wherein determining the interpolation pixel value of the pixel to be interpolated by pixel interpolation specifically comprises:

[0016] When the pixel to be predicted corresponds to the green channel, the interpolation pixel value of the pixel to be interpolated is determined by a first interpolation process.

[0017] When the pixel to be predicted corresponds to the red channel or the blue channel, the interpolation pixel value of the pixel to be interpolated is determined by a second interpolation process.

[0018] The bayer data compression method, wherein the first interpolation process specifically includes:

[0019] Select candidate pixel points pix_A, pix_B, pix_C and pix_D in the row where the pixel to be interpolated is located, select candidate pixel point pix_E in the row before the pixel to be interpolated, and select candidate pixel point pix_F in the two rows before the pixel to be interpolated, where pix_A, pix_B, the pixel to be interpolated, pix_C and pix_D are continuous from left to right, pix_E is located directly above the pixel to be interpolated, and pix_F is located directly above pix_E;

[0020] Based on the pixel values ​​of the candidate pixel pix_A, the pixel to be interpolated, pix_D and pix_F, determine the interpolation weight coefficient corresponding to the pixel to be interpolated;

[0021] Based on the pixel values ​​of pix_B, pix_C and pix_E, and the interpolation weight coefficient, an interpolation pixel value of the pixel to be interpolated is determined, wherein the interpolation pixel value corresponds to a green channel.

[0022] The bayer data compression method, wherein the second interpolation process specifically includes:

[0023] Select candidate pixel points pix_G and pix_H in the row where the pixel point to be interpolated is located, where pix_G, the pixel point to be interpolated and pix_H are continuous from left to right;

[0024] Determine the interpolation pixel values ​​corresponding to pix_G and pix_H through a first interpolation process;

[0025] Based on the pixel values ​​and interpolated pixel values ​​corresponding to pix_G and pix_H, respectively, and the original pixel value of the pixel to be interpolated, the interpolated pixel value corresponding to the pixel to be interpolated is determined.

[0026] The method for compressing bayer data, wherein determining the prediction residual of the pixel to be predicted based on the interpolated reference image block and outputting a code stream based on the prediction residual specifically comprises:

[0027] For each preset direction among the plurality of preset directions, selecting a target pixel point corresponding to the pixel point to be predicted in the interpolated reference image block based on the preset direction, and determining a candidate pixel value corresponding to the pixel point to be predicted based on the target pixel point;

[0028] Determining a predicted pixel value of the pixel to be predicted according to the determined candidate pixel values;

[0029] A prediction residual is determined according to the predicted pixel value and the original pixel value of the pixel to be predicted, and a code stream is output based on the prediction residual.

[0030] A second aspect of an embodiment of the present application provides a bayer data compression system, the system comprising:

[0031] an acquisition module, used to acquire a reference image block corresponding to a pixel to be predicted, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, the reference pixels pix_c, pix_a, pix_d and pix_e are continuous and located in a preceding row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted;

[0032] an interpolation module, configured to obtain a pixel to be interpolated in the reference image block, and determine an interpolation pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolation pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted;

[0033] The encoding module is used to determine the prediction residual of the pixel to be predicted based on the interpolated reference image block, and output a code stream based on the prediction residual.

[0034] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the bayer data compression methods described above.

[0035] A fourth aspect of the embodiments of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0036] The communication bus realizes the connection and communication between the processor and the memory;

[0037] When the processor executes the computer-readable program, the processor implements the steps in any of the above-mentioned methods for compressing bayer data.

[0038] Beneficial effects: Compared with the prior art, the present application provides a method for compressing Bayer data and a related device, the method comprising obtaining a reference image block corresponding to a pixel to be predicted, interpolating pixels of different color channels in the reference image block according to the color difference continuity of the Bayer image, so as to obtain an interpolated continuous reference image block whose color channels are the same as the color channels of the pixel to be predicted, and then encoding based on the interpolated reference image block. In the present application, continuous reference image blocks with the same color channels are obtained by an interpolation method, which improves the spatial correlation of adjacent pixels in the continuous reference image block, thereby improving the compression rate of the Bayer data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0040] Figure 1A schematic diagram of Bayer data.

[0041] Figure 2 Flow chart of the bayer data compression method provided in this application.

[0042] Figure 3 Schematic diagram of the reference image block corresponding to the pixel to be predicted in the green channel.

[0043] Figure 4 Schematic diagram of the reference image block corresponding to the pixel to be predicted in the red channel.

[0044] Figure 5 Schematic diagram of candidate pixel points corresponding to the pixel points to be interpolated in the green channel.

[0045] Figure 6 Schematic diagram of candidate pixel points corresponding to the pixel points to be interpolated in the red channel.

[0046] Figure 7 Schematic diagram of the interpolated reference image block corresponding to the to-be-interpolated pixel point of the green channel.

[0047] Figure 8 Schematic diagram of the interpolated reference image block corresponding to the pixel to be interpolated of the red channel.

[0048] Fig. 9 A schematic diagram of the eight preset directions.

[0049] Fig.10 Structural principle diagram of the bayer data compression system provided in this application.

[0050] Fig.11 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0051] The present application provides a method for compressing bayer data and a related device. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0053] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0054] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0055] After research, it was found that most cameras and digital cameras currently use CMOS sensors for light sensing. In order to save area and power consumption, CMOS sensors generally use Bayer arrays to collect image data, so that a small number of acquisition units can be used to represent large-size image resolution. However, since CMOS sensor imaging will have image noise, especially under insufficient light conditions, the noise problem of image sensors will be more prominent, resulting in a gap between the image details and colors of the image directly obtained from the sensor and the actual scene.

[0056] Based on this, ISP (Image Signal Processing) is needed to post-process the images collected by the sensor, for example, including linear correction, noise removal, bad pixel removal, interpolation, automatic white balance, automatic exposure, etc. In most SOCs, there is only one or two ISPs, and the connected sensors may be multiple. For example, in some monitoring and driving applications, there may be as many as 8-10 sensors. In this way, ISP can only be time-division multiplexed, and the data transmitted by the sensor can only be stored in DDR first, and ISP reads data from DDR. This process increases the write-read operation of Bayer data on the one hand, and causes bandwidth and memory pressure on the DDR storage system on the other hand. Therefore, it is necessary to compress the bayer data collected by the sensor. At present, the existing image compression algorithm is generally used to compress the bayer. However, Figure 1 As shown, the same component data in the image data collected by the Bayer array is discontinuous, which makes the spatial correlation of adjacent pixels weak, resulting in the prediction performance of the Bayer data, and further leads to low compression efficiency of the Bayer data.

[0057] In order to solve the above problem, in an embodiment of the present application, a reference image block corresponding to a pixel to be predicted is obtained, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, the reference pixels pix_c, pix_a, pix_d, pix_e are continuous and located in the leading row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted; the pixel to be interpolated in the reference image block is obtained, and the interpolation pixel value of the pixel to be interpolated is determined by pixel interpolation to obtain the interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolation pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted; the prediction residual of the pixel to be predicted is determined based on the interpolated reference image block, and a code stream is output based on the prediction residual. This embodiment obtains continuous reference image blocks with the same color channel by interpolation, thereby increasing the spatial correlation of adjacent pixels in the continuous reference image blocks, thereby improving the compression rate of Bayer data.

[0058] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0059] This embodiment provides a method for compressing bayer data, such as Figure 2 As shown, the method includes:

[0060] S10: Obtain a reference image block corresponding to the pixel to be predicted.

[0061] Specifically, the reference image block is used to predict the pixel value of the pixel to be predicted, that is, the pixel to be predicted is predicted based on the pixel value of the pixel in the reference image block. The reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, pix_c, pix_a, pix_d, pix_e are all located in the leading row of the pixel to be predicted, and pix_c, pix_a, pix_d, pix_e are arranged continuously from left to right. It can be understood that the reference item pix_c is located at the left end of pix_a, pix_a is located at the left end of pix_d, pix_d is located at the left end of pix_e, and the reference pixel pix_a is located directly above the pixel to be predicted. The reference pixel pix_b is located in the row where the pixel to be predicted is located, and it is located at the left-left end of the pixel to be predicted, wherein the left-left end refers to the left side of the pixel to be predicted and is one pixel away from the pixel to be predicted. In addition, the color channel corresponding to the reference pixel pix_b is the same as the color channel corresponding to the pixel to be predicted. In other words, the reference pixel pix_b is the pixel closest to the pixel to be predicted in the row where the pixel to be predicted is located and has the same corresponding color channel.

[0062] Furthermore, the preceding row is the row closest to the current row where the pixel to be predicted is located and contains pixels with the same color channel as the pixel to be predicted. Figure 1 As shown, when the pixel to be predicted corresponds to the green channel, each pixel row before the current row carries the pixel points corresponding to the green channel, so that the preceding row is the row before the current row; when the pixel to be predicted is the red channel or the blue channel, each alternate row carries the pixel points of the color channel corresponding to the pixel to be predicted, so that its preceding rows are the first two rows.

[0063] Based on this, in one implementation, obtaining a reference image block corresponding to a pixel to be predicted specifically includes:

[0064] S11, obtaining the preceding row corresponding to the pixel to be predicted;

[0065] S12. Select reference pixel points pix_c, pix_a, pix_d and pix_e in the preceding row, and select a reference pixel point pix_b at the left end of the pixel to be predicted, so as to obtain a reference image block.

[0066] Specifically, when obtaining the preceding row corresponding to the pixel to be predicted, the color channel corresponding to the pixel to be predicted is first determined, and the preceding row corresponding to the pixel to be predicted is determined according to the color channel, wherein the corresponding relationship between the color channel and the preceding row is: Figure 3 As shown, when the pixel to be predicted corresponds to the green channel, the previous row of the current row where the pixel to be predicted is located is the preceding row; Figure 4 As shown, when the pixel to be predicted corresponds to the red image channel or the blue channel, the first two rows of the current row where the pixel to be predicted is located are the preceding rows. In addition, since serial data is obtained from the sensor, when the current row is encoded, the preceding row has been encoded and decoded, so that the reconstructed pixels pix_c, pix_a, pix_d and pix_e in the preceding row can be read from the cache, and the original pixel value of the reference pixel pix_b and the original pixel value of the pixel to be predicted can be read, so as to determine the predicted pixel value of the pixel to be predicted with reference to the image block, and then obtain the prediction residual of the pixel to be predicted.

[0067] S20, obtaining pixels to be interpolated in the reference image block, and determining interpolation pixel values ​​of the pixels to be interpolated by pixel interpolation to obtain an interpolated reference image block.

[0068] Specifically, the pixel to be interpolated is a pixel in the reference image block whose color channel is different from the color of the pixel to be predicted, that is, the color channels corresponding to some pixels in the reference image block are the same as the pixel to be predicted, and the color channels corresponding to some pixels are different from the color channels of the pixel to be predicted. For example, in the reference image block corresponding to the pixel to be predicted, the reference pixel pix_c corresponds to the green channel, pix_a corresponds to the red channel, pix_d corresponds to the green channel, pix_e corresponds to the red channel, pix_b corresponds to the green channel, and the pixel to be predicted corresponds to the green channel, then the pixel to be interpolated is the reference pixels pix_a and pix_e.

[0069] In addition, from the arrangement of the Bayer data, it can be known that the pixel points to be interpolated corresponding to the pixel points of the green channel are different from the pixel points to be interpolated corresponding to the pixel points of the red channel and the blue channel, and the pixel points to be interpolated corresponding to the red channel and the blue channel are the same. Therefore, when determining the pixel points to be interpolated corresponding to the pixel points to be predicted, it can be determined according to the color channel corresponding to the pixel points to be interpolated. Therefore, in one implementation, the acquisition of the pixel points to be interpolated in the reference image block specifically includes: when the pixel points to be predicted correspond to the green channel, the reference pixel points pix_a and the reference pixel points pix_e are used as the pixel points to be interpolated; when the pixel points to be predicted correspond to the red channel or the blue channel, the reference pixel points pix_c and the reference pixel points pix_b are used as the pixel points to be interpolated.

[0070] After obtaining the pixel to be interpolated, pixel interpolation is performed on the pixel to be interpolated to obtain the pixel value of the pixel to be interpolated on the color channel corresponding to the pixel to be predicted. For example, if the color channel corresponding to the pixel to be predicted is the green channel, then the pixel value of the pixel to be interpolated on the green channel is determined by interpolation. In other words, the color channel corresponding to the interpolated pixel value obtained by interpolation of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted. For example, if the color channel corresponding to the predicted pixel is the green channel, then the color channel corresponding to the interpolated pixel value is the green channel. In addition, since the distribution of the pixels of the green channel is different from that of the pixels of the red channel and the blue channel in the bayer data, different interpolation methods are used for the green channel, the red channel and the blue channel to improve the accuracy of the interpolated pixel values ​​obtained by interpolation.

[0071] In one implementation, determining the interpolation pixel value of the pixel to be interpolated by pixel interpolation specifically includes:

[0072] When the pixel to be predicted corresponds to the green channel, the interpolation pixel value of the pixel to be interpolated is determined by a first interpolation process.

[0073] When the pixel to be predicted corresponds to the red channel or the blue channel, the interpolation pixel value of the pixel to be interpolated is determined by a second interpolation process.

[0074] Specifically, the first interpolation process is used to determine the green channel pixel value corresponding to the pixel to be interpolated, and the second interpolation process is used to determine the red channel pixel value corresponding to the pixel to be interpolated, or the blue channel pixel value corresponding to the pixel to be interpolated, wherein the first interpolation process and the second interpolation process can be determined based on actual conditions. For example, the first interpolation process is to determine the interpolation pixel value of the pixel to be interpolated based on the pixel value of the green channel pixel in the reference image block, and the second interpolation process is to determine the interpolation pixel value of the pixel to be interpolated based on the pixel value of the red channel pixel in the reference image block, etc., as long as the color channel corresponding to the interpolation pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted.

[0075] In one implementation, the first interpolation process specifically includes:

[0076] Select candidate pixel points pix_A, pix_B, pix_C and pix_D in the row where the pixel to be interpolated is located, select candidate pixel point pix_E in the row before the pixel to be interpolated, and select candidate pixel point pix_F in the two rows before the pixel to be interpolated;

[0077] Based on the pixel values ​​of the candidate pixel pix_A, the pixel to be interpolated, pix_D and pix_F, determine the interpolation weight coefficient corresponding to the pixel to be interpolated;

[0078] Based on the pixel values ​​of pix_B, pix_C and pix_E, and the interpolation weight coefficient, an interpolation pixel value of the pixel to be interpolated is determined, wherein the interpolation pixel value corresponds to a green channel.

[0079] Specifically, Figure 5 As shown, pix_A, pix_B, the pixel to be interpolated, pix_C and pix_D are continuous from left to right, pix_E is located directly above the pixel to be interpolated, and pix_F is located directly above pix_E, wherein the color channels corresponding to pix_A, pix_D and pix_F are the same as the color channels corresponding to the pixel to be interpolated, and the color channels corresponding to pix_B, pix_C and pix_E are different from the color channels corresponding to the pixel to be interpolated. In actual applications, pix_A, pix_D and pix_F correspond to the red channel, and pix_B, pix_C and pix_E correspond to the green channel.

[0080] The interpolation weight coefficients include the interpolation weight coefficients corresponding to pix_B, the interpolation weight coefficients corresponding to pix_C, and the interpolation weight coefficients corresponding to pix_E. Among them, the interpolation weight coefficient α corresponding to pix_B is B , the interpolation weight coefficient α corresponding to pix_C C And the interpolation weight coefficient α corresponding to pix_E E The calculation formulas are:

[0081]

[0082] Among them, w1=abs(R1-R), w2=abs(R2-R), w3=abs(R3-R), sw=w1+w2+w3, R1 represents the pixel value of pix_A, R represents the pixel value of the pixel to be interpolated, R2 represents the pixel value of pix_D, and R3 represents the pixel value of pix_F.

[0083] After the interpolation weight coefficient is obtained, the calculation formula of the interpolation pixel value GR of the pixel to be interpolated can be:

[0084]

[0085] Among them, G1 represents the pixel value of pix_B, G2 represents the pixel value of pix_C, and G3 represents the pixel value of pix_E.

[0086] In one implementation, the second interpolation process specifically includes:

[0087] Select candidate pixel points pix_G and pix_H in the row where the pixel to be interpolated is located;

[0088] Determine the interpolation pixel values ​​corresponding to pix_G and pix_H through a first interpolation process;

[0089] Based on the pixel values ​​and interpolated pixel values ​​corresponding to pix_G and pix_H, respectively, and the original pixel value of the pixel to be interpolated, the interpolated pixel value corresponding to the pixel to be interpolated is determined.

[0090] Specifically, Figure 6 As shown, pix_G, the pixel to be interpolated, and pix_H are continuous from left to right, and the color channels corresponding to pix_G and pix_H are different from the color channels of the pixel to be interpolated. The color channel corresponding to the interpolated pixel value determined by the first interpolation process is the same as the color channel of the pixel to be interpolated. For example, the pixel to be interpolated corresponds to the green channel, pix_G and pix_H both correspond to the red channel, and the interpolated pixel values ​​corresponding to pix_G and pix_H determined by the first interpolation process correspond to the green channel.

[0091] Furthermore, the color channel of the interpolation pixel value corresponding to the pixel to be interpolated is the same as the color channel of the pixel to be predicted, and the calculation formula of the interpolation pixel value corresponding to the pixel to be interpolated can be:

[0092]

[0093] Among them, RG11 represents the interpolation pixel value corresponding to the pixel to be interpolated, G11 represents the pixel value of the pixel to be interpolated, R0 represents the pixel value of pix_H, GR represents the interpolation pixel value of pix_H, R11 represents the pixel value of pix_G, and GR1 represents the interpolation pixel value of pix_G.

[0094] S30. Determine a prediction residual of the pixel to be predicted based on the interpolated reference image block, and output a bitstream based on the prediction residual.

[0095] Specifically, the interpolated reference image block is a continuous image block, where Figure 3 The reference image block corresponding to the predicted pixel point of the green channel shown in FIG. Figure 7 The reference image block shown is Figure 4 The reference image block corresponding to the predicted pixel point of the red channel shown in FIG. Figure 8 The reference image block shown, wherein the interpolation process of the reference image block corresponding to the predicted pixel point of the blue channel is the same as the interpolation process of the reference image block corresponding to the to-be-predicted pixel point corresponding to the red channel, and will not be repeated here.

[0096] And the pixel points are interpolated on the original image block, so that the data volume of the interpolated reference image block does not increase, and thus the compressed data volume will not increase. At the same time, the continuous image blocks corresponding to the pixel points to be predicted can be obtained, and the correlation between adjacent pixel points can be improved, so that the accuracy of the prediction residual can be improved, and then the compression efficiency can be improved. In one implementation, after the interpolated reference image block is obtained, the predicted pixel value corresponding to the pixel point to be predicted can be determined based on the interpolated reference image block, and then the prediction residual is determined based on the predicted pixel value and the original pixel value, and finally the prediction residual is entropy encoded to obtain a bit stream.

[0097] In one implementation, in order to improve the accuracy of the prediction residual, the predicted pixel value of the pixel to be predicted can be determined in multiple directions, and then the target pixel value is selected from the predicted pixel values ​​corresponding to each direction to determine the prediction residual. Therefore, the prediction residual of the pixel to be predicted is determined based on the interpolated reference image block, and the bitstream is output based on the prediction residual specifically includes:

[0098] For each preset direction among the plurality of preset directions, selecting a target pixel point corresponding to the pixel point to be predicted in the interpolated reference image block based on the preset direction, and determining a candidate pixel value corresponding to the pixel point to be predicted based on the target pixel point;

[0099] Determining a predicted pixel value of the pixel to be predicted according to the determined candidate pixel values;

[0100] A prediction residual is determined according to the predicted pixel value and the original pixel value of the pixel to be predicted, and a code stream is output based on the prediction residual.

[0101] Specifically, a plurality of preset directions are preset, wherein the plurality of preset directions may be determined according to an actual moment, for example, the plurality of preset directions include 0, 45, 90 and 135, or the plurality of preset directions include 0, 22.5, 45, 67.5, 90, 112.5, 135, and 157.5, etc. For each prediction direction, a target pixel point is selected in the interpolated reference image block, and then a candidate pixel value corresponding to the prediction pixel point is determined based on the selected target pixel point to obtain a plurality of candidate pixel values, wherein the number of the plurality of candidate pixel values ​​is the same as the number of the plurality of preset directions.

[0102] In one implementation, if Fig. 9 As shown, the preset directions include 8 preset directions, namely 0, 22.5, 45, 67.5, 90, 112.5, 135 and 157.5, and the calculation formulas of the candidate pixel values ​​corresponding to the preset directions can be respectively:

[0103] pred0=pix_b

[0104] pred1=(_+pix_) / 2

[0105] pred2=pix_c

[0106] pred3=(_+pix_) / 2

[0107] pred4=pix_a

[0108] pred5=(_+pix_) / 2

[0109] pred6=pix_d

[0110] pred7=pix_e

[0111] Among them, pred0 represents the candidate pixel value corresponding to direction 0, pred1 represents the candidate pixel value corresponding to direction 22.5, pred2 represents the candidate pixel value corresponding to direction 45, pred3 represents the candidate pixel value corresponding to direction 67.5, pred4 represents the candidate pixel value corresponding to direction 90, pred5 represents the candidate pixel value corresponding to direction 112.5, pred6 represents the candidate pixel value corresponding to direction 135, and pred7 represents the candidate pixel value corresponding to direction 157.5.

[0112] After obtaining each candidate pixel value, the predicted pixel value of the pixel to be predicted is calculated based on each candidate pixel value, wherein the predicted pixel value may be the average of each candidate pixel value, or may be obtained based on the interpolation of each candidate pixel value, or may be obtained by weighting each candidate pixel value. Here, only an example is given without specific limitation. This embodiment predicts pixel values ​​in different directions, and then determines the predicted pixel value of the pixel to be predicted based on the predicted candidate pixel value. In this way, fractional pixels can be used to make multi-angle predictions on the coded pixels, thereby improving the prediction accuracy of the predicted pixel value.

[0113] In summary, the present embodiment provides a method for compressing Bayer data, the method comprising obtaining a reference image block corresponding to a pixel to be predicted; obtaining a pixel to be interpolated in the reference image block, and determining the interpolated pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block; determining the prediction residual of the pixel to be predicted based on the interpolated reference image block, and outputting a code stream based on the prediction residual. The present embodiment obtains continuous reference image blocks corresponding to the pixel to be predicted, and then performs a difference on the pixels in the reference image blocks, so that the color channel corresponding to the pixel value of each pixel in the interpolated reference image block is the same as the color channel corresponding to the pixel to be predicted, which can increase the spatial correlation of adjacent pixels in the continuous reference image blocks, thereby improving the compression rate of Bayer data. In addition, the present embodiment performs pixel value interpolation on the reference image block of the pixel to be encoded during the encoding process, does not increase the amount of compressed data, and thus does not affect the real-time performance of Bayer data compression.

[0114] Based on the above-mentioned Bayer data compression method, this embodiment provides a Bayer data compression system, such as Fig.10 As shown, the system comprises:

[0115] An acquisition module 100 is used to acquire a reference image block corresponding to a pixel to be predicted, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, the reference pixels pix_c, pix_a, pix_d and pix_e are continuous and located in a preceding row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted;

[0116] An interpolation module 200 is used to obtain a pixel to be interpolated in the reference image block, and determine an interpolated pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolated pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted;

[0117] The encoding module 300 is used to determine the prediction residual of the pixel to be predicted based on the interpolated reference image block, and output a code stream based on the prediction residual.

[0118] Based on the above-mentioned bayer data compression method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the bayer data compression method as described in the above-mentioned embodiment.

[0119] Based on the above bayer data compression method, the present application also provides a terminal device, such as Fig.11 As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0120] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0121] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0122] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0123] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for compressing bayer data, characterized in that: The method includes: Obtain a reference image block corresponding to the pixel to be predicted, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, the reference pixels pix_c, pix_a, pix_d and pix_e are continuous and located in the preceding row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted; Obtaining a pixel to be interpolated in the reference image block, and determining an interpolated pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolated pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted; Determine a prediction residual of the pixel to be predicted based on the interpolated reference image block, and output a bitstream based on the prediction residual; The obtaining of the reference image block corresponding to the pixel to be predicted specifically includes: Obtain the preceding row corresponding to the pixel to be predicted, wherein when the pixel to be predicted corresponds to the green channel, the preceding row of the current row where the pixel to be predicted is located is the preceding row; when the pixel to be predicted corresponds to the red channel or the blue channel, the preceding two rows of the current row where the pixel to be predicted is located are the preceding rows; Select reference pixel points pix_c, pix_a, pix_d and pix_e in the preceding row, and select reference pixel point pix_b at the left end of the pixel point to be predicted, so as to obtain a reference image block; The step of obtaining the pixel points to be interpolated in the reference image block specifically includes: When the pixel to be predicted corresponds to the green channel, the reference pixel pix_a and the reference pixel pix_e are used as the pixel to be interpolated; When the pixel to be predicted corresponds to the red channel or the blue channel, the reference pixel pix_c and the reference pixel pix_b are used as the pixel to be interpolated.

2. The method for compressing bayer data according to claim 1, characterized in that: The determining the interpolation pixel value of the pixel to be interpolated by pixel interpolation specifically includes: When the pixel to be predicted corresponds to the green channel, an interpolation pixel value of the pixel to be interpolated is determined by a first interpolation process; When the pixel to be predicted corresponds to the red channel or the blue channel, the interpolation pixel value of the pixel to be interpolated is determined by a second interpolation process.

3. The method for compressing bayer data according to claim 2, characterized in that: The first interpolation process specifically includes: Select candidate pixel points pix_A, pix_B, pix_C and pix_D in the row where the pixel to be interpolated is located, select candidate pixel point pix_E in the row before the pixel to be interpolated, and select candidate pixel point pix_F in the two rows before the pixel to be interpolated, where pix_A, pix_B, the pixel to be interpolated, pix_C and pix_D are continuous from left to right, pix_E is located directly above the pixel to be interpolated, and pix_F is located directly above pix_E; Based on the pixel values ​​of the candidate pixel pix_A, the pixel to be interpolated, pix_D and pix_F, determine the interpolation weight coefficient corresponding to the pixel to be interpolated; Based on the pixel values ​​of pix_B, pix_C and pix_E, and the interpolation weight coefficient, an interpolation pixel value of the pixel to be interpolated is determined, wherein the interpolation pixel value corresponds to a green channel.

4. The method for compressing bayer data according to claim 3, characterized in that: The second interpolation process specifically includes: Select candidate pixel points pix_G and pix_H in the row where the pixel point to be interpolated is located, where pix_G, the pixel point to be interpolated and pix_H are continuous from left to right; Determine the interpolation pixel values ​​corresponding to pix_G and pix_H through a first interpolation process; Based on the pixel values ​​and interpolated pixel values ​​corresponding to pix_G and pix_H, respectively, and the original pixel value of the pixel to be interpolated, the interpolated pixel value corresponding to the pixel to be interpolated is determined.

5. The method for compressing bayer data according to claim 1, characterized in that: The step of determining the prediction residual of the pixel to be predicted based on the interpolated reference image block, and outputting a bitstream based on the prediction residual specifically includes: For each preset direction among the plurality of preset directions, selecting a target pixel point corresponding to the pixel point to be predicted in the interpolated reference image block based on the preset direction, and determining a candidate pixel value corresponding to the pixel point to be predicted based on the target pixel point; Determining a predicted pixel value of the pixel to be predicted according to the determined candidate pixel values; A prediction residual is determined according to the predicted pixel value and the original pixel value of the pixel to be predicted, and a code stream is output based on the prediction residual.

6. A compression system for bayer data, characterized in that: The system comprises: an acquisition module, used to acquire a reference image block corresponding to a pixel to be predicted, wherein the reference image block includes reference pixels pix_c, pix_a, pix_d, pix_e and pix_b, the reference pixels pix_c, pix_a, pix_d and pix_e are continuous and located in a preceding row of the pixel to be predicted, the reference pixel pix_a is located directly above the pixel to be predicted, and the reference pixel pix_b is located to the left of the pixel to be predicted; an interpolation module, configured to obtain a pixel to be interpolated in the reference image block, and determine an interpolation pixel value of the pixel to be interpolated by pixel interpolation to obtain an interpolated reference image block, wherein the color channel corresponding to the original pixel value of the pixel to be interpolated is different from the color channel corresponding to the pixel to be predicted, and the color channel corresponding to the interpolation pixel value of the pixel to be interpolated is the same as the color channel corresponding to the pixel to be predicted; An encoding module, configured to determine a prediction residual of the pixel to be predicted based on the interpolated reference image block, and output a bit stream based on the prediction residual; The obtaining of the reference image block corresponding to the pixel to be predicted specifically includes: Obtain the preceding row corresponding to the pixel to be predicted, wherein when the pixel to be predicted corresponds to the green channel, the preceding row of the current row where the pixel to be predicted is located is the preceding row; when the pixel to be predicted corresponds to the red channel or the blue channel, the preceding two rows of the current row where the pixel to be predicted is located are the preceding rows; Select reference pixel points pix_c, pix_a, pix_d and pix_e in the preceding row, and select reference pixel point pix_b at the left end of the pixel point to be predicted, so as to obtain a reference image block; The step of obtaining the pixel points to be interpolated in the reference image block specifically includes: When the pixel to be predicted corresponds to the green channel, the reference pixel pix_a and the reference pixel pix_e are used as the pixel to be interpolated; When the pixel to be predicted corresponds to the red channel or the blue channel, the reference pixel pix_c and the reference pixel pix_b are used as the pixel to be interpolated.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the bayer data compression method as described in any one of claims 1-5.

8. A terminal device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the processor implements the steps in the bayer data compression method according to any one of claims 1 to 5.

Images