Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: calculating first gradient values of a plurality of adjacent pixels of a to-be-coded pixel in different gradient directions, and obtaining a plurality of first gradient values; the adjacent pixels are coded pixels, and the color components corresponding to the pixels to be coded and the adjacent pixels are the same; determining a second gradient value with the minimum numerical value from the plurality of first gradient values, and determining a gradient direction corresponding to the second gradient value to obtain a target gradient direction; calculating an estimated pixel value of the to-be-coded pixel based on a pixel value of an adjacent pixel of the to-be-coded pixel in the target gradient direction; calculating a difference value between the estimated pixel value of the to-be-coded pixel and the actual pixel value of the to-be-coded pixel to obtain a target residual error of the to-be-coded pixel; the target residual error is used for encoding the pixel to be encoded. According to the scheme provided by the invention, the coding efficiency of the Bayer image data can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In the related art, the storage space and transmission bandwidth of Bayer data can be reduced by lossless compression or lossy compression of Bayer images. In the lossy compression process, the pixel information of the current coded pixel can be predicted by the pixel information of the previous or previous coded pixels, and the residual between the predicted result and the actual pixel information can be quantized to reduce the redundant information of the Bayer image data, so that the Bayer image data can be compressed by encoding the quantized result. However, when the Bayer image data is encoded using the lossy compression method of the related art, the encoding efficiency is low. Summary of the invention

[0003] The present disclosure provides an image processing method, device, electronic device and storage medium to solve the problems in the related art and improve the encoding efficiency of Bayer image data.

[0004] A first aspect of the present disclosure provides an image processing method, which includes:

[0005] Calculating first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as the adjacent pixels;

[0006] Determine a second gradient value with the smallest value from the multiple first gradient values, and determine a gradient direction corresponding to the second gradient value to obtain a target gradient direction;

[0007] Calculating an estimated pixel value of the pixel to be encoded based on pixel values ​​of adjacent pixels of the pixel to be encoded in the target gradient direction;

[0008] The difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded is calculated to obtain a target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

[0009] In the above solution, the first gradient values ​​of a plurality of adjacent pixels of the pixel to be encoded in different gradient directions are calculated to obtain a plurality of first gradient values, including:

[0010] Determine whether the color component corresponding to the pixel to be encoded is a G component;

[0011] When the color component corresponding to the pixel to be encoded is the G component, first gradient values ​​of a plurality of adjacent pixels of the pixel to be encoded in different gradient directions are calculated to obtain a plurality of first gradient values.

[0012] In the above scheme, the method further includes:

[0013] When the color component corresponding to the pixel to be encoded is an R pixel or a B pixel, the actual pixel value of the previous encoded pixel in the first encoding row is used as the estimated pixel value of the pixel to be encoded; wherein the first encoding row is the encoding row where the pixel to be encoded is located.

[0014] In the above scheme, the calculating the estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction includes one of the following:

[0015] The pixel value of the first adjacent pixel in the first coding row that was previously coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 180° direction;

[0016] The pixel value of the second adjacent pixel in the second coding row located at a 45° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 45° direction;

[0017] Calculate the average pixel value of the second adjacent pixel located at 45° direction of the pixel to be encoded and the third adjacent pixel located at 135° direction of the pixel to be encoded in the second encoding row, and use the average pixel value as the estimated pixel value of the pixel to be encoded; the target gradient direction is the 90° direction;

[0018] The pixel value of the third adjacent pixel in the second coding row located at 135° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 135° direction; wherein,

[0019] The second coding row is a previous coding row of the color component corresponding to the pixel to be coded.

[0020] In the above scheme, the method further includes:

[0021] Calculate a target k value of the pixel to be encoded based on the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and a preset coefficient;

[0022] The target k value is used to perform Golomb coding on the pixel to be coded; wherein,

[0023] The initial k value is determined according to a preset initialization k value, and different color components correspond to different initialization k values.

[0024] In the above scheme, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixel on the left side of the pixel to be encoded, the k value actually applied by the adjacent pixel on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient, including:

[0025] Adding the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient to obtain a first k value prediction result;

[0026] The first k value prediction result is shifted right by two bits to obtain the target k value of the pixel to be encoded.

[0027] In the above scheme, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixel on the left side of the pixel to be encoded, the k value actually applied by the adjacent pixel on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient, including:

[0028] Determining whether the pixel to be encoded is the first pixel to be encoded in the first encoding row;

[0029] When the pixel to be encoded is not the first pixel to be encoded in the first encoding row, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixels on the left side of the pixel to be encoded, the actually applied k value of the adjacent pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient.

[0030] In the above scheme, the method further includes:

[0031] In the case that the pixel to be encoded is the first pixel to be encoded in the first encoding row, the initial k value of the first encoding row is used as the target k value of the pixel to be encoded.

[0032] In the above scheme, the method further includes:

[0033] Determine that the initial k value of the first coding row is the initialization k value; the first coding row is the first coding row of the first color component in the current coding sub-block;

[0034] or,

[0035] The optimal k value of the reference pixel, the k value actually applied by the reference pixel, the initial k value of the second coding line and the corresponding preset coefficient are added to obtain the first k value prediction result of the reference pixel; the first k value prediction result of the reference pixel is shifted two bits to the right to obtain the second k value prediction result of the reference pixel; the second k value prediction result of the reference pixel is added to the initial k value of the second coding line to obtain the third k value prediction result of the reference pixel; the third k value prediction result is shifted one bit to the right to obtain the initialization k value of the first coding line; the first coding line is the other coding lines in the coding lines of the second color component of the current coding sub-block except the first coding line; wherein,

[0036] The first color component is the color component corresponding to the pixel to be encoded, and the reference pixel is the last reference pixel to be encoded in the second encoding row.

[0037] A second aspect of the present disclosure provides an image processing device, the device comprising:

[0038] A first processing unit is used to calculate first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as the adjacent pixels;

[0039] a determining unit, configured to determine a second gradient value having a minimum value from among the plurality of first gradient values, and determine a gradient direction corresponding to the second gradient value, to obtain a target gradient direction;

[0040] A second processing unit, configured to calculate an estimated pixel value of the pixel to be encoded based on pixel values ​​of adjacent pixels of the pixel to be encoded in the target gradient direction;

[0041] The third processing unit is used to calculate the difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded to obtain the target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

[0042] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect embodiment of the present disclosure.

[0043] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0044] A fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which executes the method described in the first aspect embodiment of the present disclosure when a processor is used to execute the computer program.

[0045] The sixth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect embodiment of the present disclosure.

[0046] In summary, the image processing method, device, electronic device and storage medium proposed in the present invention can improve the accuracy of the prediction result by calculating the gradient changes of adjacent pixels in different directions, predicting the pixel changes of the current pixel to be encoded in different directions, and using the pixel value in the direction with the smallest gradient to predict the pixel value of the current pixel to be encoded. Since the prediction result can more accurately reflect the distribution of redundant information around the current pixel to be encoded, it can improve the accuracy of redundant information processing, thereby improving the encoding efficiency of Bayer image data.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0049] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0050] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0051] Figure 3 A schematic diagram of a Bayer image in the related art;

[0052] Figure 4 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0053] Figure 5 Schematic diagram of the gradient of G pixel in different gradient directions in the embodiment of the present disclosure;

[0054] Figure 6 A schematic diagram of predicting the pixel values ​​of R pixels and B pixels in an embodiment of the present disclosure;

[0055] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0056] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0057] Fig. 9 A schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0059] The Bayer format is a data format in which one R pixel, one B pixel and two G pixels are arranged in a fixed pattern. The pixels of the same color component are not arranged continuously, resulting in low compression efficiency of Bayer image data. Therefore, how to improve the compression efficiency of Bayer images is an urgent problem to be solved.

[0060] In the related technology, the compression methods of Bayer image data mainly include lossless compression and lossy compression. Among them, lossless compression is to compress data, which can maintain the integrity and accuracy of the data while reducing the storage space or transmission bandwidth of the data, that is, the original data can be completely restored after decompression; while lossy compression is to reduce the storage space or transmission bandwidth of the data by discarding the accuracy or details of some data.

[0061] Among them, in the lossy compression process, the pixel value of the current coding pixel is often predicted by the pixel value of the previous coding pixel or the previous few coding pixels to obtain the residual between the prediction result and the actual pixel value, and then the residual of each pixel in the Bayer data is quantized to reduce redundant information. Exemplarily, as a commonly used lossy compression method, the Joint Photographic Experts Group (JPEG) lossy compression process includes: dividing the image into blocks and performing discrete cosine transform (DCT), quantizing the DCT coefficients of each pixel, and in the quantization process, the coefficients of the high-frequency part are reduced and discarded, and the coefficients of the lower-frequency part are retained; then, entropy coding (such as Huffman coding) is used to encode the quantized coefficients to achieve compression.

[0062] However, when using the previous one or several previous encoded pixel values ​​to predict the current encoded pixel value, the statistical characteristics of the redundant information around the pixel to be encoded are not taken into account, so the prediction result accuracy is low; for example, for positions where the image data changes greatly, the data difference between the two previous and next pixels is large. At this time, if the pixel value of the previous pixel is still used as a guide to predict the pixel value of the current pixel, the prediction result accuracy will be low, resulting in inaccurate residual results, affecting the accuracy of the quantization results, and thus reducing the encoding efficiency of the Bayer data.

[0063] In order to solve the problems existing in the related art, the present disclosure proposes an image processing method, device, electronic device and storage medium, which can improve the accuracy of the prediction result by calculating the gradient changes of adjacent pixels in different directions, predicting the pixel changes of the current pixel to be encoded in different directions, and using the pixel value in the direction with the smallest gradient to predict the pixel value of the current pixel to be encoded. Since the prediction result can more accurately reflect the distribution of redundant information around the current pixel to be encoded, it can improve the accuracy of redundant information processing, thereby improving the encoding efficiency of Bayer image data.

[0064] Figure 1 FIG. 1 is an application scenario diagram of an image processing method in an embodiment. Figure 1 As shown, the application scenario includes an electronic device 104, in which a camera module and several applications can be installed. The application can initiate an image acquisition instruction to acquire an image, and the camera module acquires the image 102. Among them, the camera module can include a front camera module and / or a rear camera module. Finally, the target image is sent to the target application. Among them, the electronic device 104 can be a smart phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0065] In some optional embodiments, the electronic device may also be a vehicle-mounted device or a vehicle networking device, such as a smart car, etc. In the present disclosure, only a smart phone is taken as an example, but it does not limit the scope of the present disclosure.

[0066] A camera can be installed on an electronic device, and images can be acquired through the installed camera. Cameras can be divided into laser cameras, visible light cameras and other types according to the different images acquired. Laser cameras can acquire images formed by irradiating lasers onto objects, and visible light cameras can acquire images formed by irradiating visible light onto objects. Several cameras can be installed on an electronic device, and the installation locations are not limited. For example, a camera can be installed on the front panel of an electronic device, and two cameras can be installed on the back panel. The camera can also be installed inside the electronic device in an embedded manner, and then the camera can be opened by rotating or sliding. Specifically, a front camera and a rear camera can be installed on an electronic device. The front camera and the rear camera can acquire images from different perspectives. Generally, the front camera can acquire images from the front perspective of the electronic device, and the rear camera can acquire images from the back perspective of the electronic device.

[0067] It should be understood that in the present disclosure, the front camera or the rear camera is only used as an example to distinguish the shooting angles of different cameras, rather than to limit the functions of multiple cameras. The multiple cameras in the present disclosure can be rear cameras or front cameras at the same time, which is not limited in the present disclosure.

[0068] Several applications can be installed in electronic devices. Applications refer to software written for a certain application purpose in electronic devices. Electronic devices can provide services for users through applications. When an application needs to capture an image, it will initiate an image capture instruction. The electronic device will call the camera module according to the image capture instruction to capture the image. Image capture instructions refer to instructions used to trigger image capture operations.

[0069] The electronic device is also provided with a processor, and the image processing module in the processor can perform correction processing on the image captured by the camera module, for example, execute the image processing method provided in the present disclosure.

[0070] Figure 2 An image processing method is provided in an embodiment of the present disclosure. Figure 2 As shown, the method may include:

[0071] Step 201: Calculate first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as that of the adjacent pixels.

[0072] In one embodiment of the present disclosure, the number of G pixels is twice the number of R pixels or the number of B pixels, and the G pixels in two adjacent coding rows are arranged in an interlaced manner. Figure 3As shown, when predicting the pixel value of each G pixel, there is more reference information available for prediction. For example, the pixel to the left, the pixel to the upper left, and the pixel to the upper right of each G pixel can be used to predict the pixel value of the G pixel. For RB pixels, only the pixel to the left of each RB pixel has reference significance. The encoded RB pixels at other positions have little reference significance due to their far distance. Based on this, in some embodiments, when the pixel to be encoded is a G pixel, the first gradient values ​​of multiple adjacent pixels of the pixel to be encoded in different gradient directions can be calculated to obtain multiple first gradient values.

[0073] In some embodiments, the first adjacent pixel on the left side, the second adjacent pixel on the upper left side, and the third adjacent pixel on the upper right side of the pixel to be encoded may be determined first. Specifically, the G pixel on the left side of the pixel to be encoded in the current encoding row is the first adjacent pixel, the G pixel on the upper left side of the pixel to be encoded and closest to the pixel to be encoded in the previous encoding row is the second adjacent pixel, and the G pixel on the upper right side of the pixel to be encoded and closest to the pixel to be encoded in the previous encoding row is the third adjacent pixel; then, the gradient values ​​of the first adjacent pixel in different gradient directions, as well as the gradient values ​​of the second adjacent pixel and the third adjacent pixel in different gradient directions are calculated. The gradient values ​​of the neighboring pixels can be used to determine the gradient direction with the smallest value change, and the pixel value of the pixel to be encoded can be estimated according to the G pixel value of the gradient direction with the smallest value change; specifically, the gradient values ​​of the first neighboring pixel in the 135° direction, the 90° direction and the 45° direction can be calculated, and the gradient values ​​of the second neighboring pixel and the third neighboring pixel can be calculated, so as to determine the change of the pixel values ​​of the encoded pixels around the pixel to be encoded, and the pixel value of the current pixel to be encoded can be estimated according to the change of the pixel values ​​of the surrounding encoded pixels by using the pixel value of the gradient direction with the smallest pixel value change.

[0074] Step 202: Determine a second gradient value with the smallest value from the multiple first gradient values, and determine a gradient direction corresponding to the second gradient value to obtain a target gradient direction.

[0075] Step 203: Calculate an estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction.

[0076] In some embodiments, when the gradient value of the first adjacent pixel of the pixel to be encoded in the 135° direction is the smallest, the pixel value of the adjacent pixel to the upper left of the pixel to be encoded can be used as the estimated pixel value of the pixel to be encoded; when the gradient value of the first adjacent pixel of the pixel to be encoded in the 45° direction is the smallest, the pixel value of the adjacent pixel to the upper right of the pixel to be encoded can be used as the estimated pixel value of the pixel to be encoded; when the gradient value of the first adjacent pixel of the pixel to be encoded in the 90° direction is the smallest, the pixel average of the adjacent pixel to the upper left of the pixel to be encoded and the adjacent pixel to the upper right of the pixel to be encoded can be used as the estimated pixel value of the pixel to be encoded; when the gradient value of the G pixel above the pixel to be encoded in the 180° direction is the smallest, the pixel value of the first adjacent pixel of the pixel to be encoded can be used as the estimated pixel value of the pixel to be encoded.

[0077] It should be pointed out that the gradient value between pixels of the same color component can reflect the distribution characteristics of redundant information. Therefore, by selecting the pixel value of the pixel to be encoded in the direction of the minimum gradient value to estimate the pixel value of the pixel to be encoded, the accuracy of the prediction result can be improved, thereby improving the accuracy of the residual calculation result.

[0078] Step 204: Calculate the difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded to obtain a target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

[0079] In some embodiments, the target residual of each pixel in the current coding unit is calculated and quantized to discard high-frequency data, thereby reducing the amount of data processing in the coding process and improving coding efficiency.

[0080] In summary, the image processing method proposed in the present invention can improve the accuracy of the prediction result by calculating the gradient changes of adjacent pixels in different directions, predicting the pixel changes of the current pixel to be encoded in different directions, and using the pixel value in the direction with the smallest gradient to predict the pixel value of the current pixel to be encoded. Since the prediction result can more accurately reflect the distribution of redundant information around the current pixel to be encoded, it can improve the accuracy of redundant information processing, thereby improving the encoding efficiency of Bayer image data.

[0081] Figure 4 An image processing method is provided in an embodiment of the present disclosure. Figure 4 As shown, the method may include:

[0082] Step 401: When the color component corresponding to the pixel to be encoded is the G component, first gradient values ​​of multiple adjacent pixels of the pixel to be encoded in different gradient directions are calculated to obtain multiple first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as that of the adjacent pixels.

[0083] In some embodiments, it may also be determined whether the color component corresponding to the pixel to be encoded is a G component.

[0084] In practical applications, since the number of G pixels is twice the number of R pixels or B pixels, the G pixels in two adjacent coding rows are arranged alternately, such as Figure 5 As shown, when predicting the pixel value of each G pixel, more reference information can be used for prediction. For example, the pixel on the left, the pixel on the upper left, and the pixel on the upper right of each G pixel can be used to predict the pixel value of the G pixel. For RB pixels, only the pixel on the left of each RB pixel has reference significance, and the encoded RB pixels at other positions have little reference significance due to their farther positions. Therefore, when the pixel to be encoded is a G pixel, by calculating the first gradient values ​​of multiple adjacent pixels of the pixel to be encoded in different gradient directions, the data change of the pixels around the pixel to be encoded can be determined, and according to the change of the surrounding pixel values, the direction in which the pixel value is closest can be determined, so that the estimated pixel value of the pixel to be encoded can be calculated according to the pixel value in the direction in which the pixel value is closest, thereby improving the accuracy of the prediction result.

[0085] In some embodiments, the adjacent pixels of the pixel to be encoded may include the pixels at the upper left corner and the upper right corner of the pixel to be encoded, that is, the G pixel closest to the pixel to be encoded in the previous encoding row, and may also include the G pixel to the left of the pixel to be encoded; the pixels may also be called components, and the pixel to be encoded may also be called components to be encoded.

[0086] In some embodiments, when calculating the first gradient values ​​of multiple adjacent pixels of a pixel to be encoded in different gradient directions, the gradient values ​​of the first adjacent pixel of the pixel to be encoded in the directions of 135°, 90°, and 45° can be calculated, and the gradient value between the two G pixels closest to the pixel to be encoded in the previous encoding row of the current encoding row can also be calculated; wherein, the gradient value of the first adjacent pixel in the 135° direction can be the absolute value of the image value of the G pixel to the upper left of the first adjacent pixel and the first adjacent pixel, the gradient value of the first adjacent pixel in the 90° direction can be the absolute value of the average of the G pixel to the upper left and the G pixel to the upper right of the first adjacent pixel and the pixel difference of the first adjacent pixel, and the gradient value of the first adjacent pixel in the 45° direction can be the absolute value of the pixel difference between the G pixel in the upper right corner of the first adjacent pixel and the first adjacent pixel.

[0087] For example, Figure 5 As shown, Current_G is the G pixel to be encoded currently, and the gradient calculation formulas in different directions can be expressed as:

[0088]

[0089] Here, Gi represents the pixel value of pixel Gi, and abs(*) represents finding the absolute value.

[0090] Step 402: Determine a second gradient value with the smallest value from the multiple first gradient values, and determine a gradient direction corresponding to the second gradient value to obtain a target gradient direction.

[0091] In some embodiments, the target gradient direction may be 180°, 135°, 90°, or 45°, that is, above, in the upper left corner, or in the upper right corner of the pixel to be encoded.

[0092] Step 403: Calculate an estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction.

[0093] Specifically, the gradient values ​​of the first adjacent pixel of the pixel to be encoded in the 135° direction, the 90° direction and the 45° direction can be calculated, and the gradient value between the second adjacent pixel and the third adjacent pixel can be calculated, so as to judge the change of the pixel values ​​of the encoded pixels around the pixel to be encoded, and the pixel value of the current pixel to be encoded can be estimated according to the change of the pixel values ​​of the surrounding encoded pixels by using the pixel value in the gradient direction with the smallest pixel value change.

[0094] Based on this, in some embodiments, calculating the estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction includes one of the following:

[0095] The pixel value of the first adjacent pixel in the first coding row that was previously coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 180° direction;

[0096] The pixel value of the second adjacent pixel in the second coding row located at a 45° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 45° direction;

[0097] Calculate the average pixel value of the second adjacent pixel located at 45° direction of the pixel to be encoded and the third adjacent pixel located at 135° direction of the pixel to be encoded in the second encoding row, and use the average pixel value as the estimated pixel value of the pixel to be encoded; the target gradient direction is the 90° direction;

[0098] The pixel value of the third adjacent pixel in the second coding row located at 135° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 135° direction; wherein,

[0099] The second coding row is a previous coding row of the color component corresponding to the pixel to be coded.

[0100] For example, Figure 5 As shown, Current_G is the G pixel to be encoded currently, and the minimum gradient value dmin can be expressed as:

[0101]

[0102] Among them, dmin represents the minimum gradient, and MIN(x,y) represents the minimum value of x and y;

[0103] The estimated pixel value of Current_G can be expressed as:

[0104]

[0105] Among them, pred represents the estimated pixel value, which can also be called the predicted value.

[0106] It should be noted that, when the color component corresponding to the pixel to be encoded is not the G component, that is, the current pixel to be encoded is an R pixel or a B pixel, the estimated pixel value of the pixel to be encoded can be directly predicted using the pixel information of the adjacent pixels.

[0107] Based on this, in some embodiments, the method may further include:

[0108] In the case where the color component corresponding to the pixel to be encoded is an R pixel or a B pixel, the actual pixel value of the last encoded pixel in the first encoding row is used as the estimated pixel value of the pixel to be encoded.

[0109] In some embodiments, the actual pixel value may also be referred to as the original pixel value.

[0110] For example, Figure 6 As shown, the actual pixel value of the left R pixel can be used as the estimated pixel value of the right R pixel, and the actual pixel value of the left B pixel can be used as the estimated pixel value of the right B pixel.

[0111] It should be noted that when the pixel to be encoded is the first encoded pixel of the same color component in the current encoding row, the pixel value of the current pixel can be predicted based on the pixel value of the previous encoding row of the same color component. For example, the pixel value of the current pixel can be predicted using the pixel mean of the previous encoding row of the same color component. For another example, the pixel value of the current pixel can be predicted based on the pixel values ​​of the same color component in a preset window including the current pixel to be encoded. For another example, the pixel value of the current pixel can be predicted based on the pixel values ​​adjacent to the current pixel in the previous encoding row of the same color component.

[0112] Step 404: Calculate the difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded to obtain a target residual of the pixel to be encoded.

[0113] In some embodiments, after obtaining the target residual of the pixel to be encoded, the obtained target residual needs to be quantized to reduce the residual of the high-frequency part; then the quantized residual is mapped to an absolute value to obtain coefficients to be encoded that are all positive numbers.

[0114] Exemplarily, the coefficient to be encoded can be expressed using the following formula:

[0115]

[0116] Among them, diff represents the residual, and diff_map represents the coefficient to be encoded.

[0117] Step 405: Calculate the target k value of the pixel to be encoded based on the optimal k value of the adjacent pixels on the left side of the pixel to be encoded, the actually applied k value of the adjacent pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient; wherein the initial k value is determined according to a preset initialization k value, and the initialization k values ​​corresponding to different color components are different.

[0118] In some embodiments, the optimal k value of the left neighboring pixel may also be determined.

[0119] Based on this, in some embodiments, the method may further include:

[0120] Based on the target residual of the left adjacent pixel of the pixel to be encoded, the optimal k value of the left adjacent pixel is calculated.

[0121] It should be noted that the left adjacent pixel may be the last encoded pixel in the current encoding row having the same color component as the current pixel to be encoded.

[0122] For example, the optimal k value can be expressed by the following formula:

[0123] rb_best_k=log2(diff_map+1), the diff_map position is the R / B component;

[0124] g_best_k=log2(diff_map+1), the diff_map position is the G component;

[0125] Among them, rb_best_k represents the optimal k value of the R pixel or B pixel to be encoded currently, and g_best_k represents the optimal k value of the G pixel to be encoded currently.

[0126] In actual application, for pixels with the same color components in a unified coding row, the target k value of the current pixel can be predicted by calculating the optimal k value of the previous coded pixel, or the target k value of the next coded pixel can be predicted by calculating the optimal k value of the pixel; the optimal k value can also be called the best k value.

[0127] In some embodiments, the calculating the target k value of the pixel to be encoded based on the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient includes:

[0128] Adding the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient to obtain a first k value prediction result;

[0129] The first k value prediction result is shifted right by two bits to obtain the target k value of the pixel to be encoded.

[0130] For example, the target k value may be expressed by the following formula:

[0131] rb_updata_k=(best_k+k+k_init+coeff)>>2, the current encoding position is the R / B component;

[0132] g_updata_k=(best_k+k+k_init+coeff)>>2, the current encoding position is the G component;

[0133] Among them, rb_updata_k represents the target k value of the R pixel or B pixel to be encoded, g_updata_k represents the target k value of the G pixel to be encoded, best_k represents the optimal k value of the left adjacent pixel, k represents the actual k value of the left adjacent pixel, k_init represents the initialization k value of the corresponding color component, and coeff represents the preset coefficient.

[0134] In some embodiments, the preset coefficient may be a fixed coefficient, the value range of the preset coefficient may be 0 to 4, and the preset coefficients of different color components may be the same or different. In actual application, the specific preset coefficient to be used may be determined according to application requirements, and the embodiments of the present disclosure do not limit this.

[0135] In some embodiments, for the first pixel to be encoded in each coding unit, due to the lack of reference information for prediction, the k value can be initialized according to the component, that is, different initialization k values ​​are defined for the pixel according to different color components for encoding the pixel; wherein the numerical value of the initialization k value of each color component can be determined based on the compression rate and content repetition of the Bayer data.

[0136] In actual application, for each coding row, since the first coded pixel lacks reference information of the left pixel, that is, lacks the optimal k value and actual pixel of the left coded pixel, it is possible to first determine whether the pixel to be coded is the first coded pixel of the current coding row, and use different methods to predict the target k values ​​of the first coded pixel and other coded pixels.

[0137] Based on this, in some embodiments, the step of calculating the target k value of the pixel to be encoded based on the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row, and the preset coefficient includes:

[0138] Determining whether the pixel to be encoded is the first pixel to be encoded in the first encoding row;

[0139] When the pixel to be encoded is not the first pixel to be encoded in the first encoding row, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixels on the left side of the pixel to be encoded, the actually applied k value of the adjacent pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient.

[0140] In some embodiments, a different initial k value may be set for each coding row, and the initial k value may be used to encode the first coding pixel of the corresponding coding row.

[0141] Based on this, in some embodiments, the method may further include:

[0142] In the case that the pixel to be encoded is the first pixel to be encoded in the first encoding row, the initial k value of the first encoding row is used as the target k value of the pixel to be encoded.

[0143] In practical applications, the initial k value of the next coding row can be calculated by using the target k value of the last coded pixel in the previous coding row with the same color component as the initial k value.

[0144] Based on this, in some embodiments, the method further includes:

[0145] Determine that the initial k value of the first coding row is the initialization k value; the first coding row is the first coding row of the first color component in the current coding sub-block;

[0146] or,

[0147] The optimal k value of the reference pixel, the k value actually applied by the reference pixel, the initial k value of the second coding line and the corresponding preset coefficient are added to obtain the first k value prediction result of the reference pixel; the first k value prediction result of the reference pixel is shifted two bits to the right to obtain the second k value prediction result of the reference pixel; the second k value prediction result of the reference pixel is added to the initial k value of the second coding line to obtain the third k value prediction result of the reference pixel; the third k value prediction result is shifted one bit to the right to obtain the initialization k value of the first coding line; the first coding line is the other coding lines in the coding lines of the second color component of the current coding sub-block except the first coding line; wherein,

[0148] The first color component is the color component corresponding to the pixel to be encoded, and the reference pixel is the last reference pixel to be encoded in the second encoding row.

[0149] Step 406: Perform Golomb coding on the pixel to be coded using the target k value.

[0150] In some embodiments, when the pixel to be encoded is encoded using the target k value, the coefficient to be encoded of the pixel to be encoded may be encoded using the target k value.

[0151] It should be pointed out that after executing step 403, in the process of encoding the target residual, different encoding methods can be used for different situations. Specifically, fixed-length encoding methods and variable-length encoding methods can be used for entropy encoding, such as Golomb entropy encoding belonging to variable-length encoding and direct difference data transport (DTS) entropy encoding belonging to fixed-length encoding. Different encoding methods are applicable to different application scenarios. Therefore, in the related art, these two encoding modes are often traversed to select a mode with a smaller number of encoding bits as the final encoding mode of the current encoding unit. In the embodiment of the present disclosure, steps 404 to 405 are steps of performing variable-length encoding when the number of encoding bits of the variable-length encoding mode is smaller, specifically, the encoding process of performing Golomb encoding. Accordingly, when the number of encoding bits of the fixed-length encoding mode is smaller, that is, when performing DTS encoding, the maximum coefficient to be encoded of the current encoding unit can be calculated, and all pixels in the current encoding unit can be encoded according to the bit length required by the maximum coefficient to be encoded.

[0152] In summary, the image processing method proposed in the present invention can improve the accuracy of the prediction result by calculating the gradient changes of adjacent pixels in different directions, predicting the pixel changes of the current pixel to be encoded in different directions, and using the pixel value in the direction with the smallest gradient to predict the pixel value of the current pixel to be encoded. Since the prediction result can more accurately reflect the distribution of redundant information around the current pixel to be encoded, it can improve the accuracy of redundant information processing, thereby improving the encoding efficiency of Bayer image data.

[0153] The technical solution of the present disclosure is further described in detail below in conjunction with specific application examples.

[0154] The present application embodiment provides a Bayer data lossy compression method, which may include:

[0155] Step 1: Divide the input Bayer image into multiple sub-blocks, and then perform step 2.

[0156] Here, one sub-block is one coding unit.

[0157] Step 2: Perform adaptive prediction on the sub-block to obtain the predicted pixel value, and then proceed to step 3.

[0158] Among them, for G pixels, such as Figure 5 As shown, an angle adaptive prediction method is adopted. Specifically, the gradient values ​​of adjacent G components at different angles are calculated, and the G component reconstruction value corresponding to the direction with the smallest gradient is used as the prediction value of the current G pixel. The gradient calculation formulas at different angles are as follows:

[0159]

[0160] Among them, abs(*) means finding the absolute value, and MIN(x,y) means finding the minimum value of x and y.

[0161] After calculating the minimum gradient, the prediction value of the current coded G pixel can be calculated according to the direction corresponding to the minimum gradient; specifically, the prediction value is calculated as follows:

[0162]

[0163] Among them, pred represents the predicted value.

[0164] For R pixels and B pixels, the left prediction method is used, that is, the same color component that has been encoded and reconstructed on the left is taken as the prediction value, as shown in the following example: Figure 6 shown.

[0165] Among them, for the first encoded component in each encoded row of same-color components, that is, the first encoded pixel, the pixel value of the current component can be predicted based on the same-color components in the previous row. For example, the pixel value of the current component can be predicted using the mean of the same-color components in the previous row. For another example, the pixel value of the current component can be predicted based on the pixel values ​​of the same-color components in a preset window including the current component.

[0166] Step 3: Subtract the predicted pixel value from the original pixel to get the residual, and then perform step 4.

[0167] Specifically, the original pixel value is subtracted from the predicted value to obtain the residual diff, and the calculation formula is as follows:

[0168] diff = pixel – pred;

[0169] Among them, pixel represents the original pixel value.

[0170] Step 4: quantize the residual obtained after prediction, map the absolute value of the quantized residual to obtain the coefficients to be encoded diff_map which are all positive numbers, and then execute step 5.

[0171] The coefficients to be encoded can be expressed using the following formula:

[0172]

[0173] Step 5: Columbus entropy coding and DTS entropy coding are respectively used for diff_map, and the two coding modes are traversed. The mode with the smaller number of coding bits is selected as the final coding mode of the current coding unit, and then step 6 is executed.

[0174] The present disclosure uses Golomb entropy coding in the application embodiment.

[0175] Step 6: Initialize k value according to the first pixel of the first row in the current coding unit of the component;

[0176] Among them, the initialization k of the RB component is equal to rb_k_init, and the initialization k of the G component is equal to g_k_init. rb_k_init and g_k_init are different, and the initialization k value can be adjusted according to the actual content.

[0177] Step 7: Based on the initial k value, predict the k value of each component in the corresponding component.

[0178] Here, since the initialization k values ​​of different color components are different, the k values ​​used in the process of using the previous component to predict the k value of the next component will also be different; here, we can first determine whether the component corresponding to the current encoded diff_map is R / B or G, and then perform Golomb encoding on the diff_map according to the k value of the corresponding component.

[0179] Specifically, find the optimal k value based on the current diff_map:

[0180] rb_best_k=log2(diff_map+1), if the diff_map position is the R / B component;

[0181] g_best_k=log2(diff_map+1), if the diff_map position is the G component;

[0182] Then, the k value of the next diff_map to be encoded is calculated based on the current optimal k value, the actual k value used, and the initial k value:

[0183] rb_updata_k=(rb_best_k+k+k_init+coeff)>>2, if the current encoding position is the R / B component;

[0184] g_updata_k=(g_best_k+k+k_init+coeff)>>2, if the current encoding position is the G component;

[0185] Among them, coeff is a fixed coefficient.

[0186] Then, determine whether to encode to the next line, if yes, execute step 8, if not, set k=updata_k, and execute step 7 to encode the next component;

[0187] Step 8: Reinitialize the k value, that is, use the initial k value of the current coding row and the target k value of the last coding row in the current coding row to calculate the initial k value of the next coding row, and use the calculated initial k value as the k value of the first component in the corresponding coding row; the calculation formula is as follows:

[0188] rb_k_init=(rb_k_init+rb_updata_k)>>1;

[0189] g_k_init=(g_k_init+g_updata_k)>>1.

[0190] Step 9: Determine whether the current block has been encoded. If so, end the current update. If not, execute step 7.

[0191] In summary, the solution provided by the application embodiment of the present disclosure can make full use of the spatial redundancy characteristics of pixels, adaptively predict pixel values, improve the accuracy of prediction results, and effectively reduce the amount of data to be encoded; at the same time, through the adaptive K value update method, it can effectively predict the K value of the next value to be encoded, thereby improving the effectiveness of Columbus coding.

[0192] In order to implement the above-mentioned image processing method, the embodiment of the present disclosure also provides an image processing device. Figure 7 As shown, the image processing device 700 includes:

[0193] The first processing unit 701 is used to calculate first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as the adjacent pixels;

[0194] A determining unit 702 is configured to determine a second gradient value having a minimum value from among the plurality of first gradient values, and determine a gradient direction corresponding to the second gradient value to obtain a target gradient direction;

[0195] A second processing unit 703 is used to calculate an estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction;

[0196] The third processing unit 704 is used to calculate the difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded to obtain a target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

[0197] In some embodiments, the first processing unit 701 may be specifically configured to:

[0198] Determine whether the color component corresponding to the pixel to be encoded is a G component;

[0199] When the color component corresponding to the pixel to be encoded is the G component, first gradient values ​​of a plurality of adjacent pixels of the pixel to be encoded in different gradient directions are calculated to obtain a plurality of first gradient values.

[0200] In some embodiments, the first processing unit 701 may also be used to:

[0201] When the color component corresponding to the pixel to be encoded is an R pixel or a B pixel, the actual pixel value of the previous encoded pixel in the first encoding row is used as the estimated pixel value of the pixel to be encoded; wherein the first encoding row is the encoding row where the pixel to be encoded is located.

[0202] In some embodiments, the second processing unit 703 is specifically used for one of the following:

[0203] The pixel value of the first adjacent pixel in the first coding row that was previously coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 180° direction;

[0204] The pixel value of the second adjacent pixel in the second coding row located at a 45° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 45° direction;

[0205] Calculate the average pixel value of the second adjacent pixel located at 45° direction of the pixel to be encoded and the third adjacent pixel located at 135° direction of the pixel to be encoded in the second encoding row, and use the average pixel value as the estimated pixel value of the pixel to be encoded; the target gradient direction is the 90° direction;

[0206] The pixel value of the third adjacent pixel in the second coding row located at 135° direction of the pixel to be coded is used as the estimated pixel value of the pixel to be coded; the target gradient direction is the 135° direction; wherein,

[0207] The second coding row is a previous coding row of the color component corresponding to the pixel to be coded.

[0208] In some embodiments, the apparatus may further include a fourth processing unit, wherein the fourth processing unit is configured to:

[0209] Calculate a target k value of the pixel to be encoded based on the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and a preset coefficient;

[0210] The target k value is used to perform Golomb coding on the pixel to be coded; wherein,

[0211] The initial k value is determined according to a preset initialization k value, and different color components correspond to different initialization k values.

[0212] In some embodiments, the fourth processing unit may be specifically configured to:

[0213] Adding the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient to obtain a first k value prediction result;

[0214] The first k value prediction result is shifted right by two bits to obtain the target k value of the pixel to be encoded.

[0215] In some embodiments, the fourth processing unit may be specifically configured to:

[0216] Determining whether the pixel to be encoded is the first pixel to be encoded in the first encoding row;

[0217] When the pixel to be encoded is not the first pixel to be encoded in the first encoding row, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixels on the left side of the pixel to be encoded, the actually applied k value of the adjacent pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient.

[0218] In some embodiments, the fourth processing unit may also be used to:

[0219] In the case that the pixel to be encoded is the first pixel to be encoded in the first encoding row, the initial k value of the first encoding row is used as the target k value of the pixel to be encoded.

[0220] In some embodiments, the fourth processing unit may also be used to:

[0221] Determine that the initial k value of the first coding row is the initialization k value; the first coding row is the first coding row of the first color component in the current coding sub-block;

[0222] or,

[0223] The optimal k value of the reference pixel, the k value actually applied by the reference pixel, the initial k value of the second coding line and the corresponding preset coefficient are added to obtain the first k value prediction result of the reference pixel; the first k value prediction result of the reference pixel is shifted two bits to the right to obtain the second k value prediction result of the reference pixel; the second k value prediction result of the reference pixel is added to the initial k value of the second coding line to obtain the third k value prediction result of the reference pixel; the third k value prediction result is shifted one bit to the right to obtain the initialization k value of the first coding line; the first coding line is the other coding lines in the coding lines of the second color component of the current coding sub-block except the first coding line; wherein,

[0224] The first color component is the color component corresponding to the pixel to be encoded, and the reference pixel is the last reference pixel to be encoded in the second encoding row.

[0225] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0226] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0227] Figure 8 8 is a block diagram of an electronic device 800 for implementing the above-mentioned image processing method according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0228] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0229] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0230] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0231] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0232] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0233] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0234] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0235] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0236] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0237] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0238] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0239] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image processing method described in the above embodiment of the present disclosure.

[0240] An embodiment of the present disclosure further provides a computer program product, including a computer program, which executes the image processing method described in the above embodiment of the present disclosure when a processor is used.

[0241] The embodiment of the present disclosure also provides a chip, which can be seen in Fig. 9 Schematic diagram of the chip structure shown. Fig. 9 The chip shown includes a processor 901 and an interface circuit 902. The number of the processor 901 can be one or more, and the number of the interface circuit 902 can be one or more.

[0242] Optionally, the chip also includes a memory 903, which is used to store necessary computer programs and data; the interface circuit 902 is used to receive signals from the memory 903 and send signals to the processor 901, and the signals include computer instructions stored in the memory 903. When the processor 901 executes the computer instructions, the electronic device executes the image processing method described in the above embodiments of the present disclosure.

[0243] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0244] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0245] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0246] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0247] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0248] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0249] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0250] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises Calculating first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as the adjacent pixels; Determine a second gradient value with the smallest value from the multiple first gradient values, and determine a gradient direction corresponding to the second gradient value to obtain a target gradient direction; Calculating an estimated pixel value of the pixel to be encoded based on pixel values ​​of adjacent pixels of the pixel to be encoded in the target gradient direction; The difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded is calculated to obtain a target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

2. The method according to claim 1, characterized in that: The step of calculating first gradient values ​​of a plurality of adjacent pixels of the pixel to be encoded in different gradient directions to obtain a plurality of first gradient values ​​comprises: Determine whether the color component corresponding to the pixel to be encoded is a G component; When the color component corresponding to the pixel to be encoded is the G component, first gradient values ​​of a plurality of adjacent pixels of the pixel to be encoded in different gradient directions are calculated to obtain a plurality of first gradient values.

3. The method according to claim 2, characterized in that The method further comprises: When the color component corresponding to the pixel to be encoded is an R pixel or a B pixel, the actual pixel value of the previous encoded pixel in the first encoding row is used as the estimated pixel value of the pixel to be encoded; wherein the first encoding row is the encoding row where the pixel to be encoded is located.

4. The method according to claim 3, characterized in that The calculating the estimated pixel value of the pixel to be encoded based on the pixel values ​​of the neighboring pixels of the pixel to be encoded in the target gradient direction includes one of the following: Using the pixel value of the last encoded first adjacent pixel in the first encoding row as the estimated pixel value of the pixel to be encoded; The target gradient direction is 180° direction; Taking the pixel value of the second adjacent pixel in the second coding row located at a direction of 45° to the pixel to be coded as the estimated pixel value of the pixel to be coded; The target gradient direction is 45° direction; Calculate the average pixel value of the second adjacent pixel located at 45° direction of the pixel to be encoded and the third adjacent pixel located at 135° direction of the pixel to be encoded in the second encoding row, and use the average pixel value as the estimated pixel value of the pixel to be encoded; The target gradient direction is a 90° direction; Using the pixel value of the third adjacent pixel in the second coding row and located at 135° in the direction of the pixel to be coded as the estimated pixel value of the pixel to be coded; The target gradient direction is 135° direction; wherein, The second coding row is a previous coding row of the color component corresponding to the pixel to be coded.

5. The method according to claim 4, characterized in that The method further comprises: Calculate a target k value of the pixel to be encoded based on the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and a preset coefficient; The target k value is used to perform Golomb coding on the pixel to be coded; wherein, The initial k value is determined according to a preset initialization k value, and different color components correspond to different initialization k values.

6. The method according to claim 5, characterized in that The step of calculating the target k value of the pixel to be encoded based on the optimal k value of the adjacent pixel on the left side of the pixel to be encoded, the k value actually applied by the adjacent pixel on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient comprises: Adding the optimal k value of the neighboring pixels on the left side of the pixel to be encoded, the k value actually applied by the neighboring pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient to obtain a first k value prediction result; The first k value prediction result is shifted right by two bits to obtain the target k value of the pixel to be encoded.

7. The method according to claim 5, characterized in that The step of calculating the target k value of the pixel to be encoded based on the optimal k value of the adjacent pixel on the left side of the pixel to be encoded, the k value actually applied by the adjacent pixel on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient comprises: Determining whether the pixel to be encoded is the first pixel to be encoded in the first encoding row; When the pixel to be encoded is not the first pixel to be encoded in the first encoding row, the target k value of the pixel to be encoded is calculated based on the optimal k value of the adjacent pixels on the left side of the pixel to be encoded, the actually applied k value of the adjacent pixels on the left side of the pixel to be encoded, the initial k value of the first encoding row and the preset coefficient.

8. The method according to claim 7, characterized in that The method further comprises: In the case that the pixel to be encoded is the first pixel to be encoded in the first encoding row, the initial k value of the first encoding row is used as the target k value of the pixel to be encoded.

9. The method according to any one of claims 5 to 8, characterized in that The method further comprises: Determine that the initial k value of the first coding row is the initialization k value; the first coding row is the first coding row of the first color component in the current coding sub-block; or, The optimal k value of the reference pixel, the k value actually applied by the reference pixel, the initial k value of the second coding line and the corresponding preset coefficient are added to obtain the first k value prediction result of the reference pixel; the first k value prediction result of the reference pixel is shifted two bits to the right to obtain the second k value prediction result of the reference pixel; the second k value prediction result of the reference pixel is added to the initial k value of the second coding line to obtain the third k value prediction result of the reference pixel; the third k value prediction result is shifted one bit to the right to obtain the initialization k value of the first coding line; the first coding line is the other coding lines in the coding lines of the second color component of the current coding sub-block except the first coding line; wherein, The first color component is the color component corresponding to the pixel to be encoded, and the reference pixel is the last reference pixel to be encoded in the second encoding row.

10. An image processing device, characterized in that: The device comprises: A first processing unit is used to calculate first gradient values ​​of a plurality of adjacent pixels of a pixel to be encoded in different gradient directions to obtain a plurality of first gradient values; wherein the adjacent pixels are encoded pixels, and the pixel to be encoded has the same color component as the adjacent pixels; a determining unit, configured to determine a second gradient value having a minimum value from among the plurality of first gradient values, and determine a gradient direction corresponding to the second gradient value, to obtain a target gradient direction; A second processing unit, configured to calculate an estimated pixel value of the pixel to be encoded based on pixel values ​​of adjacent pixels of the pixel to be encoded in the target gradient direction; The third processing unit is used to calculate the difference between the estimated pixel value of the pixel to be encoded and the actual pixel value of the pixel to be encoded to obtain the target residual of the pixel to be encoded; the target residual is used to encode the pixel to be encoded.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

13. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 9.

Images