Image data processing method and device and electronic equipment

Abstract

The invention provides an image data processing method and device, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining a bit width reduced image of an original image, and original image data of the original image in a first preset region; the bit width reduction image is an image generated by performing bit width reduction processing on the original image; performing blurring processing and bit width recovery processing on the bit width reduced image in sequence to generate an intermediate image with the same bit width as the original image; and replacing the image data corresponding to the first preset area in the intermediate image with the original image data to generate a target image. The finally generated target image is the image data of the original image in the first preset area, all features of the original image are reserved, the data transmission amount is reduced compared with the original image for the part outside the first preset area, the immersion of a user is improved during display through fuzzy processing, and the user experience is improved. And the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to an image data processing method, device and electronic equipment. BACKGROUND

[0002] With the rapid development of display technology, high-resolution and high-refresh-rate screens have been widely applied to various display devices (such as head-mounted VR / AR devices, mobile terminals, vehicle-mounted displays, etc.).

[0003] Such devices face problems such as insufficient display interface transmission bandwidth, poor experience, and high power consumption when presenting high-quality images. SUMMARY

[0004] The present application provides an image data processing method, device and electronic equipment to solve at least one defect in the prior art.

[0005] In a first aspect, the present application provides an image data processing method, comprising: obtaining a bit-width reduced image of an original image and original image data of a first preset region of the original image; the bit-width reduced image is an image generated by performing bit-width reduction processing on the original image; sequentially performing blur processing and bit-width restoration processing on the bit-width reduced image to generate an intermediate image with the same bit width as the original image; replacing image data corresponding to the first preset region in the intermediate image with the original image data to generate a target image.

[0006] According to the image data processing method provided by the present application, the original image is subjected to bit-width reduction processing, including: dividing the original image into matrix units with a size of n x n; wherein n represents the difference between the bit width of the original image and the bit width of the bit-width reduced image; generating a random dithering value matrix with the same size and bit width as the matrix unit; wherein the element size in the random dithering value matrix traverses all integers from 0 to 2 n -1, and the position is randomly distributed; adding the original data of each matrix unit to the random dithering value matrix to generate a corresponding intermediate matrix unit; performing a right shift operation on the elements in each intermediate matrix unit by n bits.

[0007] According to the image data processing method provided by the present application, the bit-width reduced image is sequentially subjected to blur processing and bit-width restoration processing to generate an intermediate image with the same bit width as the original image, including: using a low-pass filter with a preset size to perform blur processing on the bit-width reduced image; performing a left shift operation on the image data after blur processing by n bits to generate an intermediate image with the same bit width as the original image.

[0008] According to the image data processing method provided by the present application, after the target image is generated, the method further comprises: determining each pixel point in the target image located in the second preset area; determining a change coefficient of each pixel point according to position information of each pixel point; and performing attenuation mapping adjustment on a brightness value of each pixel point by using the change coefficient.

[0009] According to the image data processing method provided by the present application, the change coefficient of each pixel point is determined according to the position information of each pixel point, and the method comprises: determining a distance between each pixel point and a target position of the target image according to the position information of each pixel point; and determining a change coefficient corresponding to each pixel point at a current distance based on a preset distance-change coefficient mapping function; wherein the change coefficient has a value range of 0 to 1 and decreases with the increase of the distance between the pixel point and the target position.

[0010] According to the image data processing method provided by the present application, the first preset area comprises a gaze area of a user, and the second preset area comprises a non-gaze area of the user.

[0011] According to the image data processing method provided by the present application, the preset size is associated with n.

[0012] In a second aspect, the present application further provides an image data processing device, comprising: A first processing module configured to acquire a bit-width reduced image of an original image and original image data of the original image in a first preset area; the bit-width reduced image is an image generated by performing bit-width reduction processing on the original image; A second processing module configured to sequentially perform blur processing and bit-width recovery processing on the bit-width reduced image to generate an intermediate image having the same bit width as the original image; A third processing module configured to replace image data corresponding to the first preset area in the intermediate image with the original image data to generate a target image.

[0013] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image data processing method according to any one of the above aspects when executing the program.

[0014] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the steps of the image data processing method according to any one of the above aspects.

[0015] The image data processing method, device and electronic device provided by the present application have the following beneficial effects compared with the prior art: The application reduces the basic transmission data amount by performing bit width reduction on the original image, generates an intermediate image by performing blur processing and bit width recovery processing on a first preset area (generally a gaze area), replaces the image data corresponding to the first preset area in the intermediate image with original image data, generates a target image, and finally generates a target image in the first preset area as the image data of the original image, retains all the features of the original image, reduces the data transmission amount for the part outside the first preset area compared with the original image, and improves the user's immersion when displaying through blur processing, and improves the user's use experience.

[0016] The application can also adjust the pixel brightness in the second preset area (which can be a non-gaze area) of the target image to reduce the display power consumption of the subsequent display device when the adjusted image is displayed. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is one of the flowcharts of the image data processing method provided by the application; Figure 2 is a schematic diagram of the bit width reduction processing of the original image provided by the application; Figure 3 is a flowchart of the brightness value attenuation mapping adjustment of the target image provided by the application; Figure 4 is another flowchart of the image data processing method provided by the application; Figure 5 is a structural schematic diagram of the image data processing device provided by the application; Figure 6 is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be described clearly and completely below in combination with the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0020] It should be noted that in the description of the embodiments of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element. The specific meaning of the above terms in the present application can be understood by the person skilled in the art according to the specific circumstances.

[0021] The terms "first", "second", and the like used in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, not limited to the number of objects, for example, the first object can be one or more.

[0022] The embodiments of the present application will be described below in conjunction with Figures 1-6 The image data processing method, device and electronic equipment provided by the embodiments of the present application are described.

[0023] Figure 1 is one of the flowcharts of the image data processing method provided by the present application, as Figure 1 shown, including but not limited to the following steps: Step 101: obtaining a bit width reduced image of an original image, and original image data of the original image in a first preset area.

[0024] The image data obtained in the embodiments of the present application includes a bit width reduced image of an original image and original image data of the original image in a first preset area.

[0025] The bit width reduced image of the original image is an image generated by performing bit width reduction processing on the original image (an image with a higher bit width). For example, a bit width reduced image with a bit width of 8 bits is generated by performing bit width reduction processing on an original image with a bit width of 10 bits. The method of bit width reduction processing includes a dithering processing algorithm.

[0026] The first preset area can be the user's gaze area, and the user's gaze area can be determined according to the gaze point determined by the eye movement tracking method, which is already relatively mature and will not be described here. The present application directly obtains original image data that has not been subjected to downsampling processing from the original image for subsequent processing.

[0027] Step 102: sequentially performing blur processing and bit width recovery processing on the bit width reduced image to generate an intermediate image with the same bit width as the original image.

[0028] The application can use a filtering algorithm to perform blur processing on the bit width reduced image, such as Gaussian blur, bilateral filtering, etc.

[0029] The application maps the blurred image back to the original high bit width, avoiding color step faults caused by bit depth mismatch during subsequent synthesis. The bit width recovery processing operation includes but is not limited to linear stretching, non-linear mapping, etc. For example, 8-bit data (0-255) is linearly expanded to a 10-bit range (0-1023) through linear stretching.

[0030] It should be noted that the bit width recovery operation is only a numerical format recovery, not a data volume recovery. The original high bit width information has been lost in the reduction process, and this step only guarantees data format compatibility.

[0031] Step 103: replacing image data corresponding to the first preset region in the intermediate image with original image data to generate a target image.

[0032] This step replaces the image data of the first preset region in the intermediate image generated in step 102 with the original image data extracted in step 101 to generate a target image that takes into account global efficiency and local high fidelity.

[0033] Based on the above embodiment, as an optional embodiment, the image data processing method provided by the application includes: performing bit width reduction processing on an original image, including: dividing the original image into matrix units with a size of n x n; wherein n represents the difference between the bit width of the original image and the bit width of the bit width reduced image; generating a random dithering value matrix with the same size and bit width as the matrix unit; wherein the element size in the random dithering value matrix traverses all integers from 0 to 2 n -1, and the position is randomly distributed; adding the original data of each matrix unit to the random dithering value matrix to generate a corresponding intermediate matrix unit; performing a right shift operation on the elements in each intermediate matrix unit by n bits.

[0034] Figure 2 is a schematic diagram of the bit width reduction processing on the original image provided by the application, as shown in Figure 2 The bit width reduction processing is to convert and output an original image with a bit width of a bits to an image with a bit width of b bits (a is greater than b), such as converting a 10-bit image to an 8-bit image.

[0035] Specifically, firstly, the original image (input image) is divided into matrix units, with each unit having a size of (ab)*(ab). Next, a random dithering matrix (i.e., a dither table matrix unit) is generated. The size and bit width of the random dithering matrix are consistent with the matrix units of the original image. The values ​​in the random dithering matrix traverse from 0 to 2^(ab) - 1, with random positions. Finally, the random dithering matrix units are summed to obtain a new data with a bit width. This data is then shifted right by (ab) bits to obtain an output image with a bit width of b bits (a bit-width reduced image). After the bit-width reduction process, the data volume is reduced by (ab) / a.

[0036] Based on the above embodiments, as an optional embodiment, the present invention provides an image data processing method that sequentially performs blurring and bit-width restoration processing on a bit-width reduced image to generate an intermediate image with the same bit-width as the original image, including: blurring the bit-width reduced image using a low-pass filter of a preset size; and shifting the blurred image data to the left by n bits to generate an intermediate image with the same bit-width as the original image.

[0037] The filter can be a low-pass filter, and the filter size can be configured as an odd number of kernels such as 3*3, 5*5, or 7*7 to blur images with reduced bit width.

[0038] Larger n values ​​(increased bit width compression) often require stronger blurring to suppress compression artifacts, thus allowing for an increase in filter size. This invention associates a preset size with n, determining the preset size based on the value of n.

[0039] By shifting the blurred image data left by n bits, the image with a reduced bit width can be restored to its original data format, facilitating subsequent image stitching.

[0040] Based on the above embodiments, as an optional embodiment, the present invention can further reduce the display power consumption of the display device by adjusting the brightness value of the target image through attenuation mapping before sending it to the display device for display. Figure 3 As shown, the method for adjusting the attenuation mapping includes, but is not limited to, the following steps: Step 301: Determine each pixel in the target image located within the second preset region.

[0041] The second preset region can be dynamically defined based on the peripheral vision perception characteristics of the human eye, and is bound to gaze point tracking data, covering all non-gaze areas in the target image except for the first preset region.

[0042] Step 302: Determine the variation coefficient of each pixel based on the position information of each pixel.

[0043] This invention can determine the variation coefficient of each pixel based on its distribution position in a second preset region. For example, the closer the pixel is to the edge of the image, the smaller the variation coefficient.

[0044] As an optional embodiment, the present invention determines the change coefficient of each pixel based on the position information of each pixel, including: determining the distance of each pixel from the target position of the target image based on the position information of each pixel; determining the change coefficient corresponding to each pixel at the current distance based on a preset distance-change coefficient mapping function; wherein the change coefficient ranges from 0 to 1, and decreases as the distance of the pixel from the target position increases.

[0045] The distance can be Euclidean distance or other distances, the target position can be the center position of the preset target image, or the position corresponding to the user's gaze point; the distance-variance coefficient mapping function can be a piecewise function, a Gaussian decay function, etc., and this invention does not impose specific limitations on it, and can be set according to actual needs.

[0046] Step 303: Adjust the brightness value of each pixel by attenuation mapping using a change coefficient.

[0047] The brightness value Y of each pixel in the second preset area is calculated using the change coefficient k to form the mapped brightness value Y', such as: Y'=k×Y; The adjusted brightness value Y' can have a gradual brightness decay, which reduces display power consumption and is more in line with the characteristics of the human eye, thus improving the user experience.

[0048] To provide a clearer explanation of the present invention, a complete implementation process will be used to illustrate the invention. It should be noted that the process in this embodiment involves an image source end (used to provide a bit-width reduced image of the original image and the original image data of the original image in a first preset area), an image processing end (such as a display chip, used to execute the image processing method in the present invention), and a display end (such as an AR / VR display, used to receive the processed image for display).

[0049] Figure 4 This is a second schematic flowchart of the image data processing method provided by the present invention, as shown below. Figure 4 As shown, including but not limited to the following steps: (1) Image data acquisition stage The image source end decomposes the original image into two parts and transmits them to the image processing end: First, the bit width of the original image is reduced to generate a bit width-reduced image. Second, the original image data of the first preset region is extracted from the original image simultaneously. The coordinate information of this region is either updated in real time by the eye-tracking module or determined by the preset register configuration.

[0050] (2) Bit width reduction processing The input image (i.e., the original image) with a bit width is converted to an output image with a bit width of b bits. Taking 10 bits to 8 bits as an example, the input image is divided into matrix units, and the size of the matrix unit is (ab) * (ab). The size of the dither table matrix unit is the same as that of the input image matrix unit. The values ​​of the dither table matrix traverse from 0 to 2^(ab)-1, and the positions are random. The matrix unit of the input image is added to the matrix unit of the dither table matrix to obtain the new data with a bit width. The data is then shifted to the right by (ab) bits to obtain the output image with a bit width of b bits, i.e., the bit width reduced image. After the bit width reduction processing, the amount of data is reduced by (ab) / a.

[0051] (3) Blur processing and bit width recovery processing This invention can use a low-pass filter to process images with reduced bit width. Specifically, it uses a 3*3, 5*5, or 7*7 filter matrix to blur the image.

[0052] Shift the b bits of image data left by (ab) bits to obtain a bits of image data, thereby restoring the original bit width and obtaining the intermediate image.

[0053] (4) Replacement process After generating the intermediate image, the image data matching the coordinates of the first preset region is precisely replaced: first, the coordinate range corresponding to the first preset region in the intermediate image is located, and then the original image data is directly overwritten in that region. After the replacement is completed, the target image is generated, in which the first preset region retains its original image quality, while the remaining regions maintain the blurred reconstruction effect of the combined processing.

[0054] (5) Brightness attenuation processing Brightness mapping adjustment is performed on the target image: First, each pixel located within a second preset area (such as the user's non-focused area) is identified; then, a variation coefficient is determined based on the position information of each pixel; finally, the brightness value of each pixel is adjusted by attenuation mapping. As an adjustment method, the brightness of pixels closer to the edge decreases more after adjustment, creating a visual effect of uniform darkening from the center to the edge.

[0055] The processed target image can be directly output to a display device for display.

[0056] In summary, this invention reduces the basic data transmission volume by reducing the bit width of the original image, and performs blurring and bit width restoration processing on the first preset region (generally the gaze region) to generate an intermediate image. The image data corresponding to the first preset region in the intermediate image is replaced with the original image data to generate the target image. The final target image is the image data of the original image in the first preset region, retaining all the features of the original image. For the parts outside the first preset region, the data transmission volume is reduced compared to the original image, and the blurring process enhances the user's immersion during display, thus improving the user experience.

[0057] The present invention can also attenuate the pixel brightness in a second preset area (which may be a non-focused area) of the target image, so that when the adjusted image is displayed, the display power consumption of the subsequent display device is reduced.

[0058] In another aspect, the present invention also provides an image data processing apparatus, such as... Figure 5 As shown, the device includes: The first processing module 501 is configured to acquire a bit-width-reduced image of the original image, and the original image data of the original image in a first preset area; the bit-width-reduced image is an image generated by reducing the bit-width of the original image. The second processing module 502 is configured to perform blurring and bit width restoration processing on the bit width-reduced image in sequence to generate an intermediate image with the same bit width as the original image. The third processing module 503 is configured to replace the image data corresponding to the first preset region in the intermediate image with the original image data to generate the target image.

[0059] It should be noted that the image data processing apparatus provided in this embodiment of the invention can execute the image data processing method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0060] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an image data processing method, which includes: acquiring a bit-width-reduced image of the original image, and the original image data of the original image in a first preset region; the bit-width-reduced image is an image generated by reducing the bit-width of the original image; performing blurring and bit-width restoration processing on the bit-width-reduced image in sequence to generate an intermediate image with the same bit-width as the original image; replacing the image data corresponding to the first preset region in the intermediate image with the original image data to generate a target image.

[0061] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the image data processing method provided in the above embodiments, the method including: acquiring a bit-width reduced image of the original image, and original image data of the original image in a first preset region; the bit-width reduced image is an image generated by bit-width reduction processing of the original image; performing blur processing and bit-width restoration processing on the bit-width reduced image in sequence to generate an intermediate image with the same bit-width as the original image; replacing the image data in the intermediate image corresponding to the first preset region with the original image data to generate a target image.

[0063] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the image data processing method provided in the above embodiments. The method includes: acquiring a bit-width-reduced image of the original image, and original image data of the original image in a first preset region; the bit-width-reduced image is an image generated by bit-width reduction processing of the original image; performing blurring processing and bit-width restoration processing on the bit-width-reduced image in sequence to generate an intermediate image with the same bit-width as the original image; replacing the image data in the intermediate image corresponding to the first preset region with the original image data to generate a target image.

[0064] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image data processing method, characterized in that, include: Obtain the bit-width scaled-down image of the original image, and the original image data of the original image in the first preset region; A bit-width reduced image is an image generated by reducing the bit width of the original image; The image with reduced bit width is sequentially blurred and then restored to generate an intermediate image with the same bit width as the original image. The image data corresponding to the first preset region in the intermediate image is replaced with the original image data to generate the target image.

2. The image data processing method according to claim 1, characterized in that, Bit-width reduction processing of the original image includes: The original image is divided into matrix units of size n×n; where n represents the difference in bit width between the original image and the bit-width-reduced image. Generate a random jitter value matrix with the same size and width as the matrix elements; wherein the elements in the random jitter value matrix iterate from 0 to 2. n All integers -1, with their positions randomly distributed; The original data of each matrix unit is added to the random jitter value matrix to generate the corresponding intermediate matrix unit; The elements in each intermediate matrix cell are shifted right by n bits.

3. The image data processing method according to claim 2, characterized in that, The image with reduced bit width is sequentially blurred and then restored to its original bit width to generate an intermediate image with the same bit width as the original image, including: A low-pass filter of a preset size is used to blur images with reduced bit width. The blurred image data is shifted left by n bits to generate an intermediate image with the same bit width as the original image.

4. The image data processing method according to claim 2, characterized in that, After generating the target image, the following steps are also included: Identify each pixel in the target image located within a second preset region; Based on the position information of each pixel, determine the change coefficient of each pixel; The brightness value of each pixel is adjusted by attenuation mapping using a variation coefficient.

5. The image data processing method according to claim 4, characterized in that, Based on the position information of each pixel, determine the change coefficient of each pixel, including: Based on the position information of each pixel, determine the distance of each pixel from the target position in the target image; Based on a preset distance-change coefficient mapping function, the change coefficient of each pixel at the current distance is determined; The variation coefficient ranges from 0 to 1, and decreases as the distance between the pixel and the target location increases.

6. The image data processing method according to claim 1, characterized in that, The first preset area includes the user's gaze area; the second preset area includes the user's non-gaze area.

7. The image data processing method according to claim 3, characterized in that, The preset size is associated with n.

8. An image data processing apparatus, characterized in that, include: The first processing module is configured to acquire a bit-width reduced image of the original image, and the original image data of the original image in a first preset area; A bit-width reduced image is an image generated by reducing the bit width of the original image; The second processing module is configured to perform blurring and bit-width restoration processing on the bit-width reduced image in sequence to generate an intermediate image with the same bit-width as the original image. The third processing module is configured to replace the image data corresponding to the first preset region in the intermediate image with the original image data to generate the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image data processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image data processing method as described in any one of claims 1 to 7.

Images