Image data processing method and device and electronic equipment

Abstract

The invention provides an image data processing method and device and electronic equipment, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining a resolution reduction image of an original image, and original image data of the original image in a first preset region; performing sequence-variable combination processing on the resolution-reduced image to generate an intermediate image of which the resolution is the same as that of the original image; wherein the combination processing comprises fuzzy processing and resolution recovery processing; replacing the image data corresponding to the first preset area in the intermediate image with the original image data to generate a target image; according to the technical scheme, the data transmission amount of the display device when the display device displays the high-definition image can be reduced, key information of the original image is reserved in a partition processing mode, and the experience feeling of a user is improved.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to an image data processing method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of display technology, high-resolution, high-refresh-rate screens have been widely used in various display devices (such as head-mounted VR / AR devices, mobile terminals, and in-vehicle displays).

[0003] Such devices face problems such as insufficient display interface transmission bandwidth, poor user experience, and high power consumption when displaying high-definition images. Summary of the Invention

[0004] The present invention provides an image data processing method, apparatus, and electronic device to solve at least one of the defects mentioned in the background art.

[0005] In a first aspect, the present invention provides an image data processing method, comprising: acquiring a resolution-reduced image of an original image, and original image data of the original image in a first preset region; The image with reduced resolution is subjected to a combination process with a variable order to generate an intermediate image with the same resolution as the original image; the combination process includes blurring and resolution restoration. The image data corresponding to the first preset region in the intermediate image is replaced with the original image data to generate the target image.

[0006] According to an image data processing method provided by the present invention, after generating a target image, the method further includes: determining each pixel point located in a second preset area in the target image; determining a variation coefficient for each pixel point based on the position information of each pixel point; and using the variation coefficient to perform attenuation mapping adjustment on the brightness value of each pixel point.

[0007] According to an image data processing method provided by the present invention, a combination of processing with variable execution order is performed on a resolution-reduced image, including: first blurring the resolution-reduced image, and then performing resolution restoration processing on the blurred resolution-reduced image; and first performing resolution restoration processing on the resolution-reduced image, and then blurring the resolution-reduced image after resolution restoration processing; wherein the execution order is determined according to a mode selection identifier.

[0008] According to an image data processing method provided by the present invention, the blurring process is implemented by a low-pass filter of a preset size, and the parameters of the filter are related to the execution order of the combined processing.

[0009] According to an image data processing method provided by the present invention, the resolution restoration processing is achieved by interpolation.

[0010] According to an image data processing method provided by the present invention, the method determines the change coefficient of each pixel based on the position information of each pixel, including: determining the distance of each pixel from the target position of the target image based on the position information of each pixel; and determining the change coefficient corresponding to each pixel at the current distance based on a preset distance-change coefficient mapping function; wherein the change coefficient is in the range of 0 to 1, and decreases as the distance of the pixel from the target position increases.

[0011] According to an image data processing method provided by the present invention, the first preset region includes the user's gaze region.

[0012] According to an image data processing method provided by the present invention, the second preset region includes the user's non-focused region.

[0013] In a second aspect, the present invention also provides an image data processing apparatus, comprising: The first processing module is configured to acquire a resolution-reduced image of the original image, and the original image data of the original image in a first preset area; The second processing module is configured to perform variable-order combination processing on the downsized image to generate an intermediate image with the same resolution as the original image; wherein, the combination processing includes blurring and resolution restoration processing; The third processing module is configured to replace the image data corresponding to the first preset region in the intermediate image with the original image data to generate the target image.

[0014] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the image data processing methods described above.

[0015] The image data processing method, apparatus, and electronic device provided by this invention have the following advantages compared with existing technologies: This invention acquires a reduced-resolution image of the original image and the original image data in a first preset region. It then performs sequentially variable blurring and resolution restoration processing on the reduced-resolution image to generate an intermediate image with the same resolution as the original image. The portion of the intermediate image corresponding to the first preset region is replaced with the original image data to generate the target image. This ensures that the first preset region (generally set as the user's gaze area) remains undistorted, while other regions are blurred intermediate images. This reduces image transmission volume while preserving key information from the original image, alleviating the pressure of insufficient display bandwidth while preserving the original image quality as much as possible. Furthermore, this invention retains the original resolution in the user's gaze area and blurs the non-gaze area, which better matches the user's actual visual perception and can reduce dizziness, thus improving the user experience.

[0016] The present invention can also attenuate the pixel brightness in a second preset area (which may be a non-focused area) of the target image, so that when the adjusted image is displayed, the display power consumption of the subsequent display device is reduced. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the image data processing method provided by the present invention; Figure 2 This is a schematic diagram of the process for adjusting the brightness value of a target image by attenuation mapping, provided by the present invention. Figure 3 This is the second flowchart illustrating the image data processing method provided by the present invention; Figure 4 This is a schematic diagram of the image data processing device provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0021] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0022] Figure 1 This is one of the flowcharts illustrating the image data processing method provided by the present invention, such as... Figure 1 As shown, the image data processing method provided by the present invention includes: Step 101: Obtain a reduced-resolution image of the original image, and the original image data of the original image in the first preset region; wherein, the reduced-resolution image is generated after the original image has been reduced in resolution.

[0023] The image data obtained in this embodiment of the invention includes: a reduced-resolution image of the original image and the original image data of the original image in a first preset region.

[0024] Among them, the resolution-downscaled image of the original image is generated by downsampling the resolution of the original image (an image with a higher resolution). For example, a resolution-downscaled image of 1920 x 1080 is generated by downsampling the original image of 3840 x 2160.

[0025] The first preset region can be the user's gaze region. The user's gaze region can be determined based on the gaze point determined by the eye-tracking method. This method is relatively mature and will not be elaborated here. The present invention directly obtains the unsampled raw image data from the original image for subsequent processing.

[0026] Step 102: Perform a variable-order combination process on the reduced-resolution image to generate an intermediate image with the same resolution as the original image; wherein, the combination process includes blurring and resolution restoration.

[0027] This invention allows for the option of first performing blurring processing on the reduced-resolution image, followed by resolution restoration processing. In this sequence, the blurring process directly affects the reduced-resolution image, significantly suppressing its high-frequency detail components. Although this step can be configured with a minimum blur level, the inherent nonlinear mapping characteristics during size restoration, which require high-magnification of the reduced-resolution image in subsequent resolution restoration processing, will cause the original weak blur effect to be significantly amplified nonlinearly. The resulting intermediate image exhibits highly smoothed features in non-fixated areas, meeting the deep blurring requirements of surrounding visual content in highly immersive virtual scenes. It should be noted that the relationship between blur level control and output effect in this mode is nonlinear, requiring a pre-compensation strategy to achieve precise effect control.

[0028] The present invention can also optionally perform resolution restoration processing on the reduced-resolution image first, and then perform blur processing on the image after resolution restoration. In this order, by adjusting the convolution kernel weights of the image after resolution restoration (such as adjusting the standard deviation σ of the Gaussian filter), the correspondence between the blur intensity and the output effect can be maintained at approximately linear, achieving more refined visual control.

[0029] The execution order can be determined based on a mode selection flag, which can be a programmable register value. For example, if the mode selection flag is sel, when sel is 0, blurring is performed first and then resolution restoration is performed; when sel is 1, resolution restoration is performed first and then blurring is performed.

[0030] The blurring process is implemented using a low-pass filter (LPF) of a preset size, including but not limited to convolutional kernel structures with integer dimensions such as 3×3 and 5×5. This low-pass filter performs spatial convolution operations on the pixel neighborhood of the input image using a configurable convolution weight matrix (such as Gaussian distributed weight coefficients), directly suppressing high-frequency components to generate a smooth output. The filter parameters can be associated with the execution order of the combined processing, enabling precise control of the blur intensity and range under different processing sequences. For example, the filter's size parameters and weight distribution can be associated with a mode selection flag.

[0031] The resolution restoration process can be implemented using interpolation algorithms, including at least one of Lanczos interpolation, bilinear interpolation, nearest neighbor interpolation, and bicubic interpolation. This process reconstructs an intermediate image at the original image resolution based on the pixel matrix of the reduced-resolution image using an interpolation function.

[0032] Step 103: Replace the image data corresponding to the first preset region in the intermediate image with the original image data to generate the target image.

[0033] Because the original data is uncompressed and downsampled, the image of the first preset region of the target image after replacement is not distorted compared to the original image.

[0034] After generating the target image, it can be directly sent to the display device for display. This invention obtains a reduced-resolution image of the original image and the original image data in a first preset area, and performs sequentially variable blurring and resolution restoration processing on the reduced-resolution image to generate an intermediate image with the same resolution as the original image. The part of the intermediate image corresponding to the first preset area is replaced with the original image data to generate the target image. This ensures that the first preset area (generally set as the user's gaze area) is not distorted, while other areas are blurred intermediate images. This reduces the amount of image transmission while retaining the key information of the original image, alleviating the pressure of insufficient display transmission bandwidth while preserving the image quality of the original image as much as possible. Furthermore, this invention retains the original resolution in the user's gaze area and blurs the non-gaze area, which is more in line with the user's actual viewing experience and can reduce the user's dizziness, thus improving the user experience.

[0035] Based on the above embodiments, as an optional embodiment, the present invention can further reduce the display power consumption of the display device by adjusting the brightness value of the target image through attenuation mapping before sending it to the display device for display. Figure 2 As shown, the method for adjusting the attenuation mapping includes, but is not limited to, the following steps: Step 201: Determine each pixel in the target image located within the second preset region.

[0036] The second preset region can be dynamically defined based on the peripheral vision perception characteristics of the human eye, and is bound to gaze point tracking data, covering all non-gaze areas in the target image except for the first preset region.

[0037] Step 202: Determine the variation coefficient of each pixel based on the position information of each pixel.

[0038] This invention can determine the variation coefficient of each pixel based on its distribution position in a second preset region. For example, the closer the pixel is to the edge of the image, the smaller the variation coefficient.

[0039] As an optional embodiment, the present invention determines the change coefficient of each pixel based on the position information of each pixel, including: determining the distance of each pixel from the target position of the target image based on the position information of each pixel; determining the change coefficient corresponding to each pixel at the current distance based on a preset distance-change coefficient mapping function; wherein the change coefficient ranges from 0 to 1, and decreases as the distance of the pixel from the target position increases.

[0040] The distance can be Euclidean distance or other distances, the target position can be the center position of the preset target image, or the position corresponding to the user's gaze point; the distance-variance coefficient mapping function can be a piecewise function, a Gaussian decay function, etc., and this invention does not impose specific limitations on it, and can be set according to actual needs.

[0041] Step 203: Adjust the brightness value of each pixel by attenuation mapping using a change coefficient.

[0042] The brightness value Y of each pixel in the second preset area is calculated using the change coefficient k to form the mapped brightness value Y', such as: Y'=k×Y; The adjusted brightness value Y' can have a gradual brightness decay, which reduces display power consumption and is more in line with the characteristics of the human eye, thus improving the user experience.

[0043] To provide a clearer explanation of the present invention, a complete implementation process will be used to illustrate the invention. It should be noted that the process in this embodiment involves an image source end (used to provide a resolution-reduced image of the original image and the original image data of the original image in a first preset area), an image processing end (such as a display chip, used to execute the image processing method in the present invention), and a display end (such as an AR / VR display, used to receive the processed image and display it).

[0044] Figure 3 This is a second schematic flowchart of the image data processing method provided by the present invention, as shown below. Figure 3 As shown, including but not limited to the following steps: (1) Data acquisition stage The image source decomposes the original image into two parts and transmits them to the image processing end: First, the original image is downscaled to generate a downscaled image. Second, the original image data of a first preset region is extracted from the original image simultaneously. The coordinate information of this region is either updated in real time by the eye-tracking module or determined by preset register configuration. This step ensures that the critical information required for subsequent processing is transmitted in a way that minimizes the amount of data.

[0045] (2) Combined processing stage The image processing unit performs sequential combination processing based on the value of the mode selection flag sel: When sel=0, the image with reduced resolution is first blurred (e.g., using a 3×3 or 5×5 low-pass filter), and then resolution restoration is performed using a preset interpolation algorithm (e.g., Lanczos interpolation or bilinear interpolation). This path performs blurring in low-resolution space, significantly enhancing the blurring effect in non-focal regions. When sel=1, resolution restoration is performed first, followed by low-pass filtering; this approach preserves more image details.

[0046] (3) Regional replacement stage After generating an intermediate image with the same resolution as the original image, the image data matching the coordinates of the first preset region is precisely replaced: first, the coordinate range corresponding to the first preset region in the intermediate image is located, and then the original image data is directly overwritten over this region. After the replacement is completed, the target image is generated, in which the first preset region retains its original image quality, while the remaining regions maintain the blurred reconstruction effect of the combined processing.

[0047] (4) Brightness attenuation processing stage Brightness mapping adjustment is performed on the target image: First, each pixel located within a second preset area (such as the user's non-focused area) is identified; then, a variation coefficient is determined based on the position information of each pixel; finally, the brightness value of each pixel is adjusted by attenuation mapping. As an adjustment method, the brightness of pixels closer to the edge decreases more after adjustment, creating a visual effect of uniform darkening from the center to the edge.

[0048] (5) Output direct display terminal The processed target image can be directly output to a display device for display.

[0049] This invention acquires a reduced-resolution image of the original image and the original image data in a first preset region. It then performs sequentially variable blurring and resolution restoration processing on the reduced-resolution image to generate an intermediate image with the same resolution as the original image. The portion of the intermediate image corresponding to the first preset region is replaced with the original image data to generate the target image. This ensures that the first preset region (generally set as the user's gaze area) remains undistorted, while other regions are blurred intermediate images. This reduces image transmission volume while preserving key information from the original image, alleviating the pressure of insufficient display bandwidth while preserving the original image quality as much as possible. Furthermore, this invention retains the original resolution in the user's gaze area and blurs the non-gaze area, which better matches the user's actual visual perception and can reduce dizziness, thus improving the user experience.

[0050] The present invention can also attenuate the pixel brightness in a second preset area (which may be a non-focused area) of the target image, so that when the adjusted image is displayed, the display power consumption of the subsequent display device is reduced.

[0051] In another aspect, the present invention also provides an image data processing apparatus, such as... Figure 4 As shown, the device includes: The first processing module 401 is configured to acquire a resolution-reduced image of the original image, and the original image data of the original image in a first preset area; wherein, the resolution-reduced image is generated after the original image has undergone resolution reduction processing; The second processing module 402 is configured to perform variable-order combination processing on the reduced-resolution image to generate an intermediate image with the same resolution as the original image; wherein, the combination processing includes blurring processing and resolution restoration processing; The third processing module 403 is configured to replace the image data corresponding to the first preset region in the intermediate image with the original image data to generate the target image.

[0052] It should be noted that the image data processing apparatus provided in this embodiment of the invention can execute the image data processing method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0053] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an image data processing method, which includes: acquiring a resolution-reduced image of the original image, and original image data of the original image in a first preset region; wherein the resolution-reduced image is generated after reducing the resolution of the original image; performing a combination processing on the resolution-reduced image with a variable execution order to generate an intermediate image with the same resolution as the original image; wherein the combination processing includes blurring and resolution restoration processing; and replacing the image data in the intermediate image corresponding to the first preset region with the original image data to generate a target image.

[0054] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the image data processing method provided in the above embodiments. The method includes: acquiring a resolution-reduced image of the original image and original image data of the original image in a first preset region; wherein the resolution-reduced image is generated after the original image has undergone resolution reduction processing; performing a combination processing on the resolution-reduced image with a variable execution order to generate an intermediate image with the same resolution as the original image; wherein the combination processing includes blurring processing and resolution restoration processing; replacing the image data in the intermediate image corresponding to the first preset region with the original image data to generate a target image.

[0056] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image data processing method, characterized by, The method comprises: obtaining a resolution-reduced image of an original image, and original image data of the original image in a first preset region; performing combination processing on the resolution-reduced image in a variable execution order to generate an intermediate image with the same resolution as the original image; wherein the combination processing comprises blur processing and resolution restoration processing; replacing image data corresponding to the first preset region in the intermediate image with the original image data to generate a target image.

2. The image data processing method of claim 1, wherein, After the target image is generated, the method further comprises: determining each pixel point in a second preset region in the target image; determining a change coefficient of each pixel point according to position information of each pixel point; performing decay mapping adjustment on a brightness value of each pixel point using the change coefficient.

3. The image data processing method of claim 1, wherein, The combination processing on the resolution-reduced image in a variable execution order comprises: first performing blur processing on the resolution-reduced image, and then performing resolution restoration processing on the resolution-reduced image after the blur processing; and first performing resolution restoration processing on the resolution-reduced image, and then performing blur processing on the resolution-reduced image after the resolution restoration processing; wherein the execution order is determined according to a mode selection identifier.

4. The image data processing method of claim 1, wherein, The blur processing is implemented by a low-pass filter with a preset size, and a parameter of the filter is associated with the execution order of the combination processing.

5. The image data processing method of claim 1, wherein, The resolution restoration processing is implemented by an interpolation method.

6. The image data processing method of claim 2, wherein, The determination of the change coefficient of each pixel point according to the position information of each pixel point comprises: determining a distance of each pixel point from a target position of the target image according to the position information of each pixel point; determining a change coefficient corresponding to each pixel point at a current distance based on a preset distance-change coefficient mapping function; wherein the change coefficient has a value range of 0 to 1 and decreases with an increase in the distance of the pixel point from the target position.

7. The image data processing method of claim 1, wherein, The first preset region comprises a gaze region of a user.

8. The image data processing method of claim 2, wherein, The second preset region comprises a non-gaze region of the user.

9. An image data processing apparatus characterized by comprising: The method comprises: a first processing module configured to obtain a resolution-reduced image of an original image, and original image data of the original image in a first preset region; a second processing module configured to perform combination processing on the resolution-reduced image in a variable execution order to generate an intermediate image with the same resolution as the original image; wherein the combination processing comprises blur processing and resolution restoration processing; a third processing module configured to replace image data corresponding to the first preset region in the intermediate image with the original image data to generate a target image.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the image data processing method according to any one of claims 1 to 8 when executing the computer program.

Images