Image processing method and apparatus, display apparatus, and storage medium

Abstract

An image processing method and apparatus, a display apparatus, and a storage medium. The image processing method comprises: performing block segmentation on an image to be processed, so as to segment the image into N image blocks; on the basis of a low-power mapping curve generation model, performing a mapping operation on the N image blocks to generate N contrast-limited adaptive low-power mapping curves that are in one-to-one correspondence with the N image blocks; on the basis of the N contrast-limited adaptive low-power mapping curves, respectively performing mapping operations on the N image blocks to obtain N contrast-limited adaptive low-power mapped image blocks that are in one-to-one correspondence with the N image blocks; and performing a stitching operation on the N contrast-limited adaptive low-power mapped image blocks to obtain a low-power mapped stitched image, N being an integer greater than 0. The image processing method can reduce power consumption and improve the display quality of images, thereby greatly improving the efficiency of image processing.

Description

An image processing method and apparatus, a display device and a storage medium Technical Field

[0001] The embodiments of this disclosure relate to the field of data processing technology, and more specifically, to an image processing method, an image processing apparatus, and a storage medium. Background Technology

[0002] Histogram equalization is a simple and effective image enhancement technique that alters the grayscale of pixels in an image by changing its histogram. It is primarily used to enhance the contrast of images with a narrow dynamic range. The original image may lack clarity because its grayscale distribution is concentrated in a narrow range. For example, an overexposed image will have its grayscale levels concentrated in the high-brightness range, while an underexposed image will have its grayscale levels concentrated in the low-brightness range. Histogram equalization transforms the original image's histogram into a more uniform (equalized) distribution, thus increasing the dynamic range of grayscale differences between pixels and enhancing the overall image contrast. In other words, the basic principle of histogram equalization is to broaden the grayscale values ​​that have a large number of pixels (i.e., those that play a major role in the image) and merge the grayscale values ​​that have a small number of pixels (i.e., those that do not play a major role), thereby increasing contrast, making the image clearer, and achieving the desired enhancement. Summary of the Invention

[0003] At least one embodiment of this disclosure provides an image processing method, comprising: dividing an image to be processed into N image blocks; performing a mapping operation on the N image blocks based on a low-power mapping curve generation model to generate N low-power mapping curves with limited contrast adaptively corresponding to the N image blocks; performing mapping operations on the N image blocks based on the N low-power mapping curves with limited contrast adaptively corresponding to the N image blocks to obtain N low-power mapping image blocks with limited contrast adaptively corresponding to the N image blocks; and performing a stitching operation on the N low-power mapping image blocks with limited contrast adaptively corresponding to the N image blocks to obtain a low-power mapping stitched image; wherein N is an integer greater than 0.

[0004] For example, at least one embodiment of the image processing method provided in this disclosure further includes: performing a stitching operation on the low-power mapped stitched image to obtain a low-power mapped image with limited contrast adaptation.

[0005] For example, in an image processing method provided in at least one embodiment of this disclosure, the image to be processed is divided into blocks to divide the image into N image blocks, including: converting the image to be processed into a grayscale image; dividing the grayscale image into M image sub-blocks; calculating the pixel mean of the M image sub-blocks to obtain an image mean matrix; and performing a fusion operation based on the image mean matrix to obtain the N image blocks; wherein M is an integer greater than 0.

[0006] For example, in an image processing method provided in at least one embodiment of this disclosure, a fusion operation is performed based on the image mean matrix to obtain the N image blocks, including: setting a fusion threshold; determining whether the pixel mean of adjacent image sub-blocks meets a preset condition with the fusion threshold; if it does, then fusing the adjacent image sub-blocks into one image block.

[0007] For example, in the image processing method provided in at least one embodiment of this disclosure, determining whether the average pixel value of adjacent image sub-blocks meets the fusion threshold includes: determining whether the difference between the average pixel values ​​of the adjacent image sub-blocks is less than the fusion threshold; if it is less, then fusion the adjacent image sub-blocks into one image block.

[0008] For example, in the image processing method provided in at least one embodiment of this disclosure, the input of the low-power mapping curve generation model is the N image blocks, and the output data of the low-power mapping curve generation model is the N low-power mapping curves.

[0009] For example, in an image processing method provided in at least one embodiment of this disclosure, performing a stitching operation on the low-power mapping stitched image to obtain a low-power mapping image with limited contrast adaptation includes: performing a stitching operation on the low-power mapping stitched image based on a stitching model to obtain a low-power mapping image with limited contrast adaptation, wherein the input of the stitching model is the low-power mapping stitched image, and the output of the stitching model is the low-power mapping image with limited contrast adaptation.

[0010] For example, in the image processing method provided in at least one embodiment of this disclosure, the stitching model includes a horizontal convolution kernel that convolves the horizontal seams and b vertical convolution kernels that convolve the vertical seams, where a is an integer greater than 0 and b is an integer greater than 0.

[0011] At least one embodiment of this disclosure also provides an image processing apparatus, comprising: a segmentation unit configured to segment an image to be processed into N image blocks; a first mapping unit configured to perform mapping operations on the N image blocks based on the low-power mapping curve generation model to generate N low-power mapping curves with limited contrast adaptively corresponding to the N image blocks; a second mapping unit configured to perform mapping operations on the N image blocks based on the N low-power mapping curves with limited contrast adaptively corresponding to the N image blocks to obtain N low-power mapping image blocks with limited contrast adaptively corresponding to the N image blocks; and a stitching unit configured to stitch the N low-power mapping image blocks with limited contrast adaptively corresponding to obtain a low-power mapping stitched image; wherein N is an integer greater than 0.

[0012] For example, at least one embodiment of the image processing apparatus provided in this disclosure further includes: a stitching unit configured to perform a stitching operation on the low-power mapped stitched image to obtain a low-power mapped image with limited contrast adaptation.

[0013] At least one embodiment of this disclosure also provides an image processing apparatus, including: a processor; a memory; and one or more computer program modules, wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for executing an image processing method that implements any embodiment of this disclosure.

[0014] At least one embodiment of this disclosure also provides a display device, including the image processing device provided in any embodiment of this disclosure.

[0015] For example, a display device provided in at least one embodiment of this disclosure further includes a timing controller, a gate driving circuit, and a data driving circuit, wherein the timing controller is configured to drive the gate driving circuit and the data driving circuit to display the low-power mapped image with limited contrast adaptation.

[0016] At least one embodiment of this disclosure also provides a storage medium for non-temporarily storing computer-readable instructions that, when executed by a computer, perform the image processing method provided in any embodiment of this disclosure. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0018] Figure 1 shows an overall architecture diagram of an image processing method provided in at least one embodiment of the present disclosure.

[0019] Figure 2 shows a schematic diagram of an image processing method provided in at least one embodiment of the present disclosure.

[0020] Figures 3A and 3B are schematic diagrams of image segmentation provided in at least one embodiment of this disclosure.

[0021] Figures 4A and 4B are schematic diagrams of a low-power mapping curve provided by at least one embodiment of the present disclosure.

[0022] Figure 5A is a schematic diagram of pixel frequency redistribution in the CLAHE algorithm.

[0023] Figure 5B is a schematic diagram of a pixel frequency redistribution provided in at least one embodiment of the present disclosure.

[0024] Figure 6 shows a schematic diagram of a suture model provided in at least one embodiment of the present disclosure.

[0025] Figure 7 is a schematic block diagram of an image processing apparatus provided in at least one embodiment of the present disclosure.

[0026] Figure 8 is a schematic block diagram of another image processing apparatus provided in at least one embodiment of the present disclosure.

[0027] Figure 9 is a schematic block diagram of an electronic device provided in at least one embodiment of the present disclosure.

[0028] Figure 10 is a schematic diagram of a display device provided in at least one embodiment of the present disclosure.

[0029] Figure 11 is a schematic diagram of a storage medium provided in at least one embodiment of the present disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0031] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0032] Histogram equalization currently has the following problems:

[0033] 1) Histogram equalization is global, and its effect is not very good when there are overly bright or dark local areas of the image.

[0034] 2) Histogram equalization will increase background noise.

[0035] To solve the above two problems, there are two solutions: one is to address the global problem, and the other is to address the background noise enhancement problem.

[0036] For example, to address the global problem, a method of image segmentation is proposed, with histogram equalization performed on each region separately. This allows the use of local information to enhance the image, thereby solving the global problem. For the background noise enhancement problem, since the main issue is excessive background enhancement, the problem can be solved by limiting the contrast.

[0037] The above problem can be solved by using the Contrast Limited Adaptive Histogram Equalization (CLAHE) method.

[0038] The CLAHE algorithm mainly consists of the following steps:

[0039] 1) Preprocess the original image, such as image block filling, etc.;

[0040] 2) Process each block and calculate the mapping relationship. Contrast limitation is used when calculating the mapping relationship.

[0041] 3) Use interpolation methods to obtain the final enhanced image.

[0042] The inventors noted that while the CLAHE algorithm can solve the aforementioned global and background noise enhancement problems, its hardware implementation is slow, which seriously affects the efficiency of image processing.

[0043] To address the aforementioned technical problems, at least one embodiment of this disclosure provides an image processing method, comprising: dividing an image to be processed into N image blocks; performing a mapping operation on the N image blocks based on the low-power mapping curve generation model to generate N low-power mapping curves with adaptive contrast limiting that correspond one-to-one with the N image blocks; performing mapping operations on the N image blocks based on the N low-power mapping curves with adaptive contrast limiting to obtain N low-power mapping image blocks with adaptive contrast limiting that correspond one-to-one with the N image blocks; and performing a stitching operation on the N low-power mapping image blocks with adaptive contrast limiting to obtain a low-power mapping stitched image; wherein N is an integer greater than 0.

[0044] At least one embodiment of this disclosure also provides an image processing apparatus, a display apparatus, and a storage medium.

[0045] The image processing method provided in the above embodiments of this disclosure is based on an artificial intelligence (AI) model (e.g., a low-power mapping curve generation model) to achieve contrast-limited adaptive low-power histogram image quality enhancement. It can directly generate end-to-end mapped processed images that reduce power consumption and improve image quality, which greatly improves the efficiency of image processing.

[0046] The embodiments and some examples of this disclosure will now be described in detail with reference to the accompanying drawings.

[0047] At least one embodiment of this disclosure provides an image processing method suitable for computer execution, which, for example, can be used to obtain a low-power mapped image with limited contrast adaptation, so as to use the obtained low-power mapped image with limited contrast adaptation for display, thereby reducing power consumption.

[0048] For example, this image processing method can be implemented in software, hardware, firmware, or any combination thereof, and can be loaded and executed by a processor in devices such as mobile phones, digital cameras, tablets, laptops, desktop computers, and network servers to reduce power consumption and improve image display quality, thereby greatly improving data processing efficiency.

[0049] For example, this image processing method is applicable to a computing device, which can be any electronic device with computing capabilities, such as a mobile phone, digital camera, laptop, tablet, desktop computer, network server, etc., capable of loading and executing the image processing method. The embodiments of this disclosure do not limit this. For example, the computing device may include a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), or other processing units with data processing and / or instruction execution capabilities, storage units, etc. The computing device may also have an operating system, application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.), etc., installed on it, implementing the image processing method provided by the embodiments of this disclosure by running code or instructions. For example, the computing device may also include an output component, such as a display component, such as a Liquid Crystal Display (LCD), Organic Light Emitting Diode (OLED) display, Quantum Dot Light Emitting Diode (QLED) display, etc., the embodiments of this disclosure do not limit this. For example, the display component can display a low-power mapped image with adaptive contrast limiting.

[0050] Figure 1 illustrates the overall architecture of an image processing method provided in at least one embodiment of this disclosure. For example, as shown in Figure 1, the image processing method includes image segmentation of the original image (using a segmentation algorithm), low-power mapping curve generation (using a low-power mapping curve generation model), and image stitching, and stitching the stitched images together using a stitching model to generate the final low-power mapped image.

[0051] Figure 2 shows a schematic diagram of an image processing method provided by at least one embodiment of the present disclosure. For example, as shown in Figure 2, the image processing method includes steps S110 to S140.

[0052] Step S110: Perform block processing on the image to be processed to divide the image into N image blocks.

[0053] Step S120: Based on the low-power mapping curve generation model, perform mapping operations on N image blocks to generate N low-power mapping curves with limited contrast adaptive that correspond one-to-one with the N image blocks.

[0054] Step S130: Based on N low-power mapping curves with limited contrast adaptation, perform mapping operations on N image blocks respectively to obtain N low-power mapping image blocks with limited contrast adaptation that correspond one-to-one with the N image blocks.

[0055] Step S140: Stitch together N low-power mapped image blocks with limited contrast adaptation to obtain a low-power mapped stitched image.

[0056] For example, N is an integer greater than 0.

[0057] For step S110, for example, in some examples, the image to be processed is the original image that needs to be converted into a low-power mapped stitched image.

[0058] For example, in some examples, step S110 includes: converting the image to be processed into a grayscale image; dividing the grayscale image into M image sub-blocks; calculating the pixel mean of the M image sub-blocks to obtain an image mean matrix; and performing a fusion operation based on the image mean matrix to obtain N image blocks.

[0059] For example, M is an integer greater than 0.

[0060] For example, the image to be processed is converted from an RGB image to a grayscale image; the grayscale image is divided into 8×8 blocks (it should be noted that the 8×8 is only illustrative, and other division sizes are also possible, and the embodiments of this disclosure are not limited to this); the pixel mean of each block is calculated to obtain an 8×8 image mean matrix; blocks with similar pixel mean values ​​are merged together to obtain the final block division result.

[0061] For example, in some examples, a fusion operation is performed based on the image mean matrix to obtain N image blocks, including: setting a fusion threshold; determining whether the pixel mean of adjacent image sub-blocks meets the fusion threshold and a preset condition; if so, fusion of adjacent image sub-blocks into one image block.

[0062] For example, in some examples, determining whether the average pixel value of adjacent image sub-blocks meets the fusion threshold is a preset condition, including: determining whether the difference between the average pixel values ​​of adjacent image sub-blocks is less than the fusion threshold; if it is less, then fusion the adjacent image sub-blocks into one image block.

[0063] For example, the fusion threshold can be set according to specific circumstances. For example, if the fusion threshold is set to 20, then as long as the difference between the mean values ​​of adjacent image blocks is less than 20, the two adjacent image blocks will be fused into one image block. Thus, the original image is divided into corresponding image blocks according to the block division result, as shown in Figures 3A and 3B.

[0064] Figures 3A and 3B are schematic diagrams of image segmentation provided in at least one embodiment of the present disclosure. For example, as shown in Figure 3A, similar content is divided into a block based on the pixel mean. For example, the original image is divided into 3 image blocks according to different content such as trees, cars, and grass. For example, region 1 includes trees, region 2 includes cars, and region 3 includes grass. Similarly, in Figure 3B, the original image is divided into 6 image blocks according to different content such as people, books, photos, ceilings, bookshelves, and Christmas trees.

[0065] The image segmentation method provided in this disclosure differs from the segmentation method in the existing CLAHE algorithm, which directly divides the image into equal parts. The image segmentation method of this disclosure can achieve an adaptive image segmentation scheme. Similar content is grouped into one block, making it easier to perform targeted enhancement, thereby enabling enhancement of different regions in the image.

[0066] For step S120, for example, in some examples, the input to the low-power mapping curve generation model is N image patches, and the output data of the low-power mapping curve generation model is the N low-power mapping curves.

[0067] Figures 4A and 4B are schematic diagrams of a low-power mapping curve provided by at least one embodiment of the present disclosure.

[0068] For example, as shown in Figure 4A, the low-power mapping curve generation model can obtain a low-power mapping curve with limited contrast adaptive corresponding to the image patch of the original image. For example, the low-power mapping curve is a 1*256 dimensional vector, but the specific configuration can be determined according to the actual situation, and the embodiments of this disclosure do not limit this. For example, as shown in Figure 4B, N image patches correspond to N low-power mapping curves with limited contrast adaptive.

[0069] For example, in some examples, the training method for this low-power mapping curve generation model includes: dividing the training image into X image blocks, where X is an integer greater than 0; performing a mapping operation on the X image blocks to generate X low-power mapping curves with limited contrast and adaptive characteristics, corresponding one-to-one with the X image blocks; and training the low-power mapping curve generation model based on the X image blocks and the X low-power mapping curves with limited contrast and adaptive characteristics. For example, the training data for the low-power mapping curve generation model includes X image blocks and X low-power mapping curves with limited contrast and adaptive characteristics.

[0070] For example, in some examples, a mapping operation is performed on X image blocks to generate X low-power mapping curves with limited contrast adaptive that correspond one-to-one with N image blocks. This includes: obtaining the grayscale frequency and peak value of the x-th image block among the X image blocks; allocating grayscale frequencies higher than the peak value to the low grayscale portion to obtain the reallocated grayscale frequencies; and obtaining the x-th low-power mapping curve with limited contrast adaptive that corresponds to the x-th image block based on the reallocated grayscale frequencies.

[0071] For example, in the existing CLAHE algorithm, the generation of the mapping curve involves the following steps: histogram statistics, peak limiting, pixel frequency redistribution, and cumulative distribution function generation. The image processing method provided in this disclosure adds a low-power mapping module after the cumulative distribution function is generated. This module corrects the grayscale distribution of the image, reduces the average brightness of the image, and thus lowers display power consumption.

[0072] Figure 5A is a schematic diagram of pixel frequency redistribution in the CLAHE algorithm; Figure 5B is a schematic diagram of pixel frequency redistribution provided by at least one embodiment of this disclosure.

[0073] For example, as shown in Figure 5A, when redistributing pixel frequencies, the CLAHE algorithm evenly distributes frequencies above the peak value to all other lower frequency gray levels. As shown in regions 1 and 2 of Figure 5A, gray level frequencies above the peak value are evenly distributed to the low gray level portion (region 1 in Figure 5A) and the high gray level portion (region 2 in Figure 5A). This is beneficial for improving contrast, but not for reducing power consumption. The gray level frequencies allocated to the high gray level portion further increase the average brightness of the image, which in turn increases the display power consumption of the image.

[0074] In embodiments of this disclosure, to reduce average power consumption, a low-power mapping module is provided, as shown in FIG5B. This module allocates grayscale frequencies higher than the peak value to the low-grayscale portion (region 1 in FIG5B) instead of the high-grayscale portion. This reduces the average brightness of the image, thereby reducing display power consumption. For example, if too many grayscale levels are allocated to the low-grayscale portion, causing the number of low-grayscale levels to exceed a threshold, the peak value can be increased, and then the allocation can be redistributed. Therefore, based on the redistributed grayscale frequencies, a low-power mapping curve with adaptive contrast limiting corresponding to the x-th image block is obtained, thereby reducing display power consumption.

[0075] For example, a low-power mapping curve generation model can be trained based on the X image patches and the X low-power mapping curves with limited contrast adaptation generated by the above method corresponding to the X image patches. This will yield an AI model for generating low-power mapping curves with limited contrast adaptation. Low-power mapping curves can then be generated directly based on this low-power mapping curve generation model, greatly improving the efficiency of image processing.

[0076] It should be noted that the input and output of the low-power mapping curve generation model are the original image and the low-power mapping curve with limited contrast adaptation, respectively. The structure of the low-power mapping curve generation model can be any open-source model or a network model designed based on actual conditions. The embodiments disclosed herein do not impose any restrictions on this.

[0077] For example, in some examples, after generating a low-power mapping curve with adaptive contrast based on the low-power mapping curve generation model, this curve is used to map the original image blocks (N image blocks) to obtain N low-power mapped image blocks with adaptive contrast, each corresponding to one of the N original image blocks. These N low-power mapped image blocks are then stitched together to form a complete image, thus completing the mapping operation from the original image to the low-power mapped stitched image. However, simply stitching together the image blocks will cause image breaks at the stitching points, thus affecting the display effect.

[0078] In embodiments of this disclosure, the image processing method further includes: performing a stitching operation on the low-power mapped spliced ​​image to obtain a low-power mapped image with limited contrast adaptation, thereby eliminating the stitching seams of the low-power mapped spliced ​​image and improving the display effect and display quality.

[0079] In the existing CLAHE algorithm, the stitched image is re-divided into blocks before bilinear interpolation is performed to eliminate seams. For example, each original image block is further divided into multiple (e.g., four blocks as an example below) sub-blocks. Each sub-block then forms a new block with the sub-blocks of its neighboring blocks. Bilinear interpolation is then applied to the newly formed blocks to obtain the enhanced image. For the sub-block in the top row of the first row, only the sub-blocks in the top row of the adjacent rows need to be considered; the same applies to the last row, the first column, and the last column. For the sub-blocks at the four corners, the mapping relationship of their own blocks is used directly without interpolation. However, this method still requires re-dividing and merging, increasing the complexity of image processing and reducing its efficiency.

[0080] In embodiments of this disclosure, to improve image processing efficiency, a stitching operation is performed on a low-power mapped image based on a stitching model to obtain a low-power mapped image with limited contrast adaptation. For example, the input to the stitching model is the low-power mapped image, and the output of the stitching model is the low-power mapped image with limited contrast adaptation.

[0081] For example, the stitching model includes a horizontal convolution kernels that convolve the horizontal seams and b vertical convolution kernels that convolve the vertical seams, where a is an integer greater than 0 and b is an integer greater than 0.

[0082] For example, in some embodiments of this disclosure, a training method for a stitching model is provided, comprising: segmenting a training image into Y image blocks, where Y is an integer greater than 0; performing a mapping operation on the Y image blocks to generate Y low-power mapping curves with limited contrast adaptive properties corresponding to the Y image blocks; performing mapping operations on the Y image blocks based on the Y low-power mapping curves to obtain Y low-power mapping image blocks with limited contrast adaptive properties corresponding to the Y image blocks; stitching the Y low-power mapping image blocks with limited contrast adaptive properties to obtain a low-power mapping stitched image; and training the stitching model based on the low-power mapping stitched image and the training image. For example, the training data pair for the low-power mapping curve generation model includes y columns of pixel data near the stitching seam of the low-power mapping stitched image and pixel data at the corresponding position in the training image, where y is an integer greater than 0.

[0083] For example, when training the stitching model, three columns of pixel data (i.e., taking y=6 as an example, but more or fewer columns are also possible, and the embodiments of this disclosure do not limit this) on the left and right or top and bottom of the vertical stitch are used as the model input data, and the pixel data of the corresponding position of the original image are used as the model label data to form the data pairs for training the stitching model. For example, the loss function used in the training is a structural similarity function, such as SSIM, but other functions can also be used, and the embodiments of this disclosure do not limit this.

[0084] It should be noted that the stitching model can be any neural network model in the art that can implement the stitching function. The following description uses an example consisting of eight 6×6 convolutional kernels (for example, a and b are parameters of the stitching model, which can be different values ​​depending on the actual effect, a = b = 8. It should be noted that a = b = 8 is only illustrative, and there can be more or fewer kernels. The embodiments of this disclosure do not limit this).

[0085] Figure 6 illustrates a schematic diagram of a stitching model provided in at least one embodiment of this disclosure. For example, as shown in Figure 6, the stitching model consists of *a* 6×6 convolutional kernels and *b* 6×6 convolutional kernels. For instance, during processing, *a* 6×6 convolutional kernels are used to perform sliding convolution processing on the horizontal stitching seams, and *b* 6×6 convolutional kernels are used to perform sliding convolution processing on the vertical stitching seams. The processed image is the final complete low-power mapping image with limited contrast adaptive processing. The low-power mapping image with limited contrast adaptive processing obtained based on this method can eliminate image faults caused by image stitching, thereby improving display effect and display quality. Furthermore, directly performing image stitching processing based on the stitching model can improve the efficiency of image stitching.

[0086] For specific processing procedures, please refer to the relevant descriptions of convolutional neural networks in this field, which will not be elaborated here.

[0087] The image processing method provided in the above embodiments of this disclosure achieves contrast-limited adaptive low-power histogram image quality enhancement based on AI models (e.g., low-power mapping curve generation model and stitching model). It can directly generate end-to-end mapped processed images that reduce power consumption and improve image quality, greatly improving the efficiency of image processing.

[0088] Figure 7 is a schematic block diagram of an image processing apparatus provided in at least one embodiment of the present disclosure. For example, in the example shown in Figure 7, the image processing apparatus 100 includes a segmentation unit 110, a first mapping unit 120, a second mapping unit 130, and a stitching unit 140. These units can be implemented by hardware (e.g., circuit) modules or software modules, as described in the following embodiments. For example, these units can be implemented by a central processing unit (CPU), a general-purpose graphics processor (GPGPU), a graphics processing unit (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions. For example, in the embodiments of the present disclosure, these units can be implemented by the graphics processing unit (GPU) described in the above embodiments.

[0089] The segmentation unit 110 is configured to perform segmentation processing on the image to be processed to divide the image into N image blocks. For example, the segmentation unit 110 can implement step S110, and its specific implementation method can be referred to the relevant description of step S110, which will not be repeated here.

[0090] The first mapping unit 120 is configured to perform mapping operations on the N image blocks based on the low-power mapping curve generation model to generate N low-power mapping curves with limited contrast adaptive properties that correspond one-to-one with the N image blocks. For example, the first mapping unit 120 can implement step S120, and its specific implementation method can be found in the relevant description of step S120, which will not be repeated here.

[0091] The second mapping unit 130 is configured to perform mapping operations on the N image blocks based on the N low-power mapping curves with adaptive contrast limits, to obtain N low-power mapped image blocks with adaptive contrast limits that correspond one-to-one with the N image blocks. For example, the second mapping unit 130 can implement step S130, and its specific implementation method can be found in the relevant description of step S130, which will not be repeated here.

[0092] The stitching unit 140 is configured to stitch together N low-power mapped image blocks with limited contrast adaptive to obtain a low-power mapped stitched image. For example, the stitching unit 140 can implement step S140, and its specific implementation method can be found in the relevant description of step S140, which will not be repeated here.

[0093] For example, in other examples, the image processing apparatus 100 further includes a stitching unit configured to perform a stitching operation on the low-power mapped stitched image to obtain a low-power mapped image with limited contrast adaptation. For example, the specific implementation method of the stitching unit 140 can be found in the relevant description of the stitching operation, and will not be repeated here.

[0094] Figure 8 is a schematic block diagram of another image processing apparatus provided in at least one embodiment of the present disclosure. For example, as shown in Figure 8, the image processing apparatus 200 includes a processor 210, a memory 220, and one or more computer program modules 221.

[0095] For example, processor 210 is connected to memory 220 via bus system 230. For example, one or more computer program modules 221 are stored in memory 220. For example, one or more computer program modules 221 include instructions for performing the image processing method provided in any embodiment of this disclosure. For example, the instructions in one or more computer program modules 221 can be executed by processor 210. For example, bus system 230 can be a commonly used serial or parallel communication bus, etc., and the embodiments of this disclosure are not limited thereto.

[0096] For example, the processor 210 may be a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), or other processing units with data processing capabilities and / or instruction execution capabilities. It may be a general-purpose processor or a dedicated processor, and may control other components in the image processing apparatus 100 to perform desired functions. Embodiments of this disclosure are described using the execution of the above-described image processing method by a processor for a graphics processing unit (GPU) as an example.

[0097] The memory 220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, which the processor 210 may execute to implement the functions (implemented by the processor 210) in this embodiment of the disclosure and / or other desired functions, such as image processing methods. Various application programs and various data may also be stored in the computer-readable storage medium, such as N image blocks, N low-power mapping curves with limited contrast adaptation, and various data used and / or generated by the application programs.

[0098] It should be noted that, for clarity and brevity, this disclosure does not show all the constituent units of the image processing apparatus 200. To achieve the necessary functions of the image processing apparatus 200, those skilled in the art can provide or set other constituent units (not shown) according to specific needs, and this disclosure does not limit this.

[0099] The image processing method or apparatus according to embodiments of this disclosure can also be implemented using the architecture of the exemplary electronic device 3000 shown in FIG9. As shown in FIG9, the electronic device 3000 may include a bus 3010, one or more central processing units (CPUs) or graphics processing units (GPUs) or GPGPUs 3020, read-only memory (ROM) 3030, random access memory (RAM) 3040, a communication port 3050 connected to a network, input / output components 3060, a hard disk 3070, etc. Storage devices in the electronic device 3000, such as the ROM 3030, the hard disk 3070, or the RAM memory (i.e., video memory) inside the GPGPU itself, can store various data or files required for processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU, GPU, or GPGPU. The electronic device 3000 may also include a user interface 3080. Of course, the architecture shown in FIG8 is only exemplary, and one or more components in the electronic device shown in FIG8 may be omitted as needed when implementing different devices.

[0100] This disclosure also provides a display device in at least one embodiment. FIG10 is a schematic diagram of a display device provided in at least one embodiment of this disclosure. For example, as shown in FIG10, the display device 1 includes the image processing device 100 / 200 provided in any of the above embodiments of this disclosure.

[0101] For example, the display device 1 also includes a timing controller 50, a gate driving circuit 20, and a data driving circuit 30. For example, the timing controller 50 is configured to drive the gate driving circuit 20 and the data driving circuit 30 to display a low-power mapped image with limited contrast adaptation.

[0102] For example, as shown in FIG10, the display device 1 further includes a plurality of sub-pixel units 410 arranged in an array. For example, the display device 1 further includes a display panel 40, in which a pixel array composed of a plurality of sub-pixel units 410 is disposed.

[0103] The output of each shift register unit in the gate driving circuit 20 is electrically connected to the sub-pixel unit 410 of each row. For example, the gate driving circuit 20 is electrically connected to the sub-pixel unit 410 through a gate line GL. The gate driving circuit 20 is used to provide a driving signal to the pixel array, such as driving the scanning transistor and the light-emitting transistor in the sub-pixel unit 410.

[0104] For example, the display device 1 may further include a data driving circuit 30 for providing data signals to the pixel array, such as pixel data of a low-power mapped image with limited contrast adaptation. For example, the data driving circuit 30 is electrically connected to the sub-pixel unit 410 via a data line DL.

[0105] For details on the specific driving process, please refer to the driving methods in this field, which will not be elaborated here.

[0106] It should be noted that the display device 1 in this embodiment can be any product or component with display function, such as a liquid crystal panel, liquid crystal TV, monitor, OLED panel, OLED TV, electronic paper display device, mobile phone, tablet computer, laptop computer, digital photo frame, navigator, etc.

[0107] For example, in other examples, the low-power mapping curves generated in the image processing device 100 / 200 can be stored in the timing controller 50. After the display device 1 acquires the input image, it performs a low-power mapping operation on the input image through the timing controller 50 to generate N low-power mapping curves with limited contrast adaptive corresponding to N image blocks. Based on the low-power mapping curves generated in the timing controller 50, the final low-power mapping image with limited contrast adaptive is obtained, and the image is input to the data driving circuit 30 to complete the low-power display.

[0108] The technical effects of the display device 1 provided in the embodiments of this disclosure can be referred to the corresponding descriptions of the image processing devices 100 / 200 in the above embodiments, and will not be repeated here.

[0109] At least one embodiment of this disclosure also provides a storage medium. FIG11 is a schematic diagram of a storage medium provided in at least one embodiment of this disclosure. For example, as shown in FIG11, the storage medium 400 non-transitoryly stores computer-readable instructions 401, which, when executed by a computer (including a processor), can perform the image processing method provided in any embodiment of this disclosure.

[0110] For example, the storage medium can be any combination of one or more computer-readable storage media. One computer-readable storage medium contains computer-readable program code that performs block processing on the image to be processed to divide the image into N image blocks. Another computer-readable storage medium contains computer-readable program code that performs mapping operations on the N image blocks based on the low-power mapping curve generation model to generate N low-power mapping curves with limited contrast adaptive properties corresponding to the N image blocks. The program code then performs mapping operations on the N image blocks based on the N low-power mapping curves with limited contrast adaptive properties to obtain N low-power mapping image blocks with limited contrast adaptive properties corresponding to the N image blocks. A third computer-readable storage medium contains computer-readable program code that performs a stitching operation on the N low-power mapping image blocks with limited contrast adaptive properties to obtain a low-power mapped stitched image. For example, when the program code is read by a computer, the computer can execute the program code stored in the computer storage medium to perform, for example, the image processing method provided in any embodiment of this disclosure.

[0111] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0112] Regarding the aforementioned publicly disclosed information, the following points need to be clarified:

[0113] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0114] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0115] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. An image processing method, comprising: The image to be processed is divided into N image blocks; Based on the low-power mapping curve generation model, a mapping operation is performed on the N image blocks to generate N low-power mapping curves with limited contrast and adaptive characteristics that correspond one-to-one with the N image blocks. Based on the N low-power mapping curves with limited contrast adaptation, the N image blocks are mapped to obtain N low-power mapping image blocks with limited contrast adaptation that correspond one-to-one with the N image blocks. The N low-power mapped image blocks with limited contrast adaptive are stitched together to obtain a low-power mapped stitched image. Where N is an integer greater than 0.

2. The image processing method according to claim 1, further comprising: The low-power mapped stitched image is stitched to obtain a low-power mapped image with limited contrast adaptation.

3. The image processing method according to claim 1, wherein, The image to be processed is divided into N image blocks, including: Convert the image to be processed into a grayscale image; The grayscale image is divided into M image sub-blocks; Calculate the pixel mean of the M image sub-blocks to obtain the image mean matrix; A fusion operation is performed based on the image mean matrix to obtain the N image blocks; Where M is an integer greater than 0.

4. The image processing method according to claim 3, wherein, The N image patches are obtained by performing a fusion operation based on the image mean matrix, including: Set the fusion threshold; Determine whether the average pixel value of adjacent image sub-blocks meets the preset condition with the fusion threshold; If the conditions are met, the adjacent image sub-blocks are merged into one image block.

5. The image processing method according to claim 4, wherein, Determining whether the pixel mean of adjacent image sub-blocks meets the preset condition with respect to the fusion threshold includes: Determine whether the difference in the average pixel value of the adjacent image sub-blocks is less than the fusion threshold. If it is less, then merge the adjacent image sub-blocks into one image block.

6. The image processing method according to any one of claims 1-5, wherein, The input to the low-power mapping curve generation model is the N image patches, and the output data of the low-power mapping curve generation model is the N low-power mapping curves.

7. The image processing method according to claim 2, wherein, Performing a stitching operation on the low-power mapped stitched image to obtain a low-power mapped image with limited contrast adaptation includes: The low-power mapping stitched image is stitched using a stitching model to obtain a low-power mapping image with limited contrast adaptation. The input of the stitching model is the low-power mapping stitched image, and the output of the stitching model is the low-power mapping image with limited contrast adaptation.

8. The image processing method according to claim 7, wherein, The stitching model includes a horizontal convolution kernels that convolve the horizontal seams and b vertical convolution kernels that convolve the vertical seams, where a is an integer greater than 0 and b is an integer greater than 0.

9. An image processing apparatus, comprising: The segmentation unit is configured to perform segmentation processing on the image to be processed to divide the image to be processed into N image blocks; The first mapping unit is configured to perform mapping operations on the N image blocks based on the low-power mapping curve generation model to generate N low-power mapping curves with limited contrast adaptive that correspond one-to-one with the N image blocks. The second mapping unit is configured to perform mapping operations on the N image blocks based on the N low-power mapping curves with limited contrast adaptation to obtain N low-power mapped image blocks with limited contrast adaptation that correspond one-to-one with the N image blocks. The stitching unit is configured to stitch together the N low-power mapped image blocks with limited contrast adaptive to obtain a low-power mapped stitched image. Where N is an integer greater than 0.

10. The image processing apparatus according to claim 9, further comprising: The stitching unit is configured to perform a stitching operation on the low-power mapped stitched image to obtain a low-power mapped image with limited contrast adaptation.

11. An image processing apparatus, comprising: processor; Memory; One or more computer program modules, the one or more computer program modules being stored in the memory and configured to be executed by the processor, the one or more computer program modules including instructions for performing the method of any one of claims 1-8.

12. A display device comprising the image processing apparatus according to any one of claims 9-11.

13. The display device according to claim 12, further comprising a timing controller, a gate driving circuit, and a data driving circuit, wherein, The timing controller is configured to drive the gate drive circuit and the data drive circuit to display the low-power mapping image with limited contrast adaptation.

14. A storage medium that non-transitory stores computer-readable instructions that, when executed by a computer, can perform the method according to any one of claims 1-8.

Images