Partition backlight processing method and device and electronic equipment

Abstract

The invention provides a partition backlight processing method and device and electronic equipment. The partition backlight processing method comprises the following steps: respectively carrying out histogram statistical processing on each color channel based on an input video to obtain a processed histogram; utilizing a backlight estimation algorithm to configure the backlight data of each partition based on the processed histogram so as to obtain backlight control intensity; and compensating an image pixel value of the input video based on the backlight control intensity by using a pixel compensation algorithm to obtain a partition backlight processing result. According to the partition backlight processing method disclosed by the invention, backlight control and self-adaptive compensation of various partition data can be automatically carried out, and the display image quality is remarkably improved.

Description

Technical Field

[0001] This disclosure belongs to the field of display technology, and relates to a method for processing local dimming backlight, and particularly to a method and apparatus for processing local dimming backlight, and electronic equipment. Background Technology

[0002] With the rapid development of LCD technology, in addition to significant advancements in display materials, local dimming technology, a key method for improving the contrast and image quality of LCD monitors, has been widely applied in high-end televisions, displays, and other fields. The basic principle of local dimming is to divide the backlight into multiple independently controlled zones, dynamically adjusting the brightness of each zone according to the content of the image. This reduces the backlight in corresponding areas when displaying dark scenes, achieving deeper blacks and higher dynamic contrast.

[0003] As local dimming technology becomes more widespread in high-end display devices, its limitations are becoming increasingly apparent. Current local dimming processing faces the dual challenges of improving image quality and increasing processing complexity. Regarding image quality, traditional local dimming technology has several inherent flaws. When the displayed content includes bright objects against a dark background, insufficient backlight control precision can lead to visible halos around object edges, severely compromising image clarity and immersion. In terms of processing flow, local dimming technology faces a significant challenge in terms of algorithmic complexity. The system needs to analyze the content features of the input image in real time, including brightness distribution, object outlines, and motion information. This analysis process places extremely high demands on computing resources. To generate precise backlight control signals, the processing algorithm needs to complete the computation and optimization of massive amounts of data in a very short time, placing enormous pressure on hardware processing capabilities. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and electronic device for localized backlight processing, which addresses the problems of poor image quality and high complexity in backlight processing in the prior art.

[0005] In a first aspect, this disclosure provides a partitioned backlight processing method. The partitioned backlight processing method includes: performing histogram statistical processing on each color channel based on an input video to obtain a processed histogram; configuring the backlight data for each partition using a backlight estimation algorithm based on the processed histogram to obtain a backlight control intensity; and compensating the image pixel values ​​of the input video based on the backlight control intensity using a pixel compensation algorithm to obtain a partitioned backlight processing result.

[0006] In one implementation of the first aspect, performing histogram statistical processing on each color channel based on the input video to obtain the processed histogram includes: performing format unification processing on the input video to obtain input data; performing weighted processing on the histograms of the RGB channels of the input data to obtain an initial block histogram; and remapping the display area of ​​the input video with the spatial resolution of the initial block histogram to obtain the processed histogram.

[0007] In one implementation of the first aspect, performing format unification processing on the input video to obtain input data includes: uniformly converting the format of the input video to RGB format, and converting a limited-range signal to a full-range signal to obtain the input data.

[0008] In one implementation of the first aspect, remapping the display area of ​​the input video with the spatial resolution of the initial block histogram to obtain the processed histogram includes: performing spatial resampling interpolation on the initial block histogram using a multinomial filtering method based on the display area of ​​the input video and the spatial resolution of the initial block histogram to obtain the processed histogram.

[0009] In one implementation of the first aspect, the backlight control intensity is obtained by configuring the backlight data rows of the partition based on the processed histogram using a backlight estimation algorithm. This includes: partitioning the processed histogram according to the location to obtain backlight data for different partitions; constraining the backlight data of different partitions using a point spread function corresponding to the light pattern to obtain a loss function for the backlight estimation algorithm; and updating the parameters of the backlight estimation algorithm by combining the loss function of the smoothness constraint and the loss function of the backlight estimation algorithm, so as to obtain the backlight control intensity using the backlight estimation algorithm.

[0010] In one implementation of the first aspect, obtaining the backlight control intensity using the backlight estimation algorithm includes: adaptively configuring the backlight data of the different partitions using the activation function layer and data transformation layer of the backlight estimation algorithm to obtain the backlight control intensity of the different partitions.

[0011] In one implementation of the first aspect, obtaining the backlight control intensity further includes: correcting the backlight control intensity of the current frame based on the backlight control intensity of the previous frame of the input video to obtain the backlight control intensity.

[0012] In one implementation of the first aspect, using a pixel compensation algorithm to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain a partitioned backlight processing result includes: spatially aligning the input video with the target resolution and the backlight control intensity to perform backlight processing on the input video; obtaining the backlight brightness of the input video before and after backlight processing based on the backlight control intensity; and compensating the image pixel values ​​of the input video by combining the backlight brightness before and after backlight processing and the display parameters of the input video to obtain the partitioned backlight processing result.

[0013] In one implementation of the first aspect, the partitioned backlight processing method further includes: performing a weighted calculation using a generative adversarial network loss function and a pixel compensation neural network loss function to generate a target loss function for correcting the partitioned backlight processing result.

[0014] Secondly, this disclosure provides a partitioned backlight processing apparatus. The partitioned backlight processing apparatus includes: a data processing module configured to perform histogram statistical processing on each color channel based on the input video to obtain a processed histogram; a backlight estimation module configured to use a backlight estimation algorithm to configure the backlight data of each partition based on the processed histogram to obtain a backlight control intensity; and a pixel compensation module configured to use a pixel compensation algorithm to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain a partitioned backlight processing result.

[0015] Thirdly, this disclosure provides an electronic device. The electronic device includes: a display interface; and a chip configured to perform a local dimming backlighting method according to any one of the first aspects, and to output the local dimming backlighting processing result to the display interface.

[0016] According to embodiments of this disclosure, deploying a lightweight neural network on an edge device significantly reduces the reliance on additional resources for partitioned backlight processing. It eliminates the need for repetitive and tedious calibration processes for different screen partitions, achieving end-to-end optimization. High-precision backlight estimation algorithms suppress halos, and adaptive, precise compensation is applied to the input video, significantly improving backlight processing image quality while mitigating halos caused by inter-pixel overlap between partitions.

[0017] Furthermore, a lightweight backlight compensation algorithm is employed to flexibly adjust backlight compensation, improving performance in dark scenes. Simultaneously, a joint loss function is introduced to adjust power consumption and image quality, achieving a dynamic balance between performance and power consumption. Attached Figure Description

[0018] Figure 1The diagram shown is a schematic representation of the partitioned backlight processing method described in an embodiment of this disclosure.

[0019] Figure 2 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure.

[0020] Figure 3 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure.

[0021] Figure 4 The diagram shown is a schematic representation of the interpolation operation described in an embodiment of this disclosure.

[0022] Figure 5 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure.

[0023] Figure 6 The diagram shown is a schematic representation of the backlight estimation algorithm described in an embodiment of this disclosure.

[0024] Figure 7 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure.

[0025] Figure 8 The diagram shown is a schematic representation of the partitioned backlight processing device according to an embodiment of this disclosure.

[0026] Figure 9A The diagram shown is a schematic representation of an input video image frame as described in an embodiment of this disclosure.

[0027] Figure 9B The diagram shown is a schematic representation of the backlight processing described in an embodiment of this disclosure.

[0028] Figure 9C The diagram shows a comparison between the results of traditional zoned backlighting and the zoned backlighting described in the embodiments of this disclosure.

[0029] Figure 10 The diagram shown is a structural schematic of the electronic device described in an embodiment of this disclosure. Detailed Implementation

[0030] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. This disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0031] It should be noted that, in the embodiments of this disclosure, the word "for example" or similar terms signify an example, illustration, or description. Any embodiment or design described as "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "for example" or similar terms is intended to present the relevant concepts in a concrete manner.

[0032] In this disclosure, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0033] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. Therefore, the drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0034] Traditional local dimming algorithms suffer from significant technical bottlenecks in areas such as optimal backlight extraction, halo suppression, brightness crosstalk elimination, and adaptive compensation. AI-driven local dimming algorithms, however, leverage the powerful feature extraction and pattern recognition capabilities of deep learning to offer a novel approach to solving these challenges. Existing algorithms often struggle to determine if the extracted brightness value is globally optimal, or require highly complex models and multiple iterations. Secondly, the "halo" effect caused by backlight diffusion is a long-standing problem plaguing local dimming technology. When a bright object appears against a dark background, the surrounding LED backlight cannot be completely switched off, causing light to "overflow" into adjacent dark areas, forming a blurred bright edge that severely degrades image purity and clarity. This problem is particularly pronounced when the mixing distance is poorly designed or the algorithm is insufficiently optimized. Furthermore, the mutual influence of light between adjacent backlight zones leads to "brightness crosstalk," causing the actual displayed brightness to differ from the target brightness, resulting in distortion of image brightness and contrast.

[0035] To address at least the aforementioned problems, this disclosure provides a partitioned backlight processing method. The partitioned backlight processing method includes: performing histogram statistical processing on each color channel based on the input video to obtain a processed histogram; configuring the backlight data for each partition using a backlight estimation algorithm based on the processed histogram to obtain a backlight control intensity; and compensating the image pixel values ​​of the input video based on the backlight control intensity using a pixel compensation algorithm to obtain a partitioned backlight processing result.

[0036] In this embodiment, a lightweight neural network is deployed on the edge device, significantly reducing the reliance on additional resources for partitioned backlight processing and achieving end-to-end optimization. A high-precision backlight estimation algorithm is used to suppress halos, and adaptive and precise compensation is applied to the input video, significantly improving the image quality of backlight processing while mitigating the halos problem caused by inter-pixel overlap between partitions.

[0037] Figure 1 The diagram shown is a schematic representation of the partitioned backlight processing method described in an embodiment of this disclosure. Figure 1 As shown, for both the HDMI IN and decoding input video paths, the first step is to determine if the input video format is RGB. If not, the input video undergoes format conversion to RGB, and the limited-range signal is converted to a full-range signal to obtain input data suitable for histogram statistical analysis. Histogram statistics are performed independently on the three RGB channels. To prevent histogram jumps caused by slight image changes, a weighted histogram method is used, assigning each point to adjacent blocks according to its weight. The histogram statistics are related to the number of points in the image statistics and the weight assigned to each point to adjacent blocks. After obtaining the initial histogram, a remapping process is performed based on the spatial resolution of the input video display area partition and the initial block histogram. A multinomial filtering method is used to perform spatial resampling interpolation on the initial block histogram to obtain the processed histogram.

[0038] In the backlight estimation algorithm processing, the processed histogram is partitioned for backlight processing. Based on the different spatial positions of the backlight lamps, three types are identified: center, edge, and corner, obtaining backlight data for each partition. A point spread function (PSF) corresponding to the light pattern is applied to constrain the backlight data for each partition to obtain the loss function for the backlight estimation algorithm. The parameters of the backlight estimation algorithm are updated by combining the loss function with the smoothness constraint on the spatial distribution of the backlight data and the loss function of the backlight estimation algorithm. The backlight control intensity of the current frame is corrected based on the backlight control intensity of the previous frame of the input video to obtain an accurate backlight control intensity using the backlight estimation algorithm. The backlight control intensity is converted into a backlight signal and transmitted through the Serial Peripheral Interface (SPI) to the TCON's MCU chip for backlight control.

[0039] During the pixel compensation algorithm processing, spatial alignment is performed based on the target resolution of the input video and the backlight control intensity to perform backlight processing on the input video. The backlight brightness of the input video before and after backlight processing is obtained based on the backlight control intensity. The image pixel values ​​of the input video are compensated by combining the backlight brightness before and after backlight processing with the display screen parameters to obtain the partitioned backlight processing result. After image quality enhancement, the input video is matched with the actual display panel size. If the actual display panel size differs from the size processed by the pixel compensation algorithm, the actual display panel size is fed back to the histogram partition remapping module for further processing.

[0040] The technical solutions in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0041] The following embodiments of this disclosure provide a method for processing localized backlighting. Figure 2 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure. Figure 2 As shown, the partitioned backlight processing method may include steps S11 to S13.

[0042] Step S11: Perform histogram statistical processing on each color channel based on the input video to obtain the processed histogram.

[0043] Step S12: Configure the backlight data of each partition based on the processed histogram using the backlight estimation algorithm to obtain the backlight control intensity.

[0044] Step S13: The pixel compensation algorithm is used to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain the partitioned backlight processing result.

[0045] In some possible implementations, the input video from both the HDMI IN and decoding paths requires histogram statistical processing for each color channel. Weighted histograms are independently calculated for each channel, and a multinomial filtering method is used to spatially resample and remap the block histograms to match the histograms to the display area size, resulting in a processed histogram. A backlight estimation algorithm is then used to configure the backlight data for each zone based on the processed histogram. A point spread function corresponding to the light pattern is used for constraint processing according to the different zones. The parameters of the backlight estimation algorithm are updated by combining the loss function of the backlight estimation algorithm and the loss function of the smoothness constraint to obtain the backlight control intensity of the input video. A pixel compensation algorithm is then used to compensate the image pixel values ​​of the input video based on the backlight control intensity, ensuring display quality while reducing backlight power, thus obtaining the zoned backlight processing results.

[0046] In this embodiment, a lightweight neural network is deployed on the edge device, significantly reducing the reliance on additional resources for partitioned backlight processing and achieving end-to-end optimization. A high-precision backlight estimation algorithm is used to suppress halos, and adaptive and precise compensation is performed on the input video, significantly improving the image quality of backlight processing while mitigating the halos problem caused by inter-pixel overlap between partitions.

[0047] Figure 3 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure. Figure 3 As shown, step S11 may include steps S111 to S113.

[0048] Step S111: Perform format unification processing on the input video to obtain input data.

[0049] Step S112: The histograms of the RGB channels of the input data are weighted separately to obtain the initial histogram.

[0050] Step S113: Remap the display area of ​​the input video to the spatial resolution of the initial block histogram to obtain the processed histogram.

[0051] In some possible implementations, it's determined whether the input video is in RGB format. If not, the input video format is uniformly processed to obtain the input data. If the input is a limited-range signal, it needs to be converted to a full-range signal before histogram statistical processing can be performed. The histograms of the RGB channels of the input data are weighted separately, with each point assigned to adjacent blocks according to its weight. The latter half of the first block and the first half of the next block are integrated into one block. The statistical results of each histogram block are related to the number of points counted and the weight of each point distributed to adjacent blocks. For example, a mainstream 2000-partition size is based on a 32×48 partition configuration, and the target resolution of the input video is 3840×2160. Since the hardware histogram design may be applied to various modules and needs to be compatible with both portrait and landscape screen resolutions, the histogram statistical blocks are designed to be square, such as 32×32. The number of pixels in the horizontal direction (31) and the vertical direction (31) are the same. To avoid the last pixel being too large or failing to count any valid pixels, the size of the first 31 pixels is calculated using the following formula:

[0052] ,

[0053] The horizontal dimension of the block The width of the input video image. The downsampling step size in the horizontal direction. This is the floor function. It is a rounding function. This represents the vertical dimension of the block. The image height of the input video. This represents the downsampling step size in the vertical direction.

[0054] For the initial histogram Since there is a difference between the display area of ​​the input video and the resolution of the initial histogram, it is necessary to remap the display area of ​​the input video to the resolution of the initial histogram in order to obtain the processed histogram.

[0055] In this embodiment, a method of performing histogram statistics independently for multiple channels is used, which significantly improves the problem of insufficient color display due to high saturation in a single channel compared to the method of using histogram statistics for the Y channel alone. Furthermore, using weighted histogram statistics can prevent histogram jumps caused by slight image changes, resulting in more accurate histogram statistics.

[0056] In one embodiment of this disclosure, the input video is processed to obtain input data by uniformly converting the input video to RGB format and converting a limited-range signal to a full-range signal to obtain input data.

[0057] In some possible implementations, the input video is in YUV format, and the formula for converting YUV to RGB format is:

[0058] ,

[0059] The coefficients for standard RGB and YUV conversion. The coefficients for input range conversion.

[0060] In this embodiment of the disclosure, histogram statistics are performed independently using the three RGB channels, which can better match the backlight processing requirements of LED lights that are displayed in RGB and improve the adaptability of backlight processing.

[0061] In one embodiment of this disclosure, remapping the display area of ​​the input video with the spatial resolution of the initial block histogram to obtain the processed histogram includes: performing spatial resampling interpolation on the initial block histogram using a multinomial filtering method based on the spatial resolution of the display area of ​​the input video and the initial block histogram to obtain the processed histogram.

[0062] In some possible implementations, a multinomial filtering method is used to perform spatial resampling interpolation on the initial block histogram to obtain a processed histogram. The interpolation is performed in both the horizontal and vertical directions. Taking the horizontal interpolation as an example... Figure 4 The diagram shows an interpolation operation as described in an embodiment of this disclosure. Figure 4 As shown, the blue areas represent the interpolation points, determined using the eight original pixels before and after the interpolation. The resulting histogram after interpolation is shown below. The formula for interpolation is:

[0063] ,

[0064] The point to be interpolated. For the original pixel, These are the interpolation coefficients for each pixel.

[0065] Figure 5 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure. Figure 5 As shown, step S12 may include steps S121 to S123.

[0066] Step S121: The processed histogram is partitioned according to the location to obtain backlight data for different partitions.

[0067] Step S122: Constrain the backlight data of different partitions using the point spread function of the corresponding light pattern to obtain the loss function of the backlight estimation algorithm.

[0068] Step S123: Combine the loss function of the smoothness constraint and the loss function of the backlight estimation algorithm to update the parameters of the backlight estimation algorithm, so as to obtain the backlight control intensity using the backlight estimation algorithm.

[0069] In some possible implementations, the input to the backlight estimation algorithm is the processed histogram. The output is the backlight control intensity. If the backlight control is white light, it outputs a single-channel backlight control intensity; if the backlight control is an RGB LED display, it outputs a three-channel backlight control intensity. Figure 6 The diagram shown illustrates the backlight estimation algorithm described in an embodiment of this disclosure. Figure 6 As shown, the processed histogram is partitioned according to the spatial position of the backlight, obtaining backlight data for three partitions: center, edge, and corner. For the backlight data of different partitions, a point spread function corresponding to the light pattern is applied for constraint processing to obtain the loss function of the backlight estimation algorithm. The target loss function is obtained by combining the loss function of the smoothness constraint and the loss function of the backlight estimation algorithm to update the parameters of the backlight estimation algorithm. The formula for calculating the target loss function is:

[0070] ,

[0071] Let be the loss function of the backlight estimation algorithm. The loss function is subject to smoothness constraints. For network hyperparameters, .

[0072] In one embodiment of this disclosure, obtaining the backlight control intensity using a backlight estimation algorithm includes: adaptively configuring the backlight data of different partitions using the activation function layer and data transformation layer of the backlight estimation algorithm to obtain the backlight control intensity of different partitions.

[0073] In this embodiment, the activation function layer and data transformation layer of the last layer of the backlight estimation algorithm are adaptively configured for different partitions to meet the backlight requirements of any partition, which greatly reduces the workload of backlight estimation algorithm modeling.

[0074] In one embodiment of this disclosure, obtaining the backlight control intensity further includes: correcting the backlight control intensity of the current frame based on the backlight control intensity of the previous frame of the input video to obtain the backlight control intensity.

[0075] In some possible implementations, the backlight control intensity is adjusted in the time domain. An infinite impulse response filter is used to modify the backlight control intensity of the current frame based on the backlight control intensity of the previous frame of the input video, thus obtaining the correct backlight control intensity. The formula for calculating the backlight control intensity correction is as follows:

[0076] ,

[0077] The backlight control intensity for the current frame. This is the backlight control intensity of the previous frame. For parameters.

[0078] In this embodiment of the disclosure, the backlight control intensity of the current frame is corrected by combining the backlight control intensity of the previous frame, which can improve the problems of screen flickering and ghosting, and avoid cross-zone problems caused by sudden changes in the screen or screen movement.

[0079] Figure 7 The diagram shown is a schematic flowchart of the partitioned backlight processing method described in an embodiment of this disclosure. Figure 7 As shown, step S13 may include steps S131 to S133.

[0080] Step S131: Spatial alignment is performed based on the target resolution of the input video and the backlight control intensity to perform backlight processing on the input video.

[0081] Step S132: Obtain the backlight brightness before and after backlight processing of the input video based on the backlight control intensity.

[0082] Step S133: Combine the backlight brightness before and after backlight processing of the input video with the display parameters of the input video to compensate the image pixel values ​​of the input video in order to obtain the partitioned backlight processing result.

[0083] In some possible implementations, after obtaining the backlight control intensity, the pixel values ​​of the image are compensated to ensure that the actual light intensity remains unchanged before and after backlight processing, i.e., the visual display effect remains the same. After obtaining the two-dimensional backlight control intensity, it needs to be enlarged to the original image size, and spatial alignment is performed according to the target resolution of the input video and the backlight control intensity to obtain the corresponding pixel value. and The ratio is used as a compensation amount for pixel values ​​to perform backlight processing on the input video. The formula for spatial alignment is:

[0084] ,

[0085] The backlight brightness before backlight processing. This refers to the backlight brightness after backlight processing. The gamma value is the parameter used to input the video display screen. The R-channel image before spatial alignment to account for screen effects. For spatially aligned R-channel images that take screen effects into account, The G-channel image before spatial alignment to account for screen effects. The spatially aligned G-channel image taking screen effects into account. The B-channel image before spatial alignment to account for screen effects. To account for the spatial alignment of the B-channel image after considering screen effects, , and These are the spatially aligned R, G, and B channel images, respectively. , and These are the R, G, and B channels of the image before spatial alignment.

[0086] After compensating the image pixel values ​​of the input video, features are extracted through the convolutional layer of the backlight compensation algorithm to rapidly increase the channel dimension, and the magnification process is quickly achieved by using data rearrangement operations.

[0087] In one embodiment of this disclosure, the partitioned backlight processing method further includes: performing a weighted calculation using a generative adversarial network loss function and a pixel compensation neural network loss function to generate a target loss function for correcting the partitioned backlight processing result.

[0088] In some possible implementations, since the backlight control intensity plays a decisive role in the relationship between image quality and backlight power consumption, a generative adversarial network loss function is added to constrain the backlight control intensity so that the generated backlight control intensity can not only guarantee image quality, but also minimize the power consumption of backlight processing.

[0089] Furthermore, neural networks are used to correct the RGB three-channel images to avoid issues with detail, layering, or halo effects in dark areas caused by screen transformations. The loss functions used in the pixel value compensation stage of the input video include a generative adversarial network loss function and a loss function from the pixel compensation neural network. The formula for calculating the loss function is:

[0090] ,

[0091] Let be the loss function of the pixel compensation neural network. To generate the loss function for adversarial networks, For network hyperparameters, Configured as .

[0092] Figure 8 The diagram shown is a structural schematic of the partitioned backlight processing device described in an embodiment of this disclosure. Figure 8 As shown, the partitioned backlight processing device 100 includes a data processing module 110, a backlight estimation module 120, and a pixel compensation module 130.

[0093] The data processing module 110 is configured to perform histogram statistical processing on each color channel based on the input video to obtain the processed histogram.

[0094] The backlight estimation module 120 is configured to use a backlight estimation algorithm to configure the backlight data for each partition based on the processed histogram to obtain the backlight control intensity.

[0095] The pixel compensation module 130 is configured to use a pixel compensation algorithm to compensate for the image pixel values ​​of the input video based on the backlight control intensity in order to obtain the partitioned backlight processing result.

[0096] It should be understood that the data processing module 110, the backlight estimation module 120, and the pixel compensation module 130 can be further used to perform the corresponding steps or actions in the methods described in the above embodiments, which will not be repeated here.

[0097] In some possible implementations, the input video from both the HDMI IN and decoding paths requires histogram statistical processing for each color channel. Weighted histograms are independently calculated for each channel, and a multinomial filtering method is used to spatially remap the block histograms to match the display area size, resulting in a processed histogram. A backlight estimation algorithm is then used to configure the backlight data for each zone based on the processed histogram. A point spread function corresponding to the light pattern is used for constraint processing according to the different zones. The parameters of the backlight estimation algorithm are updated by combining the loss function of the backlight estimation algorithm and the loss function of the smoothness constraint to obtain the backlight control intensity of the input video. A pixel compensation algorithm is then used to compensate the image pixel values ​​of the input video based on the backlight control intensity, ensuring display quality while reducing backlight power, thus obtaining the zoned backlight processing results.

[0098] In some other possible implementations, Figure 9A The diagram shown is a schematic representation of an input video image frame as described in an embodiment of this disclosure. Figure 9B The diagram shown is a schematic representation of the backlight processing described in an embodiment of this disclosure. Figure 9CThis is a comparison chart showing the results of traditional local dimming backlighting and the local dimming backlighting described in the embodiments of this disclosure. The display screen of an input video image frame is as follows. Figure 9A As shown, the backlight control intensity after the partitioned backlight processing described in this embodiment is as follows: Figure 9B As shown. By Figure 9C It is evident that the contrast and clarity of the image processed by traditional partitioned backlighting are inferior to the backlighting results of the partitioned backlighting method described in the embodiments of this disclosure.

[0099] In this embodiment, a lightweight neural network is deployed on the edge device, significantly reducing the reliance on additional resources for partitioned backlight processing. This eliminates the need for repetitive and tedious calibration processes for different screen partitions, achieving end-to-end optimization. A high-precision backlight estimation algorithm is used to suppress halos, and adaptive and precise compensation is applied to the input video, significantly improving the image quality of backlight processing while mitigating the halos problem caused by inter-pixel overlap between partitions.

[0100] Figure 10 The diagram shown is a structural schematic of the electronic device described in an embodiment of this disclosure. Figure 10 As shown, the electronic device 200 includes a display interface 210 and a chip 220. The chip 220 is configured to execute the partitioned backlight processing method according to any embodiment of the present disclosure and output the partitioned backlight processing result to the display interface 210.

[0101] In this embodiment of the disclosure, a lightweight AI neural network is used to construct a backlight estimation algorithm and a backlight compensation algorithm, which enables the deployment of partitioned backlight technology on the edge device, greatly reducing the dependence of the partitioned backlight algorithm on additional resources and reducing costs.

[0102] Furthermore, compared to the traditional local dimming algorithm which requires repeated adjustments based on different LCD panel materials and different local dimming zones, this disclosure uses an AI algorithm to adaptively solve the local dimming problem, significantly reducing the deployment difficulty and improving the halo problem caused by light crosstalk between local dimming zones.

[0103] Furthermore, addressing the issue of current pixel compensation relying excessively on the accuracy of grayscale coefficient curves, this disclosure employs a lightweight backlight compensation algorithm to flexibly adjust backlight compensation, improving performance in dark scenes. Simultaneously, a joint loss function is introduced to adjust power consumption and image quality, achieving a dynamic balance between performance and power consumption.

[0104] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0105] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this disclosure, depending on actual needs. For example, the functional modules / units in the various embodiments of this disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0106] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0107] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0108] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0109] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A method for processing zoned backlight, characterized in that, include: Histogram statistics are performed on each color channel based on the input video to obtain the processed histogram. The backlight control intensity is obtained by configuring the backlight data of each partition based on the processed histogram using a backlight estimation algorithm. as well as A pixel compensation algorithm is used to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain the partitioned backlight processing result.

2. The partitioned backlight processing method according to claim 1, characterized in that, Histogram statistics are performed on each color channel of the input video to obtain the processed histograms, including: The input video is processed to unify its format in order to obtain input data; The histograms of the RGB channels of the input data are weighted separately using weights to obtain initial block histograms; and The display area of ​​the input video is remapped to the spatial resolution of the initial block histogram to obtain the processed histogram.

3. The partitioned backlight processing method according to claim 2, characterized in that, The input video is processed to unify its format in order to obtain input data, including: The input video is uniformly converted to RGB format, and the limited-range signal is converted to the full-range signal to obtain the input data.

4. The partitioned backlight processing method according to claim 2, characterized in that, Remapping the display area of ​​the input video with the spatial resolution of the initial block histogram to obtain the processed histogram includes: Based on the display area of ​​the input video and the spatial resolution of the initial block histogram, a multinomial filtering method is used to perform spatial resampling interpolation on the initial block histogram to obtain the processed histogram.

5. The partitioned backlight processing method according to claim 1, characterized in that, The backlight control intensity is obtained by configuring the backlight data rows of the partition based on the processed histogram using a backlight estimation algorithm, including: The processed histogram is partitioned according to location to obtain backlight data for different partitions; For backlight data from different partitions, point spread functions corresponding to the light patterns are used for constraint processing to obtain the loss function of the backlight estimation algorithm; and The parameters of the backlight estimation algorithm are updated by combining the loss function of the smoothness constraint and the loss function of the backlight estimation algorithm, so as to obtain the backlight control intensity using the backlight estimation algorithm.

6. The partitioned backlight processing method according to claim 5, characterized in that, Obtaining the backlight control intensity using the backlight estimation algorithm includes: The activation function layer and data transformation layer of the backlight estimation algorithm are used to adaptively configure the backlight data of different partitions to obtain the backlight control intensity of different partitions.

7. The partitioned backlight processing method according to claim 5, characterized in that, Obtaining the backlight control intensity further includes: The backlight control intensity of the current frame is corrected based on the backlight control intensity of the previous frame of the input video to obtain the backlight control intensity.

8. The partitioned backlight processing method according to claim 1, characterized in that, The pixel compensation algorithm is used to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain the partitioned backlight processing result, including: Spatially align the input video with the target resolution and the backlight control intensity to perform backlight processing on the input video; The backlight brightness before and after the input video backlight processing is obtained based on the backlight control intensity; and The image pixel values ​​of the input video are compensated by combining the backlight brightness before and after the input video backlight processing and the display parameters of the input video to obtain the partitioned backlight processing result.

9. The partitioned backlight processing method according to claim 1, characterized in that, Also includes: A weighted calculation is performed using the loss function of a generative adversarial network and the loss function of a pixel compensation neural network to generate a target loss function for correcting the partitioned backlight processing results.

10. A zoned backlight processing device, characterized in that, include: The data processing module is configured to perform histogram statistical processing on each color channel based on the input video to obtain the processed histogram. The backlight estimation module is configured to use a backlight estimation algorithm to configure the backlight data of each partition based on the processed histogram in order to obtain the backlight control intensity. as well as The pixel compensation module is configured to use a pixel compensation algorithm to compensate the image pixel values ​​of the input video based on the backlight control intensity to obtain the partitioned backlight processing result.

11. An electronic device, characterized in that, include: Display interface; as well as The chip is configured to perform the partitioned backlight processing method according to any one of claims 1 to 9 and output the partitioned backlight processing result to the display interface.

Images