A display device

Abstract

Embodiments of the present application disclose a display device, comprising a display panel, a backlight module, a controller and a memory, wherein the memory stores a network model and human eye visual perception HK effect data, the controller acquires image data, inputs the image data and the HK effect data into the network model, extracts image feature vectors and HK feature vectors, performs feature fusion after obtaining the fused features, processes the fused features through the network model to obtain target image data, generates backlight values and pixel driving values according to the target image data, and controls the backlight module and the display panel to display images respectively. The scheme can not only improve the display effect of the picture, but also improve the experience of the user.

Description

Technical Field

[0001] This invention relates to the field of display device technology, and more specifically to a display device. Background Technology

[0002] As Mini LED (Mini Light Emitting Diode) television display technology continues to develop, users' demands for television picture quality continue to increase.

[0003] Most related technologies rely on local dimming to control backlight brightness, while using color peaking technology to optimize image color and brightness. They only rely on the color information of the image itself to complete image quality enhancement processing, which makes the optimized image color prone to visual deviation and can also easily cause visual fatigue, resulting in a poor user experience. Summary of the Invention

[0004] This application discloses a display device that can not only improve the display effect but also enhance the user experience.

[0005] A first aspect of this application discloses a display device comprising: Display panel, used to display images; A backlight module is used to provide backlight for the display panel; The controller is connected to both the display panel and the backlight module. The memory is located inside the controller. The memory stores network models and human visual perception HK effect data. The HK effect data is used to characterize human visual perception of different color categories and different color intensities under the same brightness. The controller is configured as follows: Acquire image data, The image data and the HK effect data are input into the network model; The network model extracts image feature vectors and HK feature vectors respectively and performs feature fusion to obtain fused features. The image feature vectors are extracted by the network model from the image data, and the HK feature vectors are extracted by the network model from the HK effect data. The fusion features are processed using the network model to obtain target image data; Generate backlight values ​​and pixel driving values ​​based on the target image data; The backlight module is controlled according to the backlight value, and the display panel is driven according to the pixel drive value to display the target image.

[0006] In this solution, the display device is equipped with a display panel, a backlight module, and a controller with built-in memory. The memory stores a network model and human visual perception HK effect data. The controller first acquires image data and inputs it along with the HK effect data into the network model. After extracting image feature vectors and HK feature vectors, feature fusion is performed to obtain fused features. The network model then processes the fused features to obtain target image data. Next, backlight values ​​and pixel drive values ​​are generated based on the target image data, thereby controlling the backlight module and display panel to complete image display. Unlike traditional solutions that rely solely on image information for image quality adjustment, this solution combines image data with HK effect data related to human visual perception. This allows for adjustments to display parameters that align with the actual color perception patterns of the human eye, effectively improving color perception deviations, reducing color distortion and visual fatigue, and steadily enhancing the optimization of color and brightness, resulting in a final visual performance that better meets human visual perception needs.

[0007] As an optional implementation, in a first aspect of the embodiments of this application, the network model includes a first preprocessing module and a second preprocessing module. The step of extracting image feature vectors and HK feature vectors through the network model and fusing them to obtain fused features is specifically configured as follows: The image data is processed by the first preprocessing module to extract features, thereby obtaining the image feature vector; and... The HK effect data is encoded using the second preprocessing module to obtain the HK feature vector. The image feature vector and the HK feature vector are concatenated to obtain the fused feature.

[0008] In this scheme, the network model includes a first preprocessing module and a second preprocessing module. During data fusion, the first preprocessing module extracts image feature vectors from the image data, while the second preprocessing module encodes the HK effect data to obtain HK feature vectors. The two types of feature vectors are then concatenated to obtain the fused feature. It can be understood that by processing the two types of data separately through different modules, the characteristics of both the image data and the data related to human visual perception are fully preserved. The concatenation and fusion method effectively combines the two types of data information, providing complete and effective basic feature information for the subsequent generation of accurate target image data.

[0009] As an optional implementation, in a first aspect of the embodiments of this application, the first preprocessing module includes an image preprocessing submodule and a feature mapping submodule. The step of performing feature extraction processing on the image data through the first preprocessing module to obtain the image feature vector is specifically configured as follows: The image preprocessing submodule performs size scaling and normalization on the image data to obtain the processed image data. The feature mapping submodule performs feature mapping on the processed image data to obtain the image feature vector.

[0010] In this scheme, the first preprocessing module consists of an image preprocessing submodule and a feature mapping submodule. When extracting image feature vectors, the image preprocessing submodule first performs size scaling and normalization on the image data, and then the feature mapping submodule performs feature mapping on the processed image data to finally obtain the image feature vectors. It can be understood that performing normalization on the image data first unifies the image data format, eliminates the influence of data differences, and allows for accurate extraction of effective image information during subsequent feature mapping, ensuring that the obtained image feature vectors are standardized and accurate, facilitating normal splicing and fusion with other feature data.

[0011] As an optional implementation, in a first aspect of this application embodiment, the second preprocessing module includes an encoding module, wherein the encoding processing of the HK effect data by the second preprocessing module to obtain the HK feature vector is specifically configured as follows: The HK effect data is converted into HK feature vectors by one-hot encoding using the encoding module.

[0012] In this scheme, the second preprocessing module includes an encoding module. When encoding the HK effect data, the encoding module directly converts the HK effect data into binary feature vectors, thereby obtaining the HK feature vectors. This conversion method unifies the data format of the HK effect data, ensuring a standardized format for the generated HK feature vectors. This allows for successful splicing with image feature vectors, guaranteeing the normal operation of the two data fusion process.

[0013] As an optional implementation, in a first aspect of the embodiments of this application, the network model further includes a training module, the memory stores preset reference image data, and the controller is further configured to: The preset reference image data is input into the training module; The training module processes the preset reference image data, the target image data, and the HK effect data to obtain the loss function value. Based on the loss function value, the learnable parameters of the network model are updated using the backpropagation algorithm.

[0014] In this scheme, the network model has a built-in training module, and the memory stores preset reference image data. The controller inputs the preset reference image data into the training module, which then calculates the loss function value by combining this data with the target image data and HK effect data. Based on the loss function value, the backpropagation algorithm is used to update the learnable parameters of the network model. It can be understood that by using reference data to complete the model training iterations, the various parameters of the network model can be continuously corrected, the difference between the processed image and the standard image can be reduced, and the network model can learn data processing rules that conform to visual perception, gradually improving the accuracy of image data processing and making the final output target image data more reasonable and accurate.

[0015] As an optional implementation, in the first aspect of this application embodiment, the step of processing the preset reference image data, the target image data, and the HK effect data through the training module to obtain the loss function value is specifically configured as follows: A color accuracy loss value is calculated based on the color component values ​​of the target image data and the color component values ​​at the corresponding pixel positions in the preset reference image. The color accuracy loss value is used to characterize the difference between the color component values ​​of the target image data and the color component values ​​at the corresponding pixel positions in the preset reference image. Based on the target image data, the preset reference image, and the HK effect data, a perceptual loss value based on the HK effect is calculated. The perceptual loss value based on the HK effect is used to characterize the difference between the target image data and the preset reference image data in terms of human visual perception features. The norm of the learnable parameter is calculated to obtain the regularization loss value, which is used to constrain the range of values ​​of the learnable parameter. The color accuracy loss value, the perception loss value, and the regularization loss value are weighted and summed according to their respective preset weights to obtain the loss function value.

[0016] In this scheme, when the training module calculates the loss function value, it first compares the color component values ​​of corresponding pixels in the target image data with those in the preset reference image to obtain the color accuracy loss value. Then, it combines the three types of data to calculate the perceptual loss value based on the HK effect, and simultaneously calculates the norm of the learnable parameters to obtain the regularization loss value. Finally, it sums the three types of loss values ​​according to preset weights to obtain the final loss function value. This approach can be understood as simultaneously considering the degree of image color matching, differences in human visual perception, and the range of parameter values, controlling the model training direction from multiple dimensions. It ensures that the image colors conform to the standard reference image and the effect of human visual perception, while also limiting abnormal parameter values, effectively improving the training effect and operational stability of the network model.

[0017] As an optional implementation, in the first aspect of this application embodiment, the network model further includes an attention module and a color adjustment module, wherein the process of processing the fused features through the network model to obtain target image data is specifically configured as follows: The attention module processes the fused features to generate multiple attention weights. The color adjustment module processes the multiple attention weights, the fusion features, and the image data to obtain the target image data.

[0018] In this scheme, the network model also includes an attention module and a color adjustment module. When obtaining target image data from fused features, the attention module first processes the fused features to generate multiple attention weights. Then, the color adjustment module combines the attention weights, fused features, and image data for computation to obtain the target image data. It can be understood that by using attention weights to distinguish the importance of different features, and then working with the color adjustment module to complete data integration computation, it is possible to selectively adjust effective image information, accurately complete the color and brightness adjustment of the image, and make the final target image data more in line with visual display requirements.

[0019] As an optional implementation, in a first aspect of this application, the attention module includes a channel attention module and / or a spatial attention module, and the process of the fused features by the attention module to generate attention weights is specifically configured as follows: The fusion features are processed by the channel attention module to obtain multiple channel attention weights. The channel attention weights are used to characterize the importance weight of different color attributes to color enhancement. Each color attribute corresponds to one of a variety of abstract visual attributes, including brightness attribute, saturation attribute and hue attribute. And / or, the fusion features are processed by the spatial attention module to obtain multiple spatial attention weights, which are used to characterize the importance weight of different spatial locations to color enhancement, and each spatial location corresponds to a local region in the image data.

[0020] In this scheme, the attention module includes a channel attention module and / or a spatial attention module. The channel attention module processes the fusion features to obtain channel attention weights, which reflect the importance of each color attribute in color enhancement. Each channel corresponds to visual attributes such as brightness, saturation, and hue. The spatial attention module processes the fusion features to obtain spatial attention weights, which reflect the importance of different local areas of the image in color enhancement. It can be understood that channel attention weights can distinguish the optimization priority of various visual attributes, while spatial attention weights can differentiate the adjustment focus of different areas of the image. This allows for targeted and differentiated adjustments to different image attributes and locations, making the color enhancement operation more closely aligned with the actual display requirements of the image.

[0021] As an optional implementation, in a first aspect of the embodiments of this application, the attention module further includes a weight adjustment module, and the controller is further configured to: The weight adjustment module is used to obtain the perceived importance coefficients of different color combinations in the HK effect data. The weight adjustment module multiplies the perceived importance coefficient with the channel attention weight at the same position, and / or multiplies the perceived importance coefficient with the spatial attention weight at the same position to obtain the multiple attention weights.

[0022] In this scheme, the attention module also includes a weight adjustment module. The controller first extracts the perceptual importance coefficients corresponding to different color combinations from the HK effect data through this module, and then multiplies these coefficients by the corresponding channel attention weights or spatial attention weights to obtain the required multiple attention weights. It can be understood that incorporating perceptual importance coefficients related to human visual perception into the weight calculation allows for the correction of the original attention weights based on the actual visual perception of different colors, making the generated attention weights more closely aligned with human visual perception and ensuring that subsequent image color adjustments better conform to actual visual perception patterns.

[0023] As an optional implementation, in a first aspect of the embodiments of this application, the color adjustment module includes a feature fusion submodule, a parameter generation submodule, and a color mapping submodule. The step of processing the multiple attention weights, the fusion features, and the image data through the color adjustment module to obtain target image data is specifically configured as follows: The feature fusion submodule fuses the multiple attention weights with the fused features to obtain attention-enhanced features. The attention enhancement features are convolved by the parameter generation submodule to generate color adjustment parameters, which include at least one of hue adjustment, saturation adjustment and brightness adjustment. The color mapping submodule adjusts the color component values ​​of corresponding pixels in the image data according to the color adjustment parameters to obtain the target image data.

[0024] In this scheme, the color adjustment module consists of a feature fusion submodule, a parameter generation submodule, and a color mapping submodule. First, the feature fusion submodule weights and fusion features to obtain attention-enhanced features. Then, the parameter generation submodule performs convolution processing on these features to generate color adjustment parameters for hue, saturation, and brightness categories. Finally, the color mapping submodule adjusts the color components of the image data pixels according to the adjustment parameters to obtain the target image data. In essence, this method relies on attention weights to highlight key optimization areas and visual attributes, outputs adjustment parameters that conform to visual rules through convolution operations, and then performs precise color correction pixel by pixel. This approach balances the overall display effect with differentiated color adjustments for different pixels, effectively conforming to the human eye's perception characteristics corresponding to the HK effect, reducing color deviation, and making the color and brightness adjustments of the image more natural and harmonious. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic structural diagram of a display device provided in an embodiment of this application; Figure 2 A schematic structural diagram of another display device provided in the embodiments of this application; Figure 3 A flowchart illustrating a control method provided in an embodiment of this application; Figure 4 A flowchart illustrating another control method provided in an embodiment of this application; Figure 5 A flowchart illustrating another control method provided in an embodiment of this application; Figure 6 A flowchart illustrating another control method provided in an embodiment of this application; Figure 7 This is a block diagram of a control method provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] With the rapid development of Mini LED (Mini Light Emitting Diode) TV display technology and the rapid popularization of various large-screen high-definition display devices, consumers' daily viewing and audio-visual entertainment scenarios are becoming increasingly diverse. The market has raised higher standards for the color performance, contrast ratio, and visual comfort of TV images during long-term viewing, and various image quality optimization technologies have been widely promoted and applied.

[0029] Among existing display image quality optimization technologies, the industry generally adopts local dimming to achieve regional control of backlight brightness, while simultaneously using color peaking technology to enhance the image quality of colors and bright areas. This traditional optimization method relies solely on the image's inherent color information for image quality adjustment. The resulting output color often deviates from the actual visual perception of the human eye, resulting in unnatural and uncoordinated color reproduction. Prolonged viewing can also cause eye strain, and the overall image optimization effect is limited, failing to meet the current user demands for high-quality and comfortable viewing.

[0030] In order to improve the user experience while enhancing the display effect, this application provides a display device. Unlike traditional solutions that rely solely on image information for image quality adjustment, this solution combines image data with Hall Kerr (HK) effect data related to human visual perception. This allows for the adjustment of display parameters in accordance with the actual color perception rules of the human eye, effectively improving the problem of color perception deviation, reducing color distortion and visual fatigue, and simultaneously and steadily improving the optimization effect of image color and brightness, so that the final visual performance of the image is more in line with the needs of human visual perception.

[0031] The display device provided in this application will be described in detail below with reference to the accompanying drawings and embodiments, so as to make the purpose and technical solution of this application clearer and more intuitive. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0032] It is understood that the display devices provided in the embodiments of this application refer to devices with screen display and data processing capabilities, including but not limited to liquid crystal display devices, liquid crystal televisions, liquid crystal monitors, mobile phones, tablet computers, laptops, vehicle-mounted display devices, smart screens, advertising machines, digital signage, monitoring displays, and head-mounted display devices, etc., without specific limitations.

[0033] To provide a clear understanding of the display device provided in the embodiments of this application, a schematic structural diagram of the display device is provided in the embodiments of this application. Please refer to... Figure 1 The display device includes a display panel 10, a backlight module 20, a controller 30, and a memory 40.

[0034] The display panel 10 can be located on the light-emitting side of the backlight module 20 for displaying images. The backlight module 20 is used to provide backlight for the display panel 10. The controller 30 is connected to the display panel 10 and the backlight module 20 respectively. The memory 40 is located inside the controller 30. The memory 40 stores network models and human visual perception HK effect data. The HK effect data is used to characterize the human visual perception corresponding to different color categories and different color intensities under the same brightness. In some embodiments, the display panel 10 may include a plurality of pixel units and a pixel driving circuit. Each pixel unit is composed of a red sub-pixel, a green sub-pixel, a blue sub-pixel, and a white sub-pixel. The pixel driving circuit is connected to the plurality of pixel units respectively and is configured to drive the liquid crystal molecules of each sub-pixel in each pixel unit to rotate, so as to adjust the light transmittance of each sub-pixel.

[0035] In the embodiments, the shape and size of the display panel 10 are generally adapted to the shape and size of the display device. For example, when applied to fields such as televisions or mobile terminals, the display panel 10 can be configured as a rectangle, including a top side, a ground side, a left side, and a right side, wherein the top side and the ground side are opposite to each other, the left side and the right side are opposite to each other, the top side is connected to one end of the left side and one end of the right side respectively, and the ground side is connected to the other end of the left side and the other end of the right side respectively.

[0036] In some embodiments, the display panel 10 can be a transmissive display panel, capable of modulating the transmittance of light, but not emitting light itself. The display panel 10 has multiple pixel units arranged in an array, and pixel driving circuits that provide scanning and driving signals to the pixel units. Each pixel unit is composed of four sub-pixels: red, green, blue, and white. These four sub-pixels are independently configured and are responsible for basic color reproduction and screen brightness gain, respectively.

[0037] The pixel driving circuit is connected to each of the multiple pixel units distributed in the array, and outputs driving voltage signals according to a preset row-by-row and column-by-column scanning timing. The driving signals output by the pixel driving circuit can be applied to each sub-pixel in each pixel unit, changing the deflection angle of the liquid crystal molecules inside the sub-pixel. By adjusting the different degrees of rotation of the liquid crystal molecules, the light transmittance of each sub-pixel can be directly adjusted. By individually adjusting the light transmittance of each sub-pixel, images with different brightness, colors, and grayscale levels can be created and stitched together to form a complete display image.

[0038] Optionally, the pixel driving circuit can be a thin-film transistor driving circuit, a gate array driving circuit, etc. There are no specific limitations here. As long as it can realize the timing scanning, voltage output and independent driving of sub-pixels, it can be adapted to the display panel architecture of this application.

[0039] For example, such as Figure 2 As shown, the display panel 10 may also include a liquid crystal layer (LC) and a color filter layer (CF). The liquid crystal layer is set corresponding to each pixel unit. The liquid crystal layer may be composed of liquid crystal molecules placed between two conductive glass sheets. The pixel driving circuit is connected to each pixel unit. The pixel driving circuit applies an electric field to the two electrodes on both sides of the pixel unit. Under the action of the electric field, the liquid crystal molecules at the corresponding pixel unit position rotate and deflect, thereby adjusting the transmittance of the backlight emitted by the backlight module. In conjunction with the light splitting effect of each sub-pixel, the image is presented.

[0040] The filter layer is located above the liquid crystal layer. The filter layer is matched and configured one-to-one with each pixel unit. The filter layer can filter the white light emitted by the backlight, so that light of a specific wavelength, that is, a specific color, can pass through each sub-pixel. In this way, the normal display of color images can be achieved by relying on the pixel units distributed in the entire array.

[0041] It should be noted that, Figure 2 The display panel shown is merely an exemplary structure, and the display panel may also have other structures. For example, the display panel may not include a filter layer. The structure of the display panel in this application embodiment is not specifically limited.

[0042] In some embodiments, the backlight module 20 includes a backlight driving circuit and a plurality of backlight zones. Each backlight zone is provided with at least one light-emitting unit. Each light-emitting unit includes at least two light-emitting chips, and the light-emitting chips correspond one-to-one with at least two color channels. The backlight driving circuit is electrically connected to the plurality of backlight zones and is used to drive each light-emitting chip in the light-emitting unit of each backlight zone to emit light.

[0043] The backlight module 20 is typically located at the bottom of the display device, and its shape and size usually match the shape and size of the display panel 10. For example, when applied to fields such as televisions or mobile terminals, the backlight module 20 can be configured as a rectangle.

[0044] The backlight module 20 includes a backlight driving circuit and multiple light-emitting units. Each light-emitting unit may contain at least two light-emitting chips, each chip being used to emit light of different wavelengths. The backlight driving circuit is connected to each light-emitting unit and is used to independently drive each light-emitting chip therein to emit light.

[0045] In some embodiments, the light-emitting unit may be provided with three light-emitting chips corresponding to the R, G, and B color channels respectively, namely a red light-emitting chip, a green light-emitting chip, and a blue light-emitting chip, and the backlight driving circuit may drive and control these three chips respectively.

[0046] In some embodiments, the backlight module 20 includes a backplate, a backlight driving circuit, and multiple light-emitting units.

[0047] The backplate can be located at the bottom of the backlight module, serving a supporting and load-bearing function. It can adopt a rectangular structure adapted to the display device. For direct-lit backlight modules, the backlight driving circuit is located on the backplate and is used to provide driving signals to the light-emitting units. The light-emitting units are located on the backlight driving circuit and are electrically connected. Each light-emitting unit can contain a light-emitting chip with three color channels. The backlight driving circuit can output driving signals separately to drive each chip independently.

[0048] In some embodiments, the backlight module 20 may also be provided with a diffuser plate located on the light-emitting side of the light-emitting unit. The diffuser plate is used to scatter light to improve the uniformity of light emission, while maintaining a distance from the light-emitting unit to achieve sufficient light mixing. For example, a quantum dot diffuser plate may be used.

[0049] In some embodiments, the backlight module 20 may also have an optical film disposed above the diffuser plate. The optical film may be at least one of a fluorescent film, a quantum film, a prism sheet, or a brightness enhancement film. The combination of the diffuser plate and the optical film can improve light utilization, optimize light emission uniformity, enhance brightness and color performance, and make the overall brightness distribution of the display panel more uniform.

[0050] In addition, the backlight area of ​​the backlight module 20 can be divided into multiple backlight zones, and each backlight zone can contain one or more light-emitting units. For example, the light-emitting units of a backlight zone can be 1, 2, 4, etc. There is no specific limitation here, and it can be set according to the actual situation.

[0051] Each light-emitting unit has multiple light-emitting chips. The control module can independently control the operation of each light-emitting unit according to the backlight zone, and drive the light-emitting chips of different color channels in each light-emitting unit.

[0052] In this embodiment, the controller 30 is connected to both the pixel driving circuit and the backlight driving circuit.

[0053] The controller 30 is the core component for the main control and processing of the entire display device. It is connected to the pixel driving circuit and the backlight driving circuit through internal signal lines to realize signal interaction and command transmission.

[0054] Optionally, the controller 30 may be selected from controllers with high-speed data processing and signal processing capabilities, such as dedicated display main control chips and image processing main control chips, without specific restrictions.

[0055] The controller 30 can acquire raw image data from external input in real time, and complete a series of image processing and logic control tasks such as image data parsing, feature calculation, parameter conversion and control command generation. At the same time, it can retrieve data and models stored in the memory to complete image quality optimization calculations. Finally, it outputs pixel driving commands to the display panel and backlight adjustment commands to the backlight module, and uniformly manages the backlight adjustment process and image imaging driving process of the whole machine.

[0056] The memory 40 is built into the controller 30 and is used to store the programs, calculation models and various reference data required for operation.

[0057] Optionally, the memory 40 can be an embedded flash memory, a one-time programmable memory, an erasable programmable read-only memory, a static random access memory, a dynamic random access memory, or an embedded multimedia memory card, etc., to meet the needs of adapting read and write speeds to image computing scenarios, and has the performance advantages of long-term data storage, high-speed retrieval, and stable read and write, which can meet the usage requirements of network model loading, batch visual perception data storage, and real-time retrieval.

[0058] The memory 40 in this embodiment of the application pre-stores a network model that has been trained and optimized, as well as human visual perception HK effect data. The pre-stored HK effect data is specifically used to characterize the differences in actual visual perception of the human eye for different color categories and different color intensities under the same brightness environment. It can provide standard reference data for the controller to carry out image enhancement calculations that conform to the laws of human vision.

[0059] The HK effect data can be collected in various ways. For example, a multi-person subjective visual evaluation experiment can be organized, in a uniform and constant brightness environment, allowing evaluators to score the visual perception of color samples with different hues and saturations, and then summarizing and organizing the data to form perception data. Alternatively, professional optical vision testing instruments can be used to simulate the imaging principle of human vision and batch detect the visual differences in various colors under the same brightness. Alternatively, standard measured data from publicly available visual perception research databases and color vision experiment literature can be retrieved, organized, and summarized. Alternatively, simulation calculations can be performed using a simulated visual model to generate HK effect-related data that conforms to visual laws in batches.

[0060] The above section provides a schematic illustration of the structure of the display device. The control method provided in the embodiments of this application will now be described. To clearly understand the control method of the control module of this display device, a schematic flowchart of the control method of the control module disclosed in the embodiments of this application is provided, as follows: Figure 3 As shown, the flowchart includes at least the following steps.

[0061] Step S101: Acquire image data.

[0062] In this embodiment, the image data is the raw image data to be processed transmitted from an external image signal source to the display device controller. This image data is the basic data source for the display panel imaging and may include the R, G, and B color components, pixel grayscale information, spatial texture information, and screen brightness distribution information corresponding to all pixel positions in the image, etc., without specific limitations.

[0063] In this embodiment, the image data can be acquired by any of the following methods: wired signal reception, wireless signal transmission, and local storage retrieval; no specific limitation is imposed here.

[0064] Among them, the receiving device for acquiring image data can be a high-definition decoding chip, an image signal receiving interface, a video image processing chip, etc., and no specific restrictions are made here.

[0065] For example, the controller acquires image data of a single frame or multiple consecutive frames in real time, and decodes, corrects the format and filters the noise of the raw image data to ensure that the input data format is uniform and the data accuracy is stable.

[0066] Step S102: Input the image data and HK effect data into the network model.

[0067] In this embodiment, the HK effect data is pre-stored in the controller's built-in memory. The HK effect data is used to characterize the differences in human visual perception corresponding to different color categories and different color intensities under the same brightness.

[0068] The network model is a deep learning image processing network that has been pre-trained offline and has its parameters fixed. This network model is deployed in the computing unit inside the controller.

[0069] This application does not limit the specific type of network model. For example, the network model can be any one of the following: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Convolutional Long Short-Term Memory (ConvLSTM), Lightweight Convolutional Neural Network, Residual Neural Network (ResNet), Recurrent Neural Network (RNN), and Fully Connected Neural Network (FCN), without any specific limitation.

[0070] For example, the controller retrieves the network model and HK effect data from the memory, and synchronously inputs the original image data and HK effect data into the network model according to the model's preset input channel and data arrangement format, so as to realize the parallel input of the original image imaging information and human visual perception information, and provide a data foundation for subsequent feature fusion operations.

[0071] Step S103: Extract the image feature vector and HK feature vector respectively through the network model and perform feature fusion to obtain the fused features.

[0072] Among them, the image feature vector is extracted by the network model from the image data, and the HK feature vector is extracted by the network model from the HK effect data; In some embodiments, the network model is configured with two preprocessing modules to independently preprocess image data and HK effect data. One preprocessing module performs scaling, normalization, and feature mapping on the image data, removes irrelevant noise, and extracts image feature vectors containing brightness, hue, and saturation. The other preprocessing module encodes and transforms the HK effect data through an encoding module to generate standardized binary HK feature vectors. The network model, following a predefined feature concatenation logic, aligns and concatenates the image feature vectors and HK feature vectors, achieving deep fusion of objective image imaging features and subjective human visual perception features, resulting in a fused feature. This fused feature simultaneously preserves the original image texture details, color distribution, and the human visual color perception patterns corresponding to the HK effect, eliminating dimensional differences between the two types of data and providing high-dimensional, high-precision feature basis for subsequent refined image quality adjustment.

[0073] In another embodiment, besides the feature vector concatenation and fusion method described above, a feature weighted fusion method can also be used to obtain the fused features. Specifically, the network model first extracts the image feature vector and the HK effect feature vector respectively. Based on the perceptual importance coefficients of different colors within the HK effect data, differentiated fusion weights are assigned to the two types of feature vectors, increasing the weight of features corresponding to colors that are sensitive to the human eye and decreasing the weight of features corresponding to colors that are not sensitive to the human eye. Then, the two types of weighted features are superimposed element by element to generate the weighted fused features. This method differs from simple concatenation, as it can preferentially retain visual features that are sensitive to the human eye, weaken ineffective and redundant features, and adapt to the non-uniform perceptual characteristics of the human eye.

[0074] In another embodiment, a cross-attention fusion approach can be used to obtain fusion features. The network model establishes a mapping relationship between image features and HK perception features through an attention interaction matrix, calculates the correlation between image pixel colors and HK color perception data, and filters out image features with a high degree of matching with human visual perception. These correlated features are then coupled and bound to generate cross-attention fusion features. This approach can uncover implicit perceptual correlations between color and brightness, improving fusion accuracy in complex color scenes.

[0075] In another embodiment, a feature concatenated coding fusion method can be used to obtain fused features. The network model uses a coding compression layer to compress the dimensions of the image feature vector and the HK feature vector respectively, compressing them to the same feature dimension; then, deep coupling is achieved through feature channel superposition and channel mixing to generate compact fused features. This method can reduce the computational redundancy of the two types of data, reduce the computing power consumption of the controller, and adapt to the real-time processing scenarios of low-computing-power display main control chips.

[0076] In other alternative embodiments, residual fusion can also be used to obtain fused features. The network model uses the original image features as residual base features and the HK feature vector as perceptual correction features. The correction features are superimposed on the base features through residual connections. While preserving the details and textures of the original image, the HK perceptual features are used to correct the original color deviations, ultimately generating residual fused features. This fusion method can avoid the problem of image detail loss during feature fusion, balancing image quality fidelity and human visual optimization.

[0077] Step S104: Process the fusion features through a network model to obtain target image data.

[0078] The network model is equipped with a feature optimization processing structure for deep analysis and reconstruction of high-dimensional fused features. The fused features simultaneously carry objective image imaging information and human visual perception information. Based on the feature relationships within the fused features, the network model can mine color differences, brightness distribution, and visually sensitive areas in the image, and adaptively correct, filter, and reconstruct the fused features using preset visual optimization rules. Through multi-level computational processing of the fused features, it can compensate for image quality defects in the original image, optimize color performance and brightness levels, weaken image features that do not conform to the laws of human visual perception, and ultimately generate target image data that conforms to the standard format and is compatible with the display device's driving logic. The target image data is optimized image data after intelligent image quality calibration, capable of matching the backlight module's brightness control logic and the display panel's pixel rendering logic, ensuring the normal execution of subsequent display imaging processes.

[0079] In some embodiments, a feature smoothing reconstruction method can be used to process the fused features. The network model suppresses abrupt changes and noise features in the fused features through a smoothing filter layer, eliminating abnormal pixel fluctuations in the image; simultaneously, it smoothly completes missing texture features according to feature distribution patterns, obtaining optimized features with continuous texture and smooth color transitions. Finally, the target image data is generated through mapping and conversion. This method can eliminate color banding and brightness noise in the image, improve image purity, and is suitable for processing images with high contrast and strong brightness.

[0080] In other embodiments, a visual threshold constraint adjustment method can be used to process the fused features. The network model, combined with HK effect data, presets a human visual perception threshold. It then compresses and constrains color and brightness features within the fused features that exceed the comfortable perception range of the human eye, while enhancing features within the sensitive perception range. By balancing image brightness and color intensity through threshold constraints, target image data adapted to the human eye's viewing threshold is generated. This method can effectively suppress glaringly bright areas and reduce visual fatigue.

[0081] In other embodiments, a residual correction generation method can be used to process the fused features. The network model uses the fused features as the base feature map, continuously calculates the residual deviation between the optimized features and the features of the standard reference image, and continuously compensates for and corrects the color and brightness residuals. After iteratively correcting the feature deviations, stable target image data is generated. This method can continuously reduce the optimization error, improve image color accuracy, and make the imaging effect more in line with standard display quality requirements.

[0082] Step S105: Generate backlight value and pixel driving value based on target image data.

[0083] In this embodiment, the backlight value is a control parameter used to control the brightness of the backlight module. The backlight value includes a global basic backlight brightness value and an independent brightness adjustment value for each backlight zone. The magnitude of the backlight value directly determines the brightness of the light source output by the backlight module.

[0084] Pixel drive values ​​are electrical signal parameters used to drive the pixel units of the display panel. The pixel drive value corresponds to the conduction voltage of the color channel of each pixel and is used to control the color ratio, gray level and display intensity of a single pixel.

[0085] For example, the controller can incorporate a backlight conversion algorithm and a pixel drive mapping algorithm. The controller analyzes the global brightness information, zone brightness and darkness information, and pixel color information in the target image data, and calculates and generates adjustable backlight values ​​for each zone based on the Mini LED backlight zone arrangement rules. Simultaneously, according to the display panel pixel drive timing rules, it generates matching pixel drive values ​​pixel by pixel. The backlight values ​​and pixel drive values ​​are generated independently and do not interfere with each other, and are used to control the backlight source and panel pixel imaging, respectively.

[0086] Step S106: Control the backlight module according to the backlight value and drive the display panel according to the pixel drive value to display the target image.

[0087] For example, the backlight module can adopt a Mini LED partitioned backlight structure. The backlight module can include a backlight driver chip, LED light-emitting beads, a light guide plate and a light-diffusing film, which can realize independent light control of each partition. The backlight module receives the backlight value sent by the controller and adjusts the working current of each partition LED bead through the driver chip to complete the precise control of partition brightness, providing a backlight source with uniform layering and brightness matching for the display panel.

[0088] The display panel can be a liquid crystal display panel. The display panel is equipped with a pixel driving circuit, a liquid crystal deflection unit and a color filter. The display panel receives the pixel driving value, controls the deflection angle of the liquid crystal molecules according to the electrical signal, and adjusts the light transmittance of each color channel to achieve accurate pixel color rendering.

[0089] In this embodiment, the controller synchronously sends backlight control signals and pixel drive signals to keep the backlight brightness change of the backlight module and the pixel color change of the display panel synchronized in time, so as to work together to complete the image imaging.

[0090] Understandably, unlike traditional solutions that rely solely on image information for image quality adjustment, this solution combines image data with HK effect data related to human visual perception for computational processing. This allows for adjustments to display parameters that align with the actual color perception patterns of the human eye, effectively improving color perception deviations, reducing color distortion and visual fatigue, and simultaneously enhancing the optimization of image color and brightness, resulting in a final visual performance that better meets the needs of human visual perception.

[0091] In some embodiments, the network model may include a first preprocessing module and a second preprocessing module. During data fusion, the first preprocessing module extracts image data to obtain image feature vectors, and the second preprocessing module encodes HK effect data to obtain HK feature vectors. The two types of feature vectors are then concatenated to obtain the fused feature. To clearly understand this method, embodiments of this application provide a flowchart of another controller control method, such as... Figure 4 As shown, the flowchart includes at least the following steps.

[0092] Step S201: Acquire image data.

[0093] Step S202: Input the image data and HK effect data into the network model.

[0094] In this embodiment, the description of steps S201-S202 can be the same as the description of steps S101-S102 above, and will not be repeated in detail here.

[0095] Step S203: The image data is processed by the first preprocessing module to extract features and obtain the image feature vector; In this embodiment, the first preprocessing module is a functional unit within the network model specifically responsible for preprocessing and feature parsing image data. This module is integrated and runs within the image processing and computing unit of the controller, and can achieve the entire computing process by relying on the internal logic operation circuit and parallel data processing circuit of the main controller.

[0096] In some embodiments, the first preprocessing module includes an image preprocessing submodule and a feature mapping submodule. The controller performs size scaling and normalization on the image data through the image preprocessing submodule to obtain processed image data. Then, the feature mapping submodule performs feature mapping on the processed image data to obtain image feature vectors.

[0097] In this embodiment, the image preprocessing submodule performs size scaling and normalization on the image data. That is, the image preprocessing submodule can use interpolation scaling operation unit and numerical normalization operation unit to adapt to various types of original image data with different resolutions and signal formats.

[0098] In actual processing, the controller first calls the image preprocessing submodule to complete the preliminary basic processing. This submodule integrates an interpolation scaling unit and a numerical normalization calculation unit. The interpolation scaling unit is activated here to scale and transform the original image data according to the preset input size of the network, unifying the overall resolution and pixel arrangement specifications of the image. Simultaneously, the numerical normalization calculation unit is activated to perform interval-based uniform conversion on the pixel color and brightness values ​​after scaling, eliminating data amplitude differences. After the two units work together, processed image data with uniform specifications and standard values ​​is obtained.

[0099] In other embodiments, in addition to the interpolation scaling unit and the numerical normalization operation unit to complete the size unification and numerical regularization, the image preprocessing submodule may also be equipped with functional structures such as a pixel noise filtering unit, a color space conversion unit, an edge enhancement correction unit, a color gamut calibration unit, and an image cropping and alignment unit. There are no specific limitations here. It can simultaneously complete multiple preprocessing tasks such as contrast adjustment, brightness equalization, dynamic range compression, and multi-frame data alignment. The corresponding functions can be flexibly selected according to the actual use scenario to comprehensively purify image data, unify data format and color benchmark, and simplify invalid image information, thereby improving the overall quality of the input image and laying a good data foundation for the network model in the subsequent feature mapping submodule to carry out accurate and efficient feature extraction.

[0100] Optionally, the feature mapping submodule can adopt a convolutional feature mapping structure or a fully connected feature mapping structure. That is, the feature mapping submodule can be equipped with any suitable feature extraction network such as a convolutional neural network, a lightweight visual network, or a residual network.

[0101] In actual processing, standard image data after scaling and normalization is used as the network input source. Relying on the network's own multi-layer operation structure, the network completes the work of visual information mining, abstract attribute extraction and dimension transformation, and finally transforms the two-dimensional pixel data into a fixed-dimensional image feature vector.

[0102] It is understandable that performing normalization on the image data first can unify the image data format, eliminate the impact of data differences, and then perform feature mapping to accurately extract effective image information, ensuring that the obtained image feature vectors are standardized and accurate, which facilitates normal splicing and fusion with other feature data.

[0103] Step S204: Encode the HK effect data using the second preprocessing module to obtain the HK feature vector; In this embodiment, the second preprocessing module is a dedicated functional processing unit within the network model that performs format conversion and information organization specifically for HK effect data. This module also relies on the data encoding and operation circuit and logic conversion circuit inside the controller to complete various operations, and can stably realize the standardized conversion operation of various visual perception data.

[0104] In some embodiments, the second preprocessing module has a built-in encoding module as the core execution structure. After the controller retrieves the HK effect data stored in the memory, it directly hands it over to the encoding module to perform unified encoding conversion. The encoding conversion method used in this embodiment is one-hot encoding, that is, the encoding module performs one-hot encoding conversion on the discrete classification form of HK effect data, mapping the discrete data of color category, color intensity, and visual perception level originally recorded in the form of numerical matrix, level label, experimental statistical values, etc., into a standardized one-hot feature vector with fixed dimension and single effective location representation, and finally forming a HK feature vector with a fixed structure.

[0105] It is understandable that HK effect data is discrete classification data representing differences in human color perception. There is no strict numerical logical relationship between various color levels and brightness perception levels. Directly using ordinary numerical encoding can easily lead to network misjudgments. After one-hot encoding conversion by the encoding module, the disordered discrete perception data can be standardized into a uniform, unbiased vector. This eliminates problems such as messy formats, inconsistent dimensions, and numerical logical interference in the original perception data. The resulting HK feature vector has a standardized data structure and fixed data dimensions, perfectly matching the data specifications of subsequent image feature vectors. This provides a reliable data foundation for the smooth splicing and fusion of the two types of feature data, effectively ensuring the smooth and orderly progress of the subsequent multi-source feature fusion process.

[0106] In other alternative embodiments, the encoding module can also use various encoding methods such as hash encoding, dimensionality compression encoding, sequence encoding, and ordinary binary encoding to complete the data conversion, and is not limited to one-hot encoding. Other encoding methods can be flexibly selected according to the network model's computational requirements, data storage volume, and data continuity characteristics, as long as the resulting HK feature vector has a fixed dimension and a uniform format after encoding.

[0107] It is understandable that this conversion method can unify the data format of HK effect data, making the generated HK feature vector format standardized and consistent, which can be successfully spliced ​​with image feature vectors, ensuring the normal progress of the fusion process of the two types of data.

[0108] Step S205: Concatenate the image feature vector with the HK feature vector to obtain the fused features.

[0109] In this embodiment, the image feature vector and the HK feature vector are generated through different preprocessing processes, and the data information they carry has different meanings. The image feature vector mainly carries objective imaging information such as the color distribution, texture details, and brightness levels of the original image, while the HK feature vector mainly carries discrete classification information of human subjective visual perception corresponding to different colors under the same brightness conditions. After both types of feature vectors have completed preprocessing and achieved a unified data dimension and format, the controller calls the feature splicing operation unit inside the network model to perform the splicing operation.

[0110] The splicing process used in this application strictly follows the feature channel arrangement order and feature dimension combination rules preset by the network model. Under the premise of completely preserving all the effective information of the two types of feature vectors without loss, it completes the feature dimension alignment and orderly merging of feature channels, thereby realizing the orderly combination of objective image features and human eye perception features.

[0111] This standardized and orderly splicing method deeply integrates the inherent imaging features of an image with the visual perception features of the human eye, bridging the data gap between the actual image information and the visual optimization reference information. It eliminates the differences in arrangement and adaptation between the two types of feature data, ultimately generating a fused feature with richer information dimensions and more comprehensive representation. This fused feature combines the characteristics of realistic image imaging with the basis for human visual calibration, and its overall structure is regular and unified. It can be directly input into the subsequent optimization and computation units of the network model for in-depth analysis and image quality adjustment.

[0112] It is understandable that, in addition to concatenating the image feature vector with the HK feature vector to obtain the fused features, various other fusion methods can be used, such as feature weighted fusion, cross-correlation fusion, dimension superposition fusion, and residual embedding fusion. No specific restrictions are imposed here.

[0113] Step S206: Process the fusion features using a network model to obtain target image data; Step S207: Generate backlight value and pixel driving value based on target image data; Step S208: Control the backlight module according to the backlight value and drive the display panel according to the pixel drive value to display the target image.

[0114] In this embodiment, the description of steps S206-S208 can be the same as the description of steps S104-S106 above, and will not be repeated in detail here.

[0115] It is understandable that by processing the two types of data separately through different modules, the characteristics of the image data and the data related to human visual perception can be completely preserved. The splicing and fusion method can effectively combine the two types of data information, providing complete and effective basic feature information for the subsequent generation of accurate target image data.

[0116] In some embodiments, the network model further includes a training module. The controller inputs preset reference image data stored in the memory into the training module. The training module combines this data, target image data, and HK effect data to calculate a loss function value. Then, based on the loss function value, the backpropagation algorithm is used to update the learnable parameters of the network model. To clearly understand this method, embodiments of this application provide a flowchart of another controller control method, as follows: Figure 5 As shown, the flowchart includes at least the following steps.

[0117] Step S301: Acquire image data.

[0118] Step S302: Input the image data and HK effect data into the network model.

[0119] Step S303: Extract the image feature vector and HK feature vector respectively through the network model and perform feature fusion to obtain the fused features; Step S304: Process the fusion features using a network model to obtain target image data; In this embodiment, the description of steps S301-S304 can be the same as the description of steps S101-S104 above, and will not be repeated in detail here.

[0120] Step S305: Input the preset reference image data into the training module.

[0121] In this embodiment, the training module is a functional computing unit in the network model architecture specifically used to complete the iterative optimization of model parameters and the calculation of error quantization. The training module is deployed in the floating-point operation circuit and parameter iteration operation circuit inside the controller, and has the computing capabilities of sample data storage, error calculation and weight update.

[0122] In this embodiment, the preset reference image data is high-quality benchmark image data that has been pre-screened and calibrated, meets image quality standards, and conforms to the real visual perception of the human eye. This reference image data has been calibrated by professional optical instruments and screened by subjective evaluation by the human eye. It has a standard color gamut, reasonable brightness distribution, and natural color performance, and can be used as a true reference sample in the model training process.

[0123] The reference image data contains standard pixel color components, ideal brightness and darkness distribution information, and standardized texture detail information, which are used to provide the best image quality reference benchmark during the training phase.

[0124] In this embodiment, the controller retrieves reference image data pre-stored in the local database and imports the data according to the data input format and pixel arrangement order specified by the training module, so that the reference image data resides in the data cache unit of the training module, providing a true reference for the subsequent calculation of the loss function.

[0125] Step S306: The training module processes the preset reference image data, target image data, and HK effect data to obtain the loss function value.

[0126] In this embodiment, the training module can be configured with multiple functional computing units to work together to integrate multiple types of sample data and perform overall error quantification and statistics. Different functional units can divide different data processing functions according to computing needs, respectively realize the determination of data differences and rule constraints in different dimensions, and then uniformly complete the summary and conversion of various deviation data, and finally output standardized error quantification values.

[0127] In actual operation, the training module first completes the data format unification and position matching between different image data. It can statistically determine the basic differences between images at the pixel level, color level, and image feature level. At the same time, combined with the established visual rule reference data, it assigns corresponding adjustment constraint weights to different regions and color intervals of the image to form a differentiated error reference standard that conforms to visual rules. Finally, it integrates the basic image differences and visual rule differences, completes the numerical normalization and conversion according to the general operation logic, and finally obtains the loss function value that can intuitively reflect the overall deviation between the model output result and the standard sample.

[0128] In some embodiments, the training module processes preset reference image data, target image data, and HK effect data to obtain a loss function value. Specifically, this is configured as follows: A color accuracy loss value is calculated based on the color component values ​​of the target image data and the color component values ​​at corresponding pixel positions in the preset reference image. The color accuracy loss value characterizes the difference between the color component values ​​of the target image data and the color component values ​​at corresponding pixel positions in the preset reference image. A perceptual loss value based on the HK effect is calculated based on the target image data, the preset reference image, and the HK effect data. The perceptual loss value based on the HK effect characterizes the difference between the target image data and the preset reference image data in terms of human visual perception features. A norm is calculated on the values ​​of the learnable parameters to obtain a regularization loss value, which constrains the range of values ​​for the learnable parameters. The color accuracy loss value, the perceptual loss value, and the regularization loss value are weighted and summed according to their respective preset weights to obtain the loss function value.

[0129] In the actual execution process, the training module first completes the one-to-one matching of pixel positions between the target image data and the preset reference image data. Relying on the pixel comparison operation logic, it compares the specific values ​​of various color components contained in the two types of images at the same coordinate position one by one. According to the numerical difference operation rules, it completes the quantization calculation and obtains the color accuracy loss value. This type of loss value can intuitively reflect the deviation between the model output image and the standard reference image in terms of basic color representation. It is the core evaluation criterion for ensuring the basic color reproduction of the image.

[0130] Based on this, the training module combines the stored and retrieved HK effect data, links the target image data with the preset reference image data to carry out in-depth visual feature comparison calculations, and completes feature difference quantification in combination with the established human visual perception rules, and then solves the perceptual loss value based on the HK effect. This loss value goes beyond the limitations of simple pixel value comparison, and focuses on measuring the differences between the two types of images in terms of subjective perception such as conformity to human viewing habits and visual comfort, which is in line with the application needs of actual display viewing.

[0131] Meanwhile, the training module can also perform numerical statistical operations on all learnable parameters involved in iterative updates within the network model. It completes numerical regularization operations through a general norm calculation method to generate corresponding regularized loss values. This loss is mainly used to reasonably limit the overall numerical range of various learnable parameters, avoiding the problem of abnormally large or small parameter values ​​during model training, and effectively avoiding common training problems such as model overfitting and underfitting.

[0132] After solving the three independent loss values, the training module retrieves the pre-set weight ratio parameters and, according to the established weighting operation rules, performs weighted integration calculation on the color accuracy loss value, the perception loss value based on the HK effect, and the regularization loss value, and finally merges them to obtain the overall loss function value.

[0133] It is understandable that this multi-class loss combination solution can take into account the three core training requirements of image basic color restoration accuracy, actual visual viewing effect of human eyes, and training stability of the network model itself. This allows the model to have a clear and comprehensive optimization direction during the iterative optimization process. It can not only ensure that the basic color of the output image matches the standard sample, but also optimize the overall visual perception of the image to match the characteristics of human eye perception, and continuously standardize the internal parameter operation state of the model. This comprehensively optimizes the overall training effect of the network model and improves the operational reliability and image quality optimization stability of the model in actual application.

[0134] In other optional embodiments, the training module can also add or remove loss calculation categories according to actual training needs, adjust the weight ratio of each type of loss, and change the specific calculation rules for each type of loss. The overall computing architecture is flexible and adjustable, and can adapt to different image quality optimization scenarios and different model training requirements.

[0135] For example, the loss of color adjustment accuracy can be calculated using the following formula:

[0136] Among them, L MES Indicates the loss value for color adjustment accuracy; H represents the total number of vertical pixels in the image; W represents the total number of horizontal pixels in the image; C represents the total number of color channels in the image; I i,j,k This refers to the pixel values ​​of the preset reference image data at the corresponding pixel coordinates and color channels; I Target (i,j,k) The value is the pixel value of the target image data at the same pixel coordinates and the same color channel. This formula accurately measures the overall difference between the preset reference image data and the target image data at the pixel color level by calculating the average of the squares of the differences in pixel values.

[0137] For example, the perceived loss based on the HK effect can be calculated using the following formula:

[0138] Among them, L HK F represents the perceived loss value based on the HK effect; HK (I) represents the visual perception features of a pre-defined reference image obtained by combining HK effect data; F HK (I Target The formula represents the visual perception features of the target image extracted by combining the HK effect data. The formula quantifies the difference between the preset reference image data and the target image data in terms of conforming to the visual perception characteristics of the human eye by calculating the cosine similarity between the two sets of visual perception features.

[0139] For example, the regularization loss can be calculated using the following formula:

[0140] Among them, L L2 Θ represents the regularization loss value; λ is the pre-set regularization weight coefficient; Θ represents the parameter set consisting of all learnable parameters of the network model; θ represents any learnable parameter within the parameter set; this formula performs a summation of squares on all learnable parameters, constrains the parameter value range, effectively prevents overfitting of the model, and improves the model's general adaptability.

[0141] For example, the overall loss function can be calculated using the following formula:

[0142] Among them, L total The formula represents the overall loss function value after integration; α is the preset weighting coefficient corresponding to the perceptual loss based on the HK effect; and β is the preset weighting coefficient corresponding to the regularization loss. This formula linearly combines the loss values ​​of the three different dimensions according to the set weights, comprehensively taking into account the image color accuracy, human visual perception effect, and model training stability, and determines the overall training optimization bias of the model.

[0143] Step S307: Update the learnable parameters of the network model using the backpropagation algorithm based on the loss function value.

[0144] In this embodiment, the parameter update process relies on the gradient calculation unit and parameter iteration control unit inside the training module to achieve overall operation. The related calculation process relies on the gradient calculation circuit and parameter adjustment logic circuit built into the controller to complete the execution, which is the core execution link to realize the autonomous iterative optimization of the network model.

[0145] In this embodiment, the backpropagation algorithm is a common parameter iterative optimization method in the training phase of network models. The algorithm uses the calculated loss function value as the core adjustment basis and can complete the error gradient derivation calculation layer by layer according to the reverse path of data forward transmission.

[0146] In actual execution, the training module first calculates the overall error gradient based on the loss function value. Following the reverse propagation order from the output to the input of the network model, it calculates the error gradient value corresponding to the learnable parameters within each layer of the network structure, accurately determining the degree and direction of influence of the current value of each learnable parameter on the overall error result. After obtaining the complete gradient data, it fine-tunes all learnable parameters within the network model according to the pre-set learning rate adjustment parameters and standard parameter update rules, thereby completing a complete round of parameter update operation.

[0147] In other embodiments, it can also be used in conjunction with auxiliary control methods such as momentum optimization strategy and adaptive learning rate adjustment strategy, which can effectively accelerate the parameter convergence speed, avoid problems such as oscillation and slow convergence during parameter update, and further improve the stability and efficiency of parameter adjustment.

[0148] It is understandable that by continuously performing backpropagation and learning parameter update operations based on the loss function value, the overall error between the model output and the standard reference data can be continuously reduced, the values ​​of various learnable parameters within the network model can be gradually optimized, and the processing accuracy of the network model for tasks such as image color calibration and visual perception adaptation can be continuously improved until the loss function value tends to stabilize and the various learnable parameters reach their optimal values, ultimately completing the overall training and optimization of the network model.

[0149] In some embodiments, the network model further includes an attention module and a color adjustment module. When obtaining target image data from fusion features, the attention module first processes the fusion features to generate multiple attention weights, and then the color adjustment module combines the attention weights, fusion features, and image data for computation to obtain the target image data. To clearly understand this scheme, embodiments of this application provide a flowchart of another controller control method, such as... Figure 6 As shown, the flowchart includes at least the following steps.

[0150] Step S401: Acquire image data.

[0151] Step S402: Input the image data and HK effect data into the network model; Step S403: Extract the image feature vector and HK feature vector respectively through the network model and perform feature fusion to obtain the fused features.

[0152] In this embodiment, the description of steps S401-S403 can be the same as the description of steps S101-S103 above, and will not be repeated in detail here.

[0153] Step S404: Process the fused features through the attention module to generate multiple attention weights.

[0154] In this embodiment, the attention module is a functional processing unit within the network model specifically used for feature weight allocation, visual information priority judgment, and feature importance screening. This module is mounted in the feature screening and operation circuit of the controller and can complete the deep analysis and weight quantization of the fused features by relying on parallel operation logic and feature weight allocation algorithm.

[0155] This attention module is used to determine the importance of information from different channels and different spatial locations within the fused features, and to generate differentiated weight coefficients based on the image's visual performance capabilities, thereby enhancing key visual features and suppressing ineffective and redundant features.

[0156] In some embodiments, the attention module includes a channel attention module and / or a spatial attention module. The two types of attention modules perform weight analysis from the color attribute dimension and the image space dimension, respectively. They can be used alone or in combination.

[0157] The fusion features are processed by an attention module to generate attention weights. Specifically, the configuration is as follows: the fusion features are processed by a channel attention module to obtain multiple channel attention weights. The channel attention weights are used to represent the importance weight of different color attributes to color enhancement. Each color attribute corresponds to one of a variety of abstract visual attributes, including brightness, saturation, and hue attributes; and / or, the fusion features are processed by a spatial attention module to obtain multiple spatial attention weights. The spatial attention weights are used to represent the importance weight of different spatial locations to color enhancement. Each spatial location corresponds to a local region in the image data.

[0158] Among them, the channel attention module is mainly used to explore the correlation between color attributes and assign weights to the fused features along the channel dimension; the spatial attention module is used to explore the visual differences between different local regions of the image and assign weights to the fused features along the spatial position dimension.

[0159] It is understood that this application is not limited to using only the two attention methods mentioned above. In other optional embodiments, the attention module may also be configured with any one or more of the following: multi-scale attention module, frequency domain attention module, and hierarchical attention module.

[0160] Among them, the multi-scale attention module is used to determine the importance of texture and receptive field features at different scales in the image, and adapt to the target in the picture of different sizes; the frequency domain attention module is used to filter high-frequency detail information and low-frequency smooth information in the image, and optimize the edge details and color transition effect of the picture; the hierarchical attention module is used to mine the correlation weights of features at different levels of the network, and strengthen the fusion effect of deep and shallow features. Various attention structures can be flexibly matched according to the actual image enhancement needs, and no specific limitation is made here.

[0161] In actual processing, the controller calls the channel attention module to perform statistical analysis, compression mapping, and weight calculation on the fused features, obtaining multiple channel attention weights. Each color attribute corresponds to different types of abstract visual attributes such as brightness, saturation, and hue. The channel attention weights are used to quantitatively characterize the contribution and optimization priority of various abstract visual attributes to the image color enhancement task, enabling the network to autonomously determine which visual attributes in the current image need to be optimized first.

[0162] Simultaneously, the controller can invoke the spatial attention module to perform spatial dimension parsing on the fused features and obtain multiple spatial attention weights.

[0163] Each spatial location corresponds one-to-one with a local image region in the original image data. Spatial attention weights are used to characterize the adjustment importance of different image regions in the color enhancement process, and can distinguish the optimization priority of flat regions, edge regions, and regions with color abrupt changes.

[0164] It is understandable that channel attention weights can be used to distinguish the optimization priorities of various visual attributes, and spatial attention weights can be used to distinguish the adjustment focus of different areas of the image. Different adjustments can be made to different attributes and positions of the image in a targeted manner, making color enhancement operations more in line with the actual display needs of the image.

[0165] In other embodiments, the attention module further includes a weight adjustment module, which uses HK effect data to perform constraint correction work on the acquired attention weights by human eye perception. The controller is also configured to: obtain the perceptual importance coefficients corresponding to different color combinations in the HK effect data through the weight adjustment module; multiply the perceptual importance coefficients with the channel attention weights at the same index position through the weight adjustment module, and / or multiply the perceptual importance coefficients with the spatial attention weights at the same index position to obtain multiple attention weights.

[0166] For example, the controller retrieves pre-stored HK effect data through the weight adjustment module, extracts the perceptual importance coefficients corresponding to different color combinations and color levels, and these coefficients are used to quantify the sensitivity of the human eye to different colors and brightness ranges. Further, the weight adjustment module multiplies the perceptual importance coefficients with the channel attention weights and spatial attention weights at the same position, completing adaptive weight correction to obtain the final optimized multiple attention weights.

[0167] It is understandable that the uncorrected attention weights are assigned based solely on the distribution of the image's own features. However, the attention weights corrected by introducing the HK effect data can combine the characteristics of human visual perception, amplify the weights of colors and areas that are sensitive to human vision, and suppress the weights of areas that are less sensitive to human vision. This makes the attention weight allocation logic more in line with the actual viewing patterns of human vision, providing a reliable basis for subsequent fine-tuning of color.

[0168] Step S405: Process multiple attention weights, fusion features, and image data through the color adjustment module to obtain target image data.

[0169] In this embodiment, the color adjustment module is a functional processing unit at the end of the network model that is dedicated to image color correction, pixel mapping and image quality optimization. This module relies on the internal convolution operation circuit, pixel mapping operation circuit and feature fusion operation circuit of the controller to complete the entire image processing process. It is the final execution module to realize intelligent color adjustment of images and conform to the HK visual perception effect.

[0170] In some embodiments, the color adjustment module includes a feature fusion submodule, a parameter generation submodule, and a color mapping submodule. The target image data is obtained through the color adjustment module. First, the feature fusion submodule fuses the attention weights with the fusion features to obtain attention-enhanced features. Next, the parameter generation submodule performs convolution processing on the attention-enhanced features to generate color adjustment parameters, which include at least one of hue adjustment, saturation adjustment, and brightness adjustment. Finally, the color mapping submodule adjusts the color component values ​​of corresponding pixels in the image data according to the color adjustment parameters to obtain the target image data.

[0171] The feature fusion submodule is used to complete feature weighting, feature concatenation, and multi-source information fusion; the parameter generation submodule is used to mine deep features through convolution operations and generate quantization control parameters that can be used for pixel adjustment; the color mapping submodule is used to complete pixel-by-pixel color component conversion and pixel value mapping, and finally output image data with optimized image quality.

[0172] In actual processing, the controller first calls the feature fusion submodule to complete the weight superposition operation, fusing multiple attention weights generated by the attention module with the fusion features. During the weighted fusion process, important visual attribute features are amplified based on the channel attention weights, while key image region features are strengthened by relying on spatial attention weights, and redundant and ineffective features are suppressed, thereby generating attention-enhanced features with stronger feature representation capabilities and clearer visual optimization direction.

[0173] Furthermore, the feature fusion submodule performs a secondary fusion of attention-enhancing features and the original image data, preserving the original image texture structure, edge information, and basic image content. Simultaneously, it overlays weighted, high-dimensional perceptual features to generate color adjustment features that balance the original image form with visual optimization features. These color adjustment features simultaneously contain objective image imaging information, human visual perception preference information, and attention weight differential adjustment information, possessing comprehensive feature representation capabilities.

[0174] Then, the controller calls the parameter generation submodule to perform multi-layer convolution processing on the color adjustment features. Through the convolution kernel, feature extraction, feature compression, and parameter mapping are completed to generate quantized color adjustment parameters. Among them, the color adjustment parameters include at least hue adjustment, saturation adjustment, and brightness adjustment. Each type of adjustment corresponds to the color shift range, color intensity, and brightness variation range at different pixel positions, providing a precise quantitative adjustment basis for pixel color correction.

[0175] Finally, the controller calls the color mapping submodule to perform pixel-level color correction. Based on the parameters, the color adjustment parameters output by the submodule are generated. The color components of each pixel in the original image data are converted, offset, and mapped pixel by pixel to complete the fine-grained color adjustment at the pixel level and finally generate the target image data with optimized image quality.

[0176] It is understandable that this color adjustment module relies on multi-level sub-module collaborative computation. First, it distinguishes between image attribute priority and region optimization priority through an attention mechanism. Then, it combines the original image features to generate exclusive adjustment parameters adapted to the current image. Finally, it completes pixel-by-pixel color mapping. This processing method differs from global unified adjustment methods. It can achieve differentiated intelligent adjustment for different colors, different regions, and different visual attributes, making image color transitions more natural and brightness and darkness more harmonious. At the same time, it conforms to the human visual perception law corresponding to the HK effect throughout the process, effectively reducing image color deviation, eliminating visual discomfort, and significantly improving the subjective visual experience of the displayed image.

[0177] Step S406: Generate backlight value and pixel driving value based on target image data; Step S407: Control the backlight module according to the backlight value and drive the display panel according to the pixel drive value to display the target image.

[0178] In this embodiment, the description of steps S406-S407 can be the same as the description of steps S105-S106 above, and will not be repeated in detail here.

[0179] In this scheme, the network model also includes an attention module and a color adjustment module. When obtaining target image data from fused features, the attention module first processes the fused features to generate multiple attention weights. Then, the color adjustment module combines the attention weights, fused features, and image data for computation to obtain the target image data. It can be understood that by using attention weights to distinguish the importance of different features, and then working with the color adjustment module to complete data integration computation, it is possible to selectively adjust effective image information, accurately complete the color and brightness adjustment of the image, and make the final target image data more in line with visual display requirements.

[0180] For example, to better understand the control method provided in the embodiments of this application, a flowchart is also provided in this application embodiment. In this flowchart, the input image is set to an RGB three-channel image; the image preprocessing submodule's size scaling is adjusted to 224×224 pixels; the feature mapping submodule for feature extraction uses a pre-trained residual neural network ResNet; and the encoding module in the second preprocessing module is set to use one-hot encoding to convert the HK effect data into binary feature vectors, such as... Figure 7As shown, the flowchart also includes a network model 500. First, steps S501 and S503 are performed: basic data acquisition is completed, i.e., acquiring the three-channel RGB image data and HK effect data to be processed. Next, step S502 is performed, where the first preprocessing module performs feature extraction processing on the image data. Specifically, the image preprocessing submodule scales the image data to a preset size of 224×224 and performs normalization to obtain preprocessed image data. Then, the pre-trained ResNet network in the feature mapping submodule performs feature mapping processing on the preprocessed image data to obtain image feature vectors.

[0181] Then, in step S504, the obtained HK effect data is encoded and converted into HK effect feature vectors that can participate in the calculation. Next, in step S505, the previously obtained image feature vectors and HK effect feature vectors are fused using a concatenation method through the feature extraction module to obtain the fused features.

[0182] Next, in step S506, the attention mechanism module generates channel attention weights and spatial attention weights based on the fusion features. The channel attention weights distinguish the importance of different color attributes, and the spatial attention weights distinguish the importance of different spatial locations in the image. Then, in step S507, the weight adjustment module extracts the perceptual importance coefficients corresponding to different color combinations within the HK effect data; these perceptual importance coefficients are multiplied by the channel attention weights at the same position, and / or multiplied by the spatial attention weights at the same position, resulting in the multiple attention weights.

[0183] Then, in step S508, the feature fusion submodule performs weighted fusion of the multiple attention weights and the fusion features to obtain attention-enhanced features, which are then directly used as input data for the convolution operation. Next, the parameter generation submodule uses the learned convolution kernel to perform convolution operations on the input data, generating color adjustment parameters while extracting deeper features. These color adjustment parameters include at least one of hue adjustment, saturation adjustment, and brightness adjustment. Finally, the color mapping submodule adjusts the color components corresponding to each pixel in the image data one by one according to the generated color adjustment parameters.

[0184] Finally, step S509 completes all color adjustment processing, obtaining the target image data. It can be seen that the target image data maintains the same size and format as the initial input image data, with optimized color vibrancy, conforming to the characteristics of human visual perception, and effectively improving the image presentation.

[0185] In actual implementation, the controller will also convert the target image data into backlight values ​​and pixel drive values ​​to control the display panel and backlight module to display the target image.

[0186] It can be seen that the neural network in the network model 500 is pre-trained. Therefore, in some embodiments, the network model may also include a training module. For a detailed description, please refer to steps S305-S306, which will not be elaborated here.

[0187] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0188] The display devices disclosed in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A display device, characterized in that, The display device includes: Display panel, used to display images; A backlight module is used to provide backlight for the display panel; The controller is connected to both the display panel and the backlight module. A memory is located inside the controller. The memory stores a network model and human visual perception HK effect data. The HK effect data is used to characterize human visual perception of different color categories and different color intensities under the same brightness. The network model also includes an attention module and a color adjustment module. The attention module also includes a weight adjustment module. The controller is configured as follows: Acquire image data, The image data and the HK effect data are input into the network model; The network model extracts image feature vectors and HK feature vectors respectively and performs feature fusion to obtain fused features. The image feature vectors are extracted by the network model from the image data, and the HK feature vectors are extracted by the network model from the HK effect data. The attention module processes the fused features to generate multiple channel attention weights and / or multiple spatial attention weights. The channel attention weights characterize the importance of different color attributes to color enhancement, with each color attribute corresponding to one of a plurality of abstract visual attributes, including brightness, saturation, and hue. The spatial attention weights characterize the importance of different spatial locations to color enhancement, with each spatial location corresponding to a local region in the image data. The weight adjustment module is used to obtain the perceived importance coefficients of different color combinations in the HK effect data. The weight adjustment module multiplies the perceived importance coefficient with the channel attention weight at the same position, and / or multiplies the perceived importance coefficient with the spatial attention weight at the same position to obtain the multiple attention weights. The color adjustment module processes the multiple attention weights, the fusion features, and the image data to obtain target image data. Generate backlight values ​​and pixel driving values ​​based on the target image data; The backlight module is controlled according to the backlight value, and the display panel is driven according to the pixel drive value to display the target image.

2. The display device according to claim 1, characterized in that, The network model includes a first preprocessing module and a second preprocessing module. The step of extracting image feature vectors and HK feature vectors through the network model and then fusing them is specifically configured as follows: The image data is subjected to feature extraction processing by the first preprocessing module to obtain the image feature vector; and... The HK effect data is encoded using the second preprocessing module to obtain the HK feature vector. The image feature vector and the HK feature vector are concatenated to obtain the fused feature.

3. The display device according to claim 2, characterized in that, The first preprocessing module includes an image preprocessing submodule and a feature mapping submodule. The step of performing feature extraction processing on the image data through the first preprocessing module to obtain the image feature vector is specifically configured as follows: The image preprocessing submodule performs size scaling and normalization on the image data to obtain the processed image data. The feature mapping submodule performs feature mapping on the processed image data to obtain the image feature vector.

4. The display device according to claim 2, characterized in that, The second preprocessing module includes an encoding module, wherein the HK effect data is encoded by the second preprocessing module to obtain the HK feature vector, specifically configured as follows: The HK effect data is converted into HK feature vectors by one-hot encoding using the encoding module.

5. The display device according to claim 1, characterized in that, The network model further includes a training module, the memory stores preset reference image data, and the controller is further configured to: The preset reference image data is input into the training module; The training module processes the preset reference image data, the target image data, and the HK effect data to obtain the loss function value. Based on the loss function value, the learnable parameters of the network model are updated using the backpropagation algorithm.

6. The display device according to claim 5, characterized in that, The step of processing the preset reference image data, the target image data, and the HK effect data through the training module to obtain the loss function value is specifically configured as follows: A color accuracy loss value is calculated based on the color component values ​​of the target image data and the color component values ​​at the corresponding pixel positions in the preset reference image. The color accuracy loss value is used to characterize the difference between the color component values ​​of the target image data and the color component values ​​at the corresponding pixel positions in the preset reference image. Based on the target image data, the preset reference image, and the HK effect data, a perceptual loss value based on the HK effect is calculated. The perceptual loss value based on the HK effect is used to characterize the difference between the target image data and the preset reference image data in terms of human visual perception features. The norm of the learnable parameter is calculated to obtain the regularization loss value, which is used to constrain the range of values ​​of the learnable parameter. The color accuracy loss value, the perception loss value, and the regularization loss value are weighted and summed according to their respective preset weights to obtain the loss function value.

7. The display device according to claim 1, characterized in that, The attention module includes a channel attention module and / or a spatial attention module. The fused features are processed by the attention module to generate multiple channel attention weights and / or multiple spatial attention weights, specifically configured as follows: The fused features are processed by the channel attention module to obtain the multiple channel attention weights; And / or, the fused features are processed by the spatial attention module to obtain the multiple spatial attention weights.

8. The display device according to claim 1, characterized in that, The color adjustment module includes a feature fusion submodule, a parameter generation submodule, and a color mapping submodule. The step of processing the multiple attention weights, the fusion features, and the image data through the color adjustment module to obtain target image data is specifically configured as follows: The feature fusion submodule fuses the multiple attention weights with the fused features to obtain attention-enhanced features. The attention enhancement features are convolved by the parameter generation submodule to generate color adjustment parameters, which include at least one of hue adjustment, saturation adjustment and brightness adjustment. The color mapping submodule adjusts the color component values ​​of corresponding pixels in the image data according to the color adjustment parameters to obtain the target image data.

Images