A method, system, terminal, and medium for image quality enhancement using a backlight-fused convolutional neural network.

Abstract

This invention provides a backlight fusion convolutional neural network image quality enhancement method, system, terminal, and medium, relating to the field of display technology. The method includes: extracting backlight data corresponding to the number of logical partitions of the image to be displayed; inputting this data into a pre-trained target backlight fusion model to obtain backlight data with a backlight partition count less than the actual number of logical partitions output by the model after fusing the backlight data with the number of logical partitions; transmitting the backlight data with the actual number of partitions and the image to be displayed to a preset display device for display; the display effect of the backlight data with the actual number of partitions approximates the display effect of the backlight data with the number of logical partitions. This invention fuses high-partition-count backlight data into low-partition-count backlight data through a model, achieving an image quality enhancement effect close to that of a high-partition-count backlight module on a backlight module with a low number of backlight partitions, reducing the number of backlight partitions without sacrificing display performance and lowering manufacturing costs.

Description

Technical Field

[0001] This invention relates to the field of display technology, and in particular to a method, system, terminal, and medium for enhancing image quality using a backlight-fused convolutional neural network. Background Technology

[0002] In the current high-end large-screen TV market, Mini LED technology is widely used in pursuit of high contrast and high-quality display effects. However, in the current field of local dimming, a greater number of backlight zones is often required for better image enhancement, leading to a gradual increase in manufacturing costs. Moreover, as the number of different backlight zone schemes increases, the Local Dimming algorithm and the LED backlight module require a longer period of accurate tuning to achieve optimal results.

[0003] In summary, related technologies often achieve better image quality enhancement by increasing the number of backlight zones, resulting in high manufacturing costs and complex, time-consuming debugging. Therefore, providing a solution to these technical problems is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a backlight fusion convolutional neural network image quality enhancement method, system, terminal and medium to address the above-mentioned defects of the prior art. The aim is to solve the problems of high manufacturing cost and complex and time-consuming debugging when the prior art obtains better image quality enhancement effect by increasing the number of backlight partitions.

[0005] The technical solution adopted by this invention to solve the technical problem is as follows:

[0006] In a first aspect, a backlight fusion convolutional neural network image quality enhancement method is provided, wherein the method includes:

[0007] Obtain the image to be displayed, and extract the backlight data of the number of logical partitions corresponding to the image to be displayed;

[0008] The backlight data of the logical partition number is input into the pre-trained target backlight fusion model to obtain the backlight data of the actual partition number output by the target backlight fusion model after fusing the backlight data of the logical partition number.

[0009] The backlight data of the actual number of partitions and the image to be displayed are transmitted to a preset display device for display.

[0010] Wherein, the actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions satisfies a preset similarity condition.

[0011] Optionally, the training process of the pre-trained target backlight fusion model includes:

[0012] A set of image sample data is selected from the pre-constructed training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number;

[0013] The backlight samples of the logical number of current image samples are input into the backlight fusion model built based on the convolutional neural network to obtain the backlight samples of the actual number of partitions output by the backlight fusion model.

[0014] Based on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the logical number of partitions and the second simulation result corresponding to the backlight samples of the actual number of partitions.

[0015] The parameters of the backlight fusion model are fine-tuned based on the first simulation results and the second simulation results to obtain the current fine-tuned backlight fusion model;

[0016] Determine whether all image sample data in the training set have been selected as training samples;

[0017] When all image sample data in the training set are selected as training samples, it is determined that one training of the backlight fusion model has been completed, the current number of training iterations of the backlight fusion model is determined, and it is determined whether the current number of training iterations has reached the preset number of training iterations.

[0018] When the current number of training iterations reaches the preset number of training iterations, the current fine-tuned backlight fusion model is determined as the trained target backlight fusion model.

[0019] Optionally, after determining whether all image sample data in the training set have been selected as training samples, the method further includes:

[0020] If none of the image sample data in the training set is selected as training samples, it is determined that the training of the backlight fusion model has not been completed, and the step of selecting a set of image sample data from the pre-built training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number and subsequent steps are continued until all the image sample data in the training set is selected as training samples.

[0021] Furthermore, after determining whether the current training iterations have reached the preset training iterations, the process also includes:

[0022] If the current training iterations have not reached the preset number of training iterations, the steps of selecting a set of image sample data from the pre-constructed training set as training samples to obtain the current image sample and its corresponding logical partition number of backlight samples, and subsequent steps, are repeated until the current training iterations reach the preset number of training iterations.

[0023] Optionally, the backlight samples based on the logical number of partitions and the backlight samples based on the actual number of partitions are used to simulate the display effect of the current image sample to obtain a first simulation result corresponding to the backlight samples of the logical number of partitions and a second simulation result corresponding to the backlight samples of the actual number of partitions, including:

[0024] The backlight samples of the aforementioned logical partition number are subjected to diffusion processing to obtain the first diffusion result;

[0025] Pixel compensation is performed on the image sample based on the first diffusion result to obtain a first compensated image sample;

[0026] Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the number of logical partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the number of logical partitions.

[0027] Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the second simulation result corresponding to the backlight samples of the actual number of partitions.

[0028] Alternatively, the backlight samples of the logical number of partitions are diffused to obtain a first diffusion result, and the backlight samples of the actual number of partitions are diffused to obtain a second diffusion result.

[0029] Pixel compensation is performed on the image sample based on the first diffusion result to obtain a first compensated image sample, and pixel compensation is performed on the image sample based on the second diffusion result to obtain a second compensated image sample.

[0030] Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the number of logical partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the number of logical partitions.

[0031] Based on the pixel values ​​corresponding to each pixel point on the second compensated image sample, the liquid crystal module parameters, and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the second simulation result corresponding to the backlight samples of the actual number of partitions.

[0032] Optionally, the diffusion processing of the backlight samples of the logical partition number to obtain the first diffusion result includes:

[0033] The backlight samples of the logical partition number are diffused using anisotropic Gaussian blur diffusion method to obtain the first diffusion result;

[0034] Furthermore, the diffusion processing of the backlight samples of the actual number of partitions to obtain the second diffusion result includes:

[0035] The backlight samples of the actual number of partitions are diffused using an isotropic Gaussian blur diffusion method to obtain a second diffusion result.

[0036] Optionally, the convolutional neural network includes one average pooling layer, four convolutional layers, and activation function layers corresponding to each convolutional layer. The first convolutional layer and its corresponding activation function layer serve as the input layer, the second convolutional layer and its corresponding activation function layer, the third convolutional layer and its corresponding activation function layer, and the average pooling layer serve as hidden layers, and the fourth convolutional layer and its corresponding activation function layer serve as the output layer.

[0037] Optionally, the calculation formula for the convolutional layer is:

[0038] ;

[0039] ;

[0040] in, For output channel index, Indicates the convolutional layer's... The convolution result of each output channel at coordinate (i, j), where i is the row index of the input or output data of the convolutional layer, and j is the column index of the input or output data of the convolutional layer. Indicates the first The bias value of each output channel. For input channel index, Here, is the kernel size, u is the row index of the kernel, and v is the column index of the kernel. Let (i+u, j+v) be the input data for the c-th input channel of the convolutional layer at coordinates (i+u, j+v). This represents the weight data of the c-th channel of the convolution kernel at the (u, v) coordinate;

[0041] Furthermore, the formula for calculating the average pooling layer is:

[0042] ;

[0043] in, This represents the pooling result of the average pooling layer. Use the row index for the output data of the average pooling layer. Here, H represents the column index of the average pooling layer output data, H is the height of the pooling window, and W is the width of the pooling window. The input data for the average pooling layer is located at coordinates (x+h, y+w), where x is the row index of the input data for the average pooling layer and y is the column index of the input data for the average pooling layer.

[0044] Secondly, the present invention also discloses a backlight fusion convolutional neural network image quality enhancement system, wherein the system comprises:

[0045] The acquisition module is used to acquire the image to be displayed and extract the backlight data of the number of logical partitions corresponding to the image to be displayed;

[0046] The fusion module is used to input the backlight data of the logical number of partitions into the pre-trained target backlight fusion model to obtain the backlight data of the actual number of partitions output by the target backlight fusion model after fusing the backlight data of the logical number of partitions.

[0047] The display module is used to transmit the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display.

[0048] Wherein, the actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions satisfies a preset similarity condition.

[0049] Thirdly, the present invention discloses a terminal, comprising: a memory, a processor, and a backlight fusion convolutional neural network image enhancement program stored in the memory and executable on the processor, wherein the backlight fusion convolutional neural network image enhancement program, when executed by the processor, implements the steps of the backlight fusion convolutional neural network image enhancement method as described above.

[0050] Fourthly, the present invention discloses a computer-readable storage medium storing a computer program that can be executed to implement the steps of the backlight fusion convolutional neural network image enhancement method as described above.

[0051] This invention provides a method, system, terminal, and medium for enhancing image quality using a backlight fusion convolutional neural network. The method includes: acquiring an image to be displayed and extracting backlight data corresponding to the number of logical partitions of the image; inputting the backlight data of the number of logical partitions into a pre-trained target backlight fusion model to obtain backlight data of the actual number of partitions output by the target backlight fusion model after fusing the backlight data of the number of logical partitions; transmitting the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display; wherein the number of actual partitions is less than the number of logical partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the number of logical partitions satisfies a preset similarity condition. Therefore, it can be seen that the present invention can fuse the backlight data of the logical number of partitions into the backlight data of the actual number of partitions through the pre-trained target backlight fusion model, and the actual number of partitions is less than the logical number of partitions. Thus, it can achieve a picture quality enhancement effect close to that of a backlight module with a high number of backlight partitions on a backlight module with a low number of backlight partitions. In other words, it can reduce the number of backlight partitions without sacrificing display performance, thereby reducing manufacturing costs to a certain extent. Attached Figure Description

[0052] Figure 1 This is a flowchart of a preferred embodiment of the backlight fusion convolutional neural network image quality enhancement method in this invention;

[0053] Figure 2 This is a schematic diagram of a specific backlight fusion model training framework disclosed in this invention;

[0054] Figure 3 This is a schematic diagram of a specific display system structure disclosed in this invention;

[0055] Figure 4 This is a specific backlight fusion model training flowchart disclosed in this invention;

[0056] Figure 5 This is a schematic diagram of the backlighting effect of a partitioned LED disclosed in this invention;

[0057] Figure 6 This is a schematic diagram of a specific convolutional neural network structure disclosed in this invention;

[0058] Figure 7 This is another specific backlight fusion model training flowchart disclosed in this invention;

[0059] Figure 8 This is a schematic diagram illustrating the correspondence between high-zone-number backlight data and low-zone-number backlight data disclosed in this invention.

[0060] Figure 9 This is a schematic diagram of the backlighting effect of another type of partitioned backlight LED disclosed in this invention;

[0061] Figure 10 This is a functional principle block diagram of a preferred embodiment of the backlight fusion convolutional neural network image quality enhancement system of the present invention;

[0062] Figure 11 This is a functional principle block diagram of a preferred embodiment of the terminal in this invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0064] In the current high-end large-screen TV market, Mini LED technology is widely used to pursue high contrast and high-quality display effects. However, in the current field of local dimming, a greater number of backlight zones is often required for better image enhancement, leading to a gradual increase in manufacturing costs. Moreover, as the number of different backlight zone schemes increases, the Local Dimming algorithm and the LED backlight module require a longer period of accurate tuning to achieve the best results. In other words, related technologies often achieve better image enhancement by increasing the number of backlight zones, resulting in high manufacturing costs and complex, time-consuming tuning.

[0065] However, with the development of artificial intelligence technology, especially Convolutional Neural Networks (CNNs), their applications in various industries, particularly image processing, have demonstrated outstanding performance. Therefore, this application provides a backlight fusion convolutional neural network image enhancement scheme that can reduce the number of backlight zones without sacrificing display performance, thereby reducing manufacturing costs to some extent.

[0066] Please see Figure 1 , Figure 1 This is a flowchart of the image quality enhancement method using a backlight fusion convolutional neural network in this invention. For example... Figure 1 As shown, the image quality enhancement method using a backlight fusion convolutional neural network according to an embodiment of the present invention includes:

[0067] Step S11: Obtain the image to be displayed and extract the backlight data of the logical partition number corresponding to the image to be displayed.

[0068] It is understandable that, for the image to be displayed, the backlight data of the corresponding logical partition number is extracted, that is, the corresponding high partition number backlight data. Specifically, an artificial intelligence-based backlight extraction algorithm can be used to extract the backlight data of the logical partition number (high partition number backlight data) of the image to be displayed.

[0069] Step S12: Input the backlight data of the logical partition number into the pre-trained target backlight fusion model to obtain the backlight data of the actual partition number output by the target backlight fusion model after fusing the backlight data of the logical partition number; wherein, the actual partition number is less than the logical partition number, and the similarity between the display effect of the backlight data of the actual partition number output by the target backlight fusion model and the display effect of the backlight data of the logical partition number satisfies the preset similarity condition.

[0070] In this embodiment, after extracting the backlight data of the logical number of partitions corresponding to the image to be displayed, the backlight data of the logical number of partitions can be input into the pre-trained target backlight fusion model, so as to use the target backlight fusion model to fuse the backlight data of the logical number of partitions into the backlight data of the actual number of partitions (low number of partitions backlight data). Understandably, a pre-trained target backlight fusion model can fuse backlight data of the logical number of partitions corresponding to the image to be displayed into backlight data of the actual number of partitions. The actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions meets the preset similarity conditions, such as the similarity between the display effects of the two is not less than 95%. That is, the pre-trained target backlight fusion model can fuse backlight data of a high number of partitions into backlight data of a relatively low number of partitions. It can achieve a picture quality enhancement effect close to that of a backlight module with a high number of partitions on a backlight module with a low number of backlight partitions. This allows the high-performance high-partition backlight extraction algorithm to quickly adapt to backlight modules with different low numbers of backlight partitions, accelerating product development. In other words, it can effectively fuse backlight data, making it compatible between different logical backlight extraction algorithms (obtaining backlight data of a high number of partitions) and different actual backlight schemes (using backlight data of a low number of partitions).

[0071] During the training process, the target backlight fusion model compares the simulation results of the display effect of backlight data with a logical number of partitions with the simulation results of the display effect of backlight data with an actual number of partitions. Then, it adjusts the model parameters until, after a preset number of training sessions, the simulation results of the display effect of backlight data with an actual number of partitions approach the simulation results of the display effect of backlight data with a logical number of partitions. At this point, the training stops. In other words, it compares and learns the simulation results of the display effect of backlight data with a high number of partitions with the simulation results of backlight data with a low number of partitions, so that the display effect of backlight data with a low number of partitions approaches the display effect of backlight data with a high number of partitions.

[0072] For example, see Figure 2 As shown, a high-partition backlight extraction algorithm is used to extract high-partition backlight data corresponding to RGB image samples. This high-partition backlight data is then input into a backlight fusion model to fuse it into low-partition backlight data. A high-partition simulator is then used to simulate the display effect of the high-partition backlight data, resulting in a high-partition backlight simulation image. Similarly, a low-partition simulator is used to simulate the display effect of the low-partition backlight data, resulting in a low-partition backlight simulation image. The differences between the high-partition and low-partition backlight simulation images are compared to adjust the parameters of the backlight fusion model, such as weights and biases. After adjusting the model parameters, the above process is repeated until a preset number of training iterations is achieved, and the display effect of the low-partition backlight simulation image approaches that of the high-partition backlight simulation image. The target parameters of the backlight fusion model are determined and used as the final parameters of the backlight fusion model to obtain the target backlight fusion model. That is, during the pre-training process of the backlight fusion model based on the convolutional neural network, the difference between the display simulation results of low-number-partition backlight data and high-number-partition backlight data is compared. The difference between the display simulation results before and after fusion of high-number-partition backlight data is learned, so that the display simulation results of low-number-partition backlight data gradually approach the display simulation results of high-number-partition backlight data. Finally, the display effect of low-number-partition backlight data approaches the display effect of high-number-partition backlight data. Thus, the image quality can be enhanced and the display effect can be guaranteed without increasing the number of backlight partitions. In other words, the actual number of backlight module partitions can be reduced without sacrificing display performance, thereby reducing manufacturing costs.

[0073] Step S13: Transmit the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display.

[0074] In this embodiment, after the target backlight fusion model fuses the backlight data of the logical number of partitions into the backlight data of the actual number of partitions, it can directly transmit the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display. Furthermore, the preset display device can be a Mini LED display, such as transmitting the backlight data of the actual number of partitions and the image to be displayed to a Mini LED display for display.

[0075] In this embodiment, before transmitting the backlight data of the actual number of partitions and the image to be displayed to the preset display device for display, it may further include: correcting the image to be displayed to obtain the corrected image to be displayed, and then transmitting the backlight data of the actual number of partitions and the corrected image to be displayed to the preset display device for display.

[0076] It should be noted that the aforementioned backlight fusion convolutional neural network image enhancement method can be implemented using platforms such as CPU (Central Processing Unit), GPU (Graphics Processing Unit, i.e., graphics card), NPU (Neural Processing Unit), or FPGA (Field-Programmable Gate Array). For example, see [link to relevant documentation]. Figure 3As shown, a pre-trained target backlight fusion model is deployed on a GPU as a sub-module, and together with a CPU and a Mini LED display, a complete display system is constructed. This system can acquire input image data via HDMI (High-Definition Multimedia Interface) or DP (DisplayPort). The CPU extracts the high-segment backlight data corresponding to the input image data, performs certain corrections on the input image data in the central processing unit, and obtains output image data. The high-segment backlight data is then transmitted to the GPU with the deployed target backlight fusion model for processing to obtain low-segment backlight data. The low-segment backlight data and the output image data are then transmitted to a Mini LED display containing image display driver circuitry and LED backlight driver circuitry for display. Alternatively, a pre-trained target backlight fusion model can be deployed to an FPGA system. The FPGA system receives high-segment backlight data from a SoC (System on Chip), fuses the high-segment backlight data to obtain low-segment backlight data, and then outputs the low-segment backlight data, which, together with the image data output by the SoC, controls the display of a TV. The deployment was completed using a resource-constrained FPGA chip. Therefore, only two convolutional kernels were implemented for each convolutional layer. Convolutions of different layers could be calculated using time-division multiplexing. Since the backlight data was relatively small, the calculation and output could be completed within the time of one frame of image data.

[0077] For the specific training process of the pre-trained target backlight fusion model mentioned above, please refer to [link / reference]. Figure 4 As shown, the specific steps may include the following:

[0078] Step S121: Select a set of image sample data from the pre-constructed training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number.

[0079] In this embodiment, the pre-constructed training set contains multiple sets of image sample data, and each set of image sample data contains image samples and backlight samples of the corresponding logical partition number. Then, during each training, a set of image sample data is randomly selected from the training set as the current training sample data.

[0080] Step S122: Input the backlight samples of the logical number of current image samples into the backlight fusion model constructed based on the convolutional neural network to obtain the backlight samples of the actual number of partitions output by the backlight fusion model.

[0081] In this embodiment, a backlight fusion model based on a convolutional neural network is used to fuse the backlight samples with the logical number of partitions corresponding to the current image sample (high partition number backlight samples) into backlight samples with the actual number of partitions (low partition number backlight samples), and the actual number of partitions is less than the logical number of partitions.

[0082] Step S123: Simulate the display effect of the current image sample based on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions, respectively, to obtain the first simulation result corresponding to the backlight samples of the logical number of partitions and the second simulation result corresponding to the backlight samples of the actual number of partitions.

[0083] In this embodiment, after the backlight fusion model outputs the backlight samples of the actual number of partitions corresponding to the backlight samples of the logical number of partitions, the display effect simulation is performed on the current image sample based on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions. That is, the display effect simulation is performed on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions using the current image sample, to obtain the first simulation result corresponding to the backlight samples of the logical number of partitions and the second simulation result corresponding to the backlight samples of the actual number of partitions.

[0084] In one specific embodiment, the display effect simulation of the current image sample is performed based on the backlight samples with the logical number of partitions and the backlight samples with the actual number of partitions, respectively. Specifically, it may include: performing diffusion processing on the backlight samples with the logical number of partitions to obtain a first diffusion result; performing pixel compensation on the image sample based on the first diffusion result to obtain a first compensated image sample; performing display effect simulation on the current image sample based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples with the logical number of partitions to obtain a first simulation result corresponding to the backlight samples with the logical number of partitions; and performing display effect simulation on the current image sample based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples with the actual number of partitions to obtain a second simulation result corresponding to the backlight samples with the actual number of partitions.

[0085] In another specific embodiment, the display effect simulation of the current image sample is performed based on the backlight samples with the logical number of partitions and the backlight samples with the actual number of partitions, respectively. Specifically, this may include: performing diffusion processing on the backlight samples with the logical number of partitions to obtain a first diffusion result, and performing diffusion processing on the backlight samples with the actual number of partitions to obtain a second diffusion result; performing pixel compensation on the image sample based on the first diffusion result to obtain a first compensated image sample, and performing pixel compensation on the image sample based on the second diffusion result to obtain a second compensated image sample; performing display effect simulation on the current image sample based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples with the logical number of partitions to obtain a first simulation result corresponding to the backlight samples with the logical number of partitions; and performing display effect simulation on the current image sample based on the pixel values ​​corresponding to each pixel point on the second compensated image sample, the liquid crystal module parameters, and the backlight samples with the actual number of partitions to obtain a second simulation result corresponding to the backlight samples with the actual number of partitions.

[0086] Understandably, during model training and display effect simulation, when input image samples are unavailable, pixel compensation can be performed on the image samples based solely on the diffusion results of the high-division backlight data. This pixel compensation is typically performed by the upstream device extracting the high-division backlight samples. The high-division and low-division backlight samples are then used solely for display effect simulation based on this compensation result. When input image samples are available, pixel compensation is performed according to the first diffusion result of the high-division backlight samples and the second diffusion result of the low-division backlight samples, respectively, yielding corresponding compensation results. The high-division and low-division backlight samples can then be used separately for display effect simulation based on their corresponding compensation results. Here, the backlight data is used to illuminate the backlight module, and the compensated image data is sent to the liquid crystal module. Therefore, the display effect simulation is completed by combining the backlight data, image data, compensated image data, and liquid crystal module parameters.

[0087] Furthermore, the backlight samples of the logical partition number are diffused to obtain a first diffusion result, which may specifically include: using anisotropic Gaussian blur diffusion method to diffuse the backlight samples of the logical partition number to obtain a first diffusion result; the backlight samples of the actual partition number are diffused to obtain a second diffusion result, which may specifically include: using isotropic Gaussian blur diffusion method to diffuse the backlight samples of the actual partition number to obtain a second diffusion result.

[0088] Understandably, since the LEDs will affect each other, it is necessary to diffuse the backlight samples to simulate the real situation. Different diffusion coefficients need to be used according to the real situation. For example, isotropic Gaussian blur diffusion is used for backlight data with low number of zones, while anisotropic Gaussian blur diffusion is used for backlight data with high number of zones, in order to simulate different LED densities in the horizontal and vertical directions.

[0089] It should be noted that different backlight solutions will have different numbers of zones and different backlight module parameters, such as diffusion coefficient and OD (Optical Distance) value, which will lead to different display effects. See [link to relevant documentation]. Figure 5 As shown, simulation processing can be performed according to the actual situation. In this embodiment, the diffusion coefficient of a single backlight zone in the horizontal and vertical directions is calculated by using the ratio between high-zone backlight data and low-zone backlight data and backlight module parameters. The diffusion of backlight data is simulated by using a Gaussian blur kernel. Then, the display effect is simulated by combining liquid crystal module parameters, such as transmittance, and the visual effect of the human eye.

[0090] It should also be noted that since most backlight modules cannot drive all LEDs at full power, the power of the backlight sample can be limited before simulating the display effect to ensure that the module does not operate at overpower. The backlight data is converted into the current driving the backlight zones (LEDs). The power supplied by the power supply is limited, so the total power of all backlight zones must not exceed the power supplied by the power supply.

[0091] Step S124: Fine-tune the parameters of the backlight fusion model based on the first simulation result and the second simulation result to obtain the current fine-tuned backlight fusion model.

[0092] Understandably, by comparing the differences between the first and second simulation results, the parameters of the backlight fusion model are adjusted so that the second simulation results of the backlight samples with a low number of partitions gradually approach the first simulation results of the backlight samples with a high number of partitions.

[0093] Step S125: Determine whether all image sample data in the training set have been selected as training samples.

[0094] It is understandable that, since random selection may result in the same training sample being selected multiple times, a training sample can be selected from the training set once. After adjusting the backlight fusion model using this training sample, it is then confirmed again whether each set of image sample data in the training set has been used for training, ensuring that the training samples in the training set are selected for training the backlight fusion model in the current round of training.

[0095] Step S126: When all image sample data in the training set are selected as training samples, it is determined that one training of the backlight fusion model has been completed, the current number of training iterations of the backlight fusion model is determined, and it is determined whether the current number of training iterations has reached the preset number of training iterations.

[0096] Understandably, in each training round, the backlight fusion model is adjusted using all image sample data in the training set. Only when all image sample data in the training set has been selected as training samples is the training of the backlight fusion model completed. At this time, the training count is incremented by one to determine the current training count and whether the current training count has reached the preset training count.

[0097] Step S127: When the current training count reaches the preset training count, the current fine-tuned backlight fusion model is determined as the trained target backlight fusion model.

[0098] Understandably, after training for a preset number of times, when the simulation results of the display effect of the backlight samples with the actual number of partitions approach the simulation results of the display effect of the backlight samples with the logical number of partitions, the model training is stopped, and the target backlight fusion model is obtained.

[0099] In this embodiment, after determining whether all image sample data in the training set has been selected as training samples, the method may further include: if not all image sample data in the training set has been selected as training samples, then it is determined that one training of the backlight fusion model has not been completed, and the step of selecting a set of image sample data from the pre-constructed training set as training samples to obtain the current image sample and its corresponding logical partition number of backlight samples, and subsequent steps, are executed until all image sample data in the training set has been selected as training samples; and after determining whether the current training count has reached the preset training count, the method may further include: if the current training count has not reached the preset training count, then the step of selecting a set of image sample data from the pre-constructed training set as training samples to obtain the current image sample and its corresponding logical partition number of backlight samples, and subsequent steps, are executed again, until the current training count reaches the preset training count.

[0100] It should be noted that, see Figure 6 As shown, the convolutional neural network includes one average pooling layer, four convolutional layers, and activation function layers corresponding to each convolutional layer. The first convolutional layer and its corresponding activation function layer serve as the input layer, the second convolutional layer and its corresponding activation function layer, the third convolutional layer and its corresponding activation function layer, and the average pooling layer serve as hidden layers, and the fourth convolutional layer and its corresponding activation function layer serve as the output layer.

[0101] The formula for calculating the convolutional layer is as follows:

[0102] ;

[0103] ;

[0104] in, For output channel index, Indicates the convolutional layer's... The convolution result of each output channel at coordinate (i, j), where i is the row index of the input or output data of the convolutional layer, and j is the column index of the input or output data of the convolutional layer. Indicates the first The bias value of each output channel. For input channel index, Here, is the kernel size, u is the row index of the kernel, and v is the column index of the kernel. Let (i+u, j+v) be the input data for the c-th input channel of the convolutional layer at coordinates (i+u, j+v). This represents the weight data of the c-th channel of the convolution kernel at the coordinates (u, v).

[0105] It should also be noted that to ensure consistent input and output sizes for the convolutional layer, the input data needs to be padded. Padding methods can include, but are not limited to, zero-value padding, fixed-value padding, reflection padding, cyclic padding, and copy padding, up to a total of kS loops. The main parameters of convolution are kernel size, number of input channels, number of output channels, padding method, and stride. Therefore, different parameter configurations can be selected to design the convolutional neural network according to the actual situation. Activation functions can be non-linear functions including, but not limited to, Sigmoid, tanh, ReLU, and LeakyReLU as activation functions for the convolutional layer.

[0106] Furthermore, the formula for calculating the average pooling layer is:

[0107] ;

[0108] in, This represents the pooling result of the average pooling layer. Use the row index for the output data of the average pooling layer. Here, H represents the column index of the average pooling layer output data, H is the height of the pooling window, and W is the width of the pooling window. The input data for the average pooling layer is located at coordinates (x+h, y+w), where x is the row index of the input data for the average pooling layer and y is the column index of the input data for the average pooling layer.

[0109] It should be noted that the stride of a pooling layer is generally consistent with the width and height of the pooling window, and the ratios of the width and height of the input data to the width and height of the output data are W and H, respectively. The design of the pooling window is consistent with the ratios of the logical backlight data (high-number-partition backlight data) and the actual backlight module backlight data (low-number-partition backlight data), and the width-to-height ratios of the high-number-partition backlight data and the low-number-partition backlight data are W and H, respectively.

[0110] For example, see Figure 7 As shown, the overall training process of the backlight fusion model built based on a convolutional neural network can be as follows: Training sample data is read from a pre-built training set, i.e., the current image sample and its corresponding high-number-partition backlight data are read. If there is no high-number-partition backlight data, a backlight extraction algorithm can be used to extract the high-number-partition backlight data corresponding to the current image sample. Furthermore, if there is a need to accelerate the training process, multiple sets of sample data can be read at once, such as batch_size sets of sample data. Training is performed based on multiple sets of sample data. The backlight fusion model is used to fuse the high-number-partition backlight data into low-number-partition backlight data. The display effect of the image is simulated based on the high-number-partition backlight data and the low-number-partition backlight data to obtain the first simulation result corresponding to the high-number-partition backlight data and the second simulation result corresponding to the low-number-partition backlight data. Based on the first and second simulation results, the parameters of the backlight fusion model are fine-tuned. It is then determined whether all sample data in the training set has been processed. If so, it is determined whether the backlight fusion model should be trained N times. If it has been trained N times, training ends, and the backlight fusion model with the fine-tuned parameters is used as the trained target backlight fusion model. If not all sample data in the training set has been processed, the next set of sample data is read from the training set, and training continues until the backlight fusion model has been trained N times. Here, N is the preset number of training iterations.

[0111] As can be seen, in this embodiment of the invention, the pre-trained target backlight fusion model can fuse the backlight data of the logical number of partitions into the backlight data of the actual number of partitions, and the actual number of partitions is less than the logical number of partitions. This allows the image quality enhancement effect to be close to that of a backlight module with a high number of backlight partitions to be achieved on a backlight module with a low number of backlight partitions. In other words, the number of backlight partitions can be reduced without losing too much display performance, thereby reducing manufacturing costs to a certain extent.

[0112] For example, when the resolution of the image to be displayed is 3840×2160, the corresponding number of logical partitions (high backlight partitions) can be 64×36, and the number of actual physical partitions (low backlight partitions) can be 32×18, meaning that every 2×2 logical partitions correspond to one actual physical partition. Furthermore, all convolutional layers in the backlight fusion model built based on a convolutional neural network can use symmetrical padding with a stride of 1. The kernel size and number of input / output channels from the first to the second convolutional layer are as follows:

[0113] The first convolutional layer has a kernel size of 5×5, 1 input channel, and 8 output channels.

[0114] The second convolutional layer has a kernel size of 3×3, 8 input channels, and 32 output channels.

[0115] The third convolutional layer has a kernel size of 3×3, 32 input channels, and 8 output channels.

[0116] The fourth convolutional layer has a kernel size of 3×3, 8 input channels, and 1 output channel.

[0117] The pooling window size of the average pooling layer is 2×2, which corresponds to the relationship between logical partitions and actual physical partitions.

[0118] The backlight fusion model based on the convolutional neural network is trained according to the above model training process. The preset training number N can be 1000 and the batch_size is 16, until the display effect of the backlight data with a low number of partitions approaches the display effect of the backlight data with a high number of partitions. After the training is completed, the trained target backlight fusion model can be used to fuse the backlight data with a high number of partitions into backlight data with a low number of partitions. The backlight data with the actual number of partitions and the image to be displayed are then transmitted to the preset display device for display.

[0119] It should be noted that in traditional algorithms, data from corresponding partitions are typically merged directly to obtain backlight data for low-partition counts. For example, averaging the backlight data for high-partition counts would be performed.

[0120] ;

[0121] Where LED(i,j) represents the low-number backlight data, i represents the row index of the low-number backlight data, and j represents the column index of the low-number backlight data; LED_1, LED_2, LED_3, and LED_4 represent the corresponding high-number backlight data.

[0122] See Figure 8As shown, in traditional algorithms, each low-number backlight data segment is fused from 2×2 high-number backlight data segments, meaning each low-number backlight data segment contains the information of 2×2 high-number backlight data segments. However, in this application, in the pre-trained target backlight fusion model, a 3×3 convolutional layer fuses 3×3 data segments, a 5×5 convolutional layer fuses 5×5 data segments, and a 2×2 pooling layer fuses 2×2 data segments. Therefore, four convolutional layers and one pooling layer fuse a total of 11 high-number backlight data segments, fusing more information than traditional algorithms, thus having a greater probability of achieving better image quality enhancement.

[0123] For example, when the resolution of the image to be displayed is 3840×2160, the corresponding number of logical partitions (high backlight partitions) can be 64×18, and the actual number of physical partitions (low backlight partitions) can be 32×18, meaning that every 2×1 logical partitions correspond to one actual physical partition. Furthermore, all convolutional layers in the backlight fusion model built based on a convolutional neural network can use zero-padding with a stride of 1. The kernel size and number of input / output channels from the first to the second convolutional layer are as follows:

[0124] The first convolutional layer has a kernel size of 3×3, 1 input channel, and 2 output channels;

[0125] The second convolutional layer has a kernel size of 3×3, 2 input channels, and 2 output channels.

[0126] The third convolutional layer has a kernel size of 3×3, with 2 input channels and 2 output channels.

[0127] The fourth convolutional layer has a kernel size of 3×3, 2 input channels, and 1 output channel.

[0128] The pooling window size of the average pooling layer is 2×1, which corresponds to the relationship between logical partitions and actual physical partitions.

[0129] The backlight fusion model based on the convolutional neural network is trained in the same way as the model training process described above. The preset number of training iterations N can be 1000 and the batch size can be 32, until the display effect of the backlight data with a low number of partitions approaches the display effect of the backlight data with a high number of partitions. After training is completed, the trained target backlight fusion model can be used to fuse the backlight data with a high number of partitions into backlight data with a low number of partitions. The backlight data with the actual number of partitions and the image to be displayed are then transmitted to the preset display device for display.

[0130] As can be seen from the above, this application utilizes the learning capability of the backlight fusion model to calculate new low-partition backlight data based on a large range of high-partition backlight data. It can perform logical multiplication at arbitrary integer scales in rows and columns, making it more versatile and adaptable to more backlight module solutions. In simulations of backlight solutions with different partition numbers, it can perform relatively realistic simulations of the obtained backlight data based on the parameters of the actual backlight module and the partition data, thereby guiding the training of the backlight fusion model built on the convolutional neural network and giving full play to the learning capability of the backlight fusion model.

[0131] It should be noted that high-zone-count backlight data has different LED density in the horizontal and vertical directions. Therefore, the backlight module will undergo special processing, such as adding corresponding optical films and designing special brackets, to obtain uniform backlight. See [link to relevant documentation]. Figure 9 As shown. Although the LED itself is a circular diffuser (i.e., the diffusion effect is consistent in both horizontal and vertical directions), the diffusion effect of the LED becomes inconsistent in both horizontal and vertical directions after passing through the optical film, resulting in the actual diffusion effect of a single LED not being consistent in the horizontal and vertical directions.

[0132] In one embodiment, such as Figure 10 As shown, based on the above-mentioned backlight fusion convolutional neural network image quality enhancement method, the present invention also provides a backlight fusion convolutional neural network image quality enhancement system, comprising:

[0133] The acquisition module 11 is used to acquire the image to be displayed and extract the backlight data of the number of logical partitions corresponding to the image to be displayed;

[0134] The fusion module 12 is used to input the backlight data of the logical number of partitions into the pre-trained target backlight fusion model to obtain the backlight data of the actual number of partitions output by the target backlight fusion model after fusing the backlight data of the logical number of partitions.

[0135] Display module 13 is used to transmit the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display;

[0136] Wherein, the actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions satisfies a preset similarity condition.

[0137] Furthermore, it is worth noting that the working process of the backlight fusion convolutional neural network image enhancement system provided in this embodiment is the same as the working process of the backlight fusion convolutional neural network image enhancement method described above, and will not be repeated here. For details, please refer to the working process of the backlight fusion convolutional neural network image enhancement method described above.

[0138] Figure 11 A schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include:

[0139] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0140] When the processor 502 executes the program, it implements the backlight fusion convolutional neural network image quality enhancement method provided in the above embodiments.

[0141] Furthermore, the terminal also includes:

[0142] Communication interface 503 is used for communication between memory 501 and processor 502.

[0143] The memory 501 is used to store computer programs that can run on the processor 502.

[0144] Memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0145] If the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0146] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0147] Processor 502 may be a central processing unit, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of this application.

[0148] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described backlight fusion convolutional neural network image quality enhancement method.

[0149] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0150] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0151] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can read and execute instructions from and from an instruction execution system, apparatus or device).

[0152] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the n steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0153] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for enhancing image quality using a backlight-blending convolutional neural network, characterized in that, The method includes: Obtain the image to be displayed, and extract the backlight data of the number of logical partitions corresponding to the image to be displayed; The backlight data of the logical partition number is input into the pre-trained target backlight fusion model to obtain the backlight data of the actual partition number output by the target backlight fusion model after fusing the backlight data of the logical partition number. The backlight data of the actual number of partitions and the image to be displayed are transmitted to a preset display device for display. Wherein, the actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions satisfies a preset similarity condition. The training process of the pre-trained target backlight fusion model includes: A set of image sample data is selected from the pre-constructed training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number; The backlight samples of the logical number of current image samples are input into the backlight fusion model built based on the convolutional neural network to obtain the backlight samples of the actual number of partitions output by the backlight fusion model. Based on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the logical number of partitions and the second simulation result corresponding to the backlight samples of the actual number of partitions. The parameters of the backlight fusion model are fine-tuned based on the first simulation results and the second simulation results to obtain the current fine-tuned backlight fusion model; Determine whether all image sample data in the training set have been selected as training samples; When all image sample data in the training set are selected as training samples, it is determined that one training of the backlight fusion model has been completed, the current number of training iterations of the backlight fusion model is determined, and it is determined whether the current number of training iterations has reached the preset number of training iterations. When the current number of training iterations reaches the preset number of training iterations, the current fine-tuned backlight fusion model is determined as the trained target backlight fusion model.

2. The image quality enhancement method using a backlight fusion convolutional neural network according to claim 1, characterized in that, After determining whether all image sample data in the training set have been selected as training samples, the method further includes: If none of the image sample data in the training set is selected as training samples, it is determined that the training of the backlight fusion model has not been completed, and the step of selecting a set of image sample data from the pre-built training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number and subsequent steps are continued until all the image sample data in the training set is selected as training samples. Furthermore, after determining whether the current training iterations have reached the preset training iterations, the process also includes: If the current training iterations have not reached the preset number of training iterations, the steps of selecting a set of image sample data from the pre-constructed training set as training samples to obtain the current image sample and its corresponding logical partition number of backlight samples, and subsequent steps, are repeated until the current training iterations reach the preset number of training iterations.

3. The image quality enhancement method using a backlight fusion convolutional neural network according to claim 1, characterized in that, The backlight samples based on the logical number of partitions and the backlight samples based on the actual number of partitions are used to simulate the display effect of the current image sample, respectively, to obtain a first simulation result corresponding to the backlight samples of the logical number of partitions and a second simulation result corresponding to the backlight samples of the actual number of partitions, including: The backlight samples of the aforementioned logical partition number are subjected to diffusion processing to obtain the first diffusion result; Pixel compensation is performed on the image sample based on the first diffusion result to obtain a first compensated image sample; Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the number of logical partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the number of logical partitions. Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the second simulation result corresponding to the backlight samples of the actual number of partitions. Alternatively, the backlight samples of the logical number of partitions are diffused to obtain a first diffusion result, and the backlight samples of the actual number of partitions are diffused to obtain a second diffusion result. Pixel compensation is performed on the image sample based on the first diffusion result to obtain a first compensated image sample, and pixel compensation is performed on the image sample based on the second diffusion result to obtain a second compensated image sample. Based on the pixel values ​​corresponding to each pixel point on the first compensated image sample, the liquid crystal module parameters, and the backlight samples of the number of logical partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the number of logical partitions. Based on the pixel values ​​corresponding to each pixel point on the second compensated image sample, the liquid crystal module parameters, and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the second simulation result corresponding to the backlight samples of the actual number of partitions.

4. The image quality enhancement method using a backlight fusion convolutional neural network according to claim 3, characterized in that, The diffusion processing of the backlight samples of the logical partition number to obtain the first diffusion result includes: The backlight samples of the logical partition number are diffused using anisotropic Gaussian blur diffusion method to obtain the first diffusion result; Furthermore, the diffusion processing of the backlight samples of the actual number of partitions to obtain the second diffusion result includes: The backlight samples of the actual number of partitions are diffused using an isotropic Gaussian blur diffusion method to obtain a second diffusion result.

5. The image quality enhancement method using a backlight fusion convolutional neural network according to any one of claims 1 to 4, characterized in that, The convolutional neural network includes one average pooling layer, four convolutional layers, and activation function layers corresponding to each convolutional layer. The first convolutional layer and its corresponding activation function layer serve as the input layer, the second convolutional layer and its corresponding activation function layer, the third convolutional layer and its corresponding activation function layer, and the average pooling layer serve as hidden layers, and the fourth convolutional layer and its corresponding activation function layer serve as the output layer.

6. The image quality enhancement method using a backlight fusion convolutional neural network according to claim 5, characterized in that, The formula for calculating the convolutional layer is: ； ； in, For output channel index, Indicates the convolutional layer's... The convolution result of each output channel at coordinate (i, j), where i is the row index of the input or output data of the convolutional layer, and j is the column index of the input or output data of the convolutional layer. Indicates the first The bias value of each output channel. For input channel index, Here, is the kernel size, u is the row index of the kernel, and v is the column index of the kernel. Let (i+u, j+v) be the input data for the c-th input channel of the convolutional layer at coordinates (i+u, j+v). This represents the weight data of the c-th channel of the convolution kernel at the (u, v) coordinate; Furthermore, the formula for calculating the average pooling layer is: ； in, This represents the pooling result of the average pooling layer. Use the row index of the output data from the average pooling layer. Here, H represents the column index of the average pooling layer output data, H is the height of the pooling window, and W is the width of the pooling window. The input data for the average pooling layer is located at coordinates (x+h, y+w), where x is the row index of the input data for the average pooling layer and y is the column index of the input data for the average pooling layer.

7. A backlight fusion convolutional neural network image enhancement system, characterized in that, The system includes: The acquisition module is used to acquire the image to be displayed and extract the backlight data of the number of logical partitions corresponding to the image to be displayed; The fusion module is used to input the backlight data of the logical number of partitions into the pre-trained target backlight fusion model to obtain the backlight data of the actual number of partitions output by the target backlight fusion model after fusing the backlight data of the logical number of partitions. The display module is used to transmit the backlight data of the actual number of partitions and the image to be displayed to a preset display device for display. Wherein, the actual number of partitions is less than the logical number of partitions, and the similarity between the display effect of the backlight data of the actual number of partitions output by the target backlight fusion model and the display effect of the backlight data of the logical number of partitions satisfies a preset similarity condition. The training process of the pre-trained target backlight fusion model includes: A set of image sample data is selected from the pre-constructed training set as training samples to obtain the current image sample and the backlight sample of its corresponding logical partition number; The backlight samples of the logical number of current image samples are input into the backlight fusion model built based on the convolutional neural network to obtain the backlight samples of the actual number of partitions output by the backlight fusion model. Based on the backlight samples of the logical number of partitions and the backlight samples of the actual number of partitions, the display effect of the current image sample is simulated to obtain the first simulation result corresponding to the backlight samples of the logical number of partitions and the second simulation result corresponding to the backlight samples of the actual number of partitions. The parameters of the backlight fusion model are fine-tuned based on the first simulation results and the second simulation results to obtain the current fine-tuned backlight fusion model; Determine whether all image sample data in the training set have been selected as training samples; When all image sample data in the training set are selected as training samples, it is determined that one training of the backlight fusion model has been completed, the current number of training iterations of the backlight fusion model is determined, and it is determined whether the current number of training iterations has reached the preset number of training iterations. When the current number of training iterations reaches the preset number of training iterations, the current fine-tuned backlight fusion model is determined as the trained target backlight fusion model.

8. A terminal, characterized in that, include: The system includes a memory, a processor, and a backlight fusion convolutional neural network image enhancement program stored in the memory and executable on the processor, wherein the backlight fusion convolutional neural network image enhancement program, when executed by the processor, implements the steps of the backlight fusion convolutional neural network image enhancement method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed to implement the steps of the backlight fusion convolutional neural network image enhancement method as described in any one of claims 1 to 6.

Images