Color compensation system and method based on adaptive toning

Abstract

The invention relates to the technical field of image processing, and provides a color compensation system and method based on adaptive toning, through close combination of four steps, from acquisition of multi-source data, determination of key parameters and output of a compensation parameter matrix to final compensation optimization, all the steps are progressive layer by layer and support each other, and the color compensation efficiency is improved. According to the method, the color rendition degree of the image under different ambient light and target equipment conditions can be effectively improved, it is ensured that the color of the output RGB image is accurate and natural and accords with human eye visual perception, the quality and efficiency of image color processing are remarkably improved, and powerful technical support is provided for various application scenes related to image display and processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a color compensation system and method based on adaptive color adjustment. Background Art

[0002] In the current era where digital images are widely used, the accurate restoration and presentation of image colors are crucial. Whether in photography, film and television production, image display or other related fields, users expect to obtain high-quality and color-realistic image effects. However, most of them do not solve the problem of eliminating residual color differences and color compensation under different ambient lights and device scenarios through image data processing. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, this application provides a color compensation system and method based on adaptive color adjustment.

[0004] In a first aspect, this application provides a color compensation method based on adaptive color adjustment. The method includes: obtaining original image data, ambient light data, and the color configuration of the target device through the sampling frequency, aligning the timestamps of the original image data and the ambient light data. The ambient light data includes color temperature and light intensity, and the color configuration of the target device includes the target device color gamut; Identifying the color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the main image colors, calculating the offset of the color temperature under the light intensity according to the ambient light data, determining the color difference data, determining the reference color based on the main image colors and the color difference data, and mapping the ambient light data into the original image data to match the image resolution in the original image data, and outputting the light influence factor of each pixel point in the original image data; Dividing the sensitive areas in the original image data through a semantic segmentation model, outputting the area sensitivity of each pixel point in each sensitive area, and at the same time comparing the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and area sensitivity of each pixel point, combining the compensation intensity of each pixel point to output a compensation parameter matrix, and compensating the reference color according to the compensation parameter matrix; Mapping the compensated reference color to the target device color gamut, outputting an RGB image adapted to the target device, and predicting the color difference data of the sensitive areas in the RGB image through a deep residual network, and performing hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0005] As an optional implementation manner, the step of determining the color difference data includes: Training the historical ambient light data through a recurrent neural network to predict the target color temperature under the light intensity; Calculate the deviation between the actual color temperature and the target color temperature to obtain the offset of the color temperature under the illumination intensity; Set the light intensity influence coefficient according to the illumination intensity, and determine the color difference data based on the light intensity influence coefficient and the offset of the color temperature under the illumination intensity.

[0006] As an optional implementation manner, the step of determining the reference color includes: Calculate the hue weight of each image main hue according to the proportion of each image main hue in the original image data and the visual saliency of each image main hue, and determine the clustering center color of each image main hue; Calculate the standard deviation of the ambient light data and the historical ambient light data to determine the ambient light weight, and set the calibration color; Determine the reference color by weighted summation of the hue weight and the clustering center color, as well as the ambient light weight and the calibration color.

[0007] As an optional implementation manner, the output step of the light influence factor of each pixel includes: Map the ambient light data to the pixel points in the original image data through the ray tracing algorithm, and for each pixel point, output the ray propagation path from the ambient light source to the pixel point; Judge whether there is an occluder on each ray propagation path to determine the occlusion coefficient of each pixel point; Determine the influence of the illumination intensity and color temperature in the ambient light data on each pixel point to obtain the influence coefficient of each pixel point; Combine the occlusion coefficient and the influence coefficient of each pixel point to output the light influence factor of each pixel point.

[0008] As an optional implementation manner, the output step of the regional sensitivity of each pixel includes: Divide the sensitive regions in the original image data through the semantic segmentation model, and determine the semantic category of each pixel point in each sensitive region through the attention mechanism; Extract the modal feature vector combining the convolutional neural network and the long short-term memory network for each pixel point, and set the initial sensitivity for each semantic category in combination with the modal feature vector to obtain the initial regional sensitivity of each pixel point; Based on bilateral filtering, determine the spatial distance and feature similarity between pixel points, and output the regional sensitivity of each pixel point according to the spatial distance and feature similarity between pixel points.

[0009] As an optional implementation manner, the output step of the compensation parameter matrix includes: Convert the reference color and the target reference color to the perceptual hash space, and obtain the reference color deviation of each pixel point by calculating the Hamming distance between the perceptual hash values of the reference color and the target reference color; Construct reinforcement learning, determine that the state space is the reference color deviation, light influence factor, and regional sensitivity of each pixel point, and the action space is the weight combination of the reference color deviation, light influence factor, and regional sensitivity of each pixel point, and continuously iterate and train the reinforcement learning; Calculate the compensation intensity of each pixel point according to the obtained weight combination, and output a compensation parameter matrix in combination with the color correction parameter.

[0010] As an optional implementation manner, the determining step of the sampling frequency includes: Determine the load factor of the target device according to the CPU computing power and memory bandwidth of the target device, set a reference sampling rate based on the type of the target device, and determine the sampling rate in combination with the load factor of the target device; Based on the change gradient of the color temperature and the change gradient of the illumination intensity in the historical ambient light data, calculate the change rate of the ambient light, and correct the sampling rate according to the change rate of the ambient light; Determine the image complexity according to the clustering dispersion of the main color of the image and the edge density of the original image data, and optimize the sampling rate according to the image complexity to obtain the finally determined sampling frequency.

[0011] As an optional implementation manner, the execution steps of the hierarchical re-compensation include: Layer the RGB image based on the visual saliency and image complexity of the RGB image, and set the compensation priority for each layer; Predict the color difference data of the sensitive area in the RGB image through a deep residual network, calculate the average color difference of each layer according to the predicted color difference data, and set the trigger threshold for each layer, and compare the average color difference of each layer with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer; For the layers that trigger re-compensation, adjust the weight combination to re-output the compensation parameter matrix, and update the sampling frequency according to the execution effect of the re-compensation.

[0012] In a second aspect, the present application provides a color compensation system based on adaptive color adjustment. The system includes: obtaining original image data, ambient light data, and the color gamut of the target device through the sampling frequency, and aligning the time stamps of the original image data and the ambient light data; Identify the color cluster with a proportion greater than 15% in the original image data through a clustering algorithm as the main color of the image, calculate the offset of the color temperature under the illumination intensity according to the ambient light data, determine the color difference data, determine the reference color based on the main color of the image and the color difference data, and map the ambient light data into the original image data to output the light influence factor of each pixel point in the original image data; Divide the sensitive regions in the original image data through a semantic segmentation model, output the regional sensitivity of each pixel in each sensitive region, and at the same time compare the reference color with the target reference color to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor and regional sensitivity of each pixel, combine the compensation intensity of each pixel to output a compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix; Map the compensated reference color to the target device color gamut, output an RGB image adapted to the target device, and predict the color difference data of the sensitive regions in the RGB image through a deep residual network. Perform hierarchical re-compensation according to the change situation of the color difference data to update the compensation parameter matrix and sampling frequency.

[0013] Compared with the prior art, the beneficial effects of this application are as follows: Through the close combination of four steps, from the acquisition of multi-source data, the determination of key parameters, the output of the compensation parameter matrix to the final compensation optimization, each step progresses layer by layer and supports each other, which can effectively improve the color restoration degree of the image under different ambient light and target device conditions, ensure that the output RGB image has accurate and natural colors, conforms to the human eye visual perception, significantly improves the quality and efficiency of image color processing, and provides strong technical support for various application scenarios involving image display and processing.

[0014] Obtain the original image data, ambient light data and target device color configuration through the sampling frequency, and perform timestamp alignment, providing a comprehensive and synchronized data basis for subsequent color compensation. The cooperation of data from different sources enables color compensation to comprehensively consider the characteristics of the image itself, the influence of ambient light and the display ability of the target device, avoiding color compensation deviation caused by data loss or asynchronization.

[0015] Identify the main color of the image, determine the color difference data and reference color, and calculate the light influence factor. Extract key parameters from both aspects of the image content and ambient light factors. These parameters are closely related to the essential characteristics of the image color and the environmental influence, providing the core basis for the calculation of subsequent compensation parameters, enabling color compensation to be carried out based on the actual situation and environmental conditions of the image, and significantly improving the pertinence and accuracy of compensation.

[0016] Divide the sensitive regions and determine the regional sensitivity, combine the reference color deviation and light influence factor to calculate the compensation parameter matrix, providing an accurate operation direction for color compensation. Generate personalized compensation parameters for different regional sensitivity and color deviation situations, realize precise compensation for each part of the image, effectively improve the overall color quality of the image, and highlight the color performance of important regions.

[0017] Map the compensated reference color to the target device color gamut to ensure accurate display of the output image on the target device. Predict the color difference data through a deep residual network and perform hierarchical re-compensation. Dynamically monitor and optimize the color compensation effect to further improve the image color quality. At the same time, update the sampling frequency according to the re-compensation effect to achieve an adaptive match between data acquisition and compensation requirements, and improve the overall performance and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them: Figure 1 It is a flowchart of the method for color compensation based on adaptive color adjustment provided by the embodiment of the present application; Figure 2 It is an output step diagram of the light influence factor of each pixel point of the method for color compensation based on adaptive color adjustment provided by the embodiment of the present application; Figure 3 It is an execution step diagram of the hierarchical re-compensation of the method for color compensation based on adaptive color adjustment provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present application more obvious and understandable, the following clearly and completely describes the technical solutions in the embodiments of the present application with reference to the drawings in the specification. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments.

[0020] Embodiment 1 As Figure 1 shown, it is a flowchart of the method for color compensation based on adaptive color adjustment provided by the embodiment of the present application. The method includes: S1. Obtain the original image data, ambient light data, and color configuration of the target device through the sampling frequency, align the time stamps of the original image data and the ambient light data. The ambient light data includes the color temperature and the illumination intensity, and the color configuration of the target device includes the target device color gamut.

[0021] Among them, the original image data is the unprocessed image data directly obtained from the camera, usually represented in the form of a pixel matrix, containing the color and brightness information of the image; the ambient light data refers to the light information in the surrounding environment, including the color temperature and the light intensity. The color temperature reflects the color characteristics of the light, and the light intensity reflects the brightness of the light. An ambient light sensor is used to obtain the ambient light data; the color configuration of the target device refers to the color range and characteristics that the target device can display, mainly including the target device color gamut. The target device color gamut defines the set of colors that the target device can display, and the color configuration information of the target device can be obtained by querying the technical specification document of the target device; timestamp alignment means synchronizing the original image data and the ambient light data in time to ensure that the original image data and the ambient light data obtained at the same time point are corresponding. When obtaining the original image data and the ambient light data, the timestamp of each data is recorded at the same time. In subsequent processing, the corresponding image data and ambient light data are matched through the timestamp. For example, methods such as linear interpolation or nearest neighbor interpolation are used to align the timestamps.

[0022] The steps for determining the sampling frequency include: Determine the load factor of the target device according to the CPU computing power and memory bandwidth of the target device, set the reference sampling rate based on the type of the target device, and determine the sampling rate in combination with the load factor of the target device; Based on the change gradients of the color temperature and the light intensity in the historical ambient light data, calculate the change rate of the ambient light, and correct the sampling rate according to the change rate of the ambient light; Determine the image complexity according to the clustering dispersion of the main color of the image and the edge density of the original image data, and optimize the sampling rate according to the image complexity to obtain the finally determined sampling frequency.

[0023] Different target devices have different computing capabilities and resource limitations. The CPU computing power and memory bandwidth directly affect the speed and ability of the device to process data. If the sampling frequency is too high, the device cannot process all the data in time, resulting in system jams or even crashes. If the sampling frequency is too low, the changes in the ambient light and the image cannot be captured in time. Therefore, it is necessary to reasonably adjust the sampling rate according to the load situation of the device. At the same time, different types of devices have differences in usage scenarios and performance. Setting the reference sampling rate can provide a basis for subsequent adjustments.

[0024] Obtain the CPU usage rate and memory usage rate of the target device through a system monitoring tool. The load factor of the target device is obtained by weighted summing the CPU usage rate and the memory usage rate. Classify according to the type of the target device, such as mobile phones, tablets, and computers, etc. Through experiments and tests, set a suitable benchmark sampling rate for each type of device. For example, the benchmark sampling rate for mobile phones is set to 10 Hz, for tablets is 8 Hz, and for computers is 12 Hz. Then the sampling rate is obtained by multiplying the difference between 1 and the load factor by the benchmark sampling rate; thus, it is possible to dynamically adjust the sampling rate according to the actual load situation of the device, avoid performance problems caused by insufficient processing power of the device, and at the same time ensure that the sampling rate can be appropriately increased when the device load is low to obtain data more timely.

[0025] The change situation of ambient light is an important factor affecting image color. If the ambient light changes rapidly, a higher sampling frequency is required to capture these changes in a timely manner for accurate color compensation. If the ambient light changes slowly, the sampling frequency is appropriately reduced to reduce the data processing volume; record the ambient light data for a period of time (such as the past 10 seconds), including color temperature and light intensity, calculate the change gradient of color temperature and the change gradient of light intensity. The change gradient of color temperature is obtained by dividing the color temperature difference value in this time period by the time interval, and the change gradient of light intensity is obtained by dividing the light intensity difference value in this time period by the time interval, and obtain the change rate of ambient light by weighted summing the change gradient of light intensity and the change gradient of color temperature. Set a change rate threshold. When the change rate of ambient light is greater than the change rate threshold, correct the sampling rate. The corrected sampling rate = sampling rate × (1 + (change rate of ambient light - change rate threshold) / change rate threshold); thus, it is possible to dynamically adjust the sampling rate according to the actual change situation of ambient light, ensure that data can be obtained in a timely manner when the ambient light changes violently, and improve the accuracy of color compensation.

[0026] Image complexity is also an important factor affecting the sampling frequency. Complex images contain more details and variations and require a higher sampling frequency to accurately capture this information, while simple images require an appropriately reduced sampling frequency. The K-means algorithm is used to cluster the original image data to obtain the dominant color of the image. The average distance between each cluster center and other cluster centers is calculated as the cluster dispersion. The larger the cluster dispersion, the more dispersed the distribution of the dominant color of the image and the more complex the image. The Canny edge detection algorithm is used to perform edge detection on the original image data to obtain an edge image. The ratio of the number of edge pixels in the edge image to the total number of pixels is calculated as the edge density. The larger the edge density, the richer the edge information of the image and the more complex the image. The image complexity is obtained by weighted summation of the cluster dispersion and the edge density. A complexity threshold is set. When the image complexity is greater than the complexity threshold, it is determined as a complex scene, and the optimized sampling rate = the corrected sampling rate × (1 + (image complexity - complexity threshold) / (1 - complexity threshold)); otherwise, it is a simple scene, and the corrected sampling rate does not need to be optimized. Thus, the sampling rate can be dynamically adjusted according to the actual complexity of the image to ensure that more detailed information can be obtained when processing complex images, improving the effect of color compensation. The finally determined sampling frequency will determine the frequency of subsequent acquisition of the original image data, thereby affecting the recognition of the dominant color of the image and the calculation of color difference data in subsequent steps.

[0027] S2. Identify the color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the dominant color of the image. Calculate the offset of the color temperature under the illumination intensity according to the ambient light data to determine the color difference data. Determine the reference color based on the dominant color of the image and the color difference data, and map the ambient light data to the original image data to match the image resolution in the original image data, and output the light influence factor of each pixel point in the original image data.

[0028] Using the K-means clustering algorithm, first determine the number of clusters K according to experience or preliminary analysis, and determine a suitable value through multiple experiments. For example, for general scene images, K is set to 5-10. Then, according to the initialization rule of the K-means algorithm, select pixel colors that are far apart as the initial cluster centers. For each pixel point in the original image data, calculate its distance from each cluster center. In the RGB color space, the Euclidean distance can be used. Assign the pixel point to the cluster where the nearest cluster center is located, recalculate the average value of the pixel colors within each cluster, and update the cluster centers. Repeat the above assignment and update process until the cluster centers no longer change significantly. For example, the moving distance of the cluster centers in consecutive multiple iterations is less than a certain threshold. Finally, count the proportion of the number of pixel points in each cluster to the total number of pixel points, and take the color corresponding to the cluster with a proportion greater than 15% as the main color tone of the image. For example, after clustering, it is found that the number of pixel points in a certain cluster accounts for 20% of the total number of pixels, then the average color of this cluster is a main color tone of the image.

[0029] The steps for determining the color difference data include: Train the historical ambient light data through a recurrent neural network to predict the target color temperature under the illumination intensity; Calculate the deviation between the actual color temperature and the target color temperature to obtain the offset of the color temperature under the illumination intensity; Set the light intensity influence coefficient according to the illumination intensity, and determine the color difference data according to the light intensity influence coefficient and the offset of the color temperature under the illumination intensity.

[0030] There is a complex correlation between the color temperature and the illumination intensity in the ambient light data, and this relationship changes with time and the environment. Using a recurrent neural network can effectively capture the time series characteristics in the historical ambient light data, so as to accurately predict the target color temperature under the current illumination intensity. This can provide a reference color temperature value that is more in line with the actual environmental changes for subsequent color difference calculation; Continuously collect ambient light data for a period of time (such as the past week), including color temperature and illumination intensity, and record a data point every 5 minutes. Normalize the collected data so that the values of color temperature and illumination intensity are both in the range of 0-1, which is convenient for the RNN model to process. Select the long short-term memory network, which is a variant of the RNN and can better handle long-term dependence problems. The model structure of the long short-term memory network includes an input layer, several LSTM hidden layers, and an output layer. The input layer receives the normalized historical illumination intensity sequence, the hidden layer learns the relationship between the illumination intensity and the color temperature through LSTM units, and the output layer outputs the predicted target color temperature. For example, the dimension of the input layer is 1 because only the illumination intensity is input, two hidden layers are set, each hidden layer has 64 LSTM units, and the dimension of the output layer is 1, and the predicted color temperature is output.

[0031] The preprocessed data is divided into a training set and a test set in chronological order. For example, 80% of the data is used for training and 20% for testing. The mean squared error is used as the loss function. During training, the historical sequence of light intensity is used as the input, and the corresponding actual color temperature is used as the label. The model parameters of the long short-term memory network are continuously updated through the backpropagation algorithm to minimize the error between the predicted value and the actual value of the long short-term memory network model. The training process is set to iterate through the training set 50 times; with the powerful time series processing ability of the RNN, it can accurately predict the target color temperature under the light intensity, taking into account the dynamic changes of the ambient light data over time, providing a more reliable reference for subsequent color difference calculation.

[0032] By calculating the deviation between the actual color temperature and the predicted target color temperature, the difference between the ambient light color temperature and the expectation under the current light intensity can be quantified. This difference is one of the key factors in determining the color difference data and can reflect the direction and degree of the impact of ambient light on the image color. The introduction of the light intensity influence coefficient takes into account the non-linear relationship between the light intensity and the color temperature deviation, making the calculation result more in line with the human eye's perception characteristics of color under different light intensities; subtracting the target color temperature predicted by the RNN from the actual color temperature to obtain the deviation between the actual color temperature and the target color temperature, and the setting of the light intensity influence coefficient is through calculated. represents the light intensity, and are constants obtained by experimental fitting; thus, the offset of the color temperature under the light intensity is clearly quantified, and the influence of the light intensity on the perception of color temperature deviation is considered through the light intensity influence coefficient, providing the necessary data for accurate color difference calculation.

[0033] The light intensity has a great influence on the human eye's perception of color. Under different light intensities, the same color temperature change will cause different degrees of color perception differences. By setting the light intensity influence coefficient and combining the offset of the color temperature under the light intensity to determine the color difference data, the comprehensive influence of ambient light on the image color can be considered more comprehensively, improving the accuracy of color compensation; multiplying the light intensity influence coefficient obtained in the previous step by the offset of the color temperature under the light intensity to obtain the color difference data; thus, the influence of both the light intensity and the color temperature offset on the color is considered, making the calculated color difference data more capable of reflecting the change of the image color under the actual ambient light, which helps to perform more accurate color compensation subsequently. Accurate color difference data is an important basis for determining the reference color, which directly affects the accuracy of determining the reference color based on the main color tone of the image and the color difference data and performing color compensation based on the reference color.

[0034] The steps for determining the reference color include: Calculate the hue weight of each main hue of the image based on the proportion of each main hue of the image in the original image data and the visual saliency of each main hue of the image, and determine the cluster center color of each main hue of the image; Calculate the standard deviation of the ambient light data and the historical ambient light data to determine the ambient light weight, and set the calibration color; Determine the reference color by weighted summation of the hue weight and the cluster center color, as well as the ambient light weight and the calibration color.

[0035] The proportion of the main hue of the image in the image reflects its importance in the overall color composition of the image, while the visual saliency reflects the prominence of the color in the human eye's visual perception. Calculating the hue weight by combining these two factors can more reasonably determine the contribution of each main hue when determining the reference color. Determining the cluster center color as the representative color of the main hue for subsequent weighted calculations makes the determination of the reference color more representative; First, cluster the original image data through the K-means algorithm, and identify the color clusters with a proportion greater than 15% as the main hues of the image. Suppose there are main hues of the image. For the th main hue of the image, its proportion in the original image data is , and the visual saliency is obtained by calculating factors such as the contrast between the main hue of the image and the surrounding colors and its spatial distribution in the original image data, denoted as . The hue weight is calculated by the formula , where is a balance coefficient that needs to be dynamically adjusted according to the types of different original image data. During the clustering process, the K-means algorithm will automatically determine the center of each cluster. For the cluster corresponding to each main hue of the image, the color value of the center of this cluster is used as the cluster center color; thus, comprehensively considering the proportion and visual saliency of the main hue of the image to calculate the weight makes the determination of the reference color more in line with the human eye's perception of the image color and the color composition of the image itself, improving the accuracy and representativeness of the reference color.

[0036] The stability of ambient light has an important impact on image color. By calculating the standard deviation of the ambient light data and the historical ambient light data, the degree of change of the current ambient light can be measured. The smaller the change in ambient light, the greater the influence weight on the determination of the reference color, and vice versa. The calibration color is set to eliminate the errors of the sensor itself and some systematic errors in the ambient light measurement, so that the ambient light data can more accurately reflect the influence of the actual ambient light on the image color; calculate the standard deviation of the current ambient light data and the historical ambient light data. First, calculate the standard deviation of the color difference and the standard deviation of the light intensity, and then perform a weighted summation calculation on the standard deviation of the color difference and the standard deviation of the light intensity to obtain the standard deviation of the current ambient light data and the historical ambient light data. Set the standard deviation threshold, then the ambient light weight = 1 / (1 + e^(standard deviation - standard deviation threshold)). At the same time, during the calibration process, use a standard light source to measure the ambient light sensor multiple times, record the output data of the sensor under the standard light source, and establish a calibration table based on these data. When the current ambient light data is obtained, through querying the calibration table and performing interpolation calculation, the calibrated ambient light data is obtained, and the corresponding color is used as the calibration color; thus, the ambient light weight is determined through the standard deviation, which can dynamically adjust the role of ambient light in the determination of the reference color, improve the adaptability to changes in ambient light, and the setting of the calibration color improves the accuracy of the ambient light data, thereby enhancing the accuracy of the determination of the reference color.

[0037] Combining the hue weight and the clustering center color of the main hue of the image, as well as the ambient light weight and the calibration color, the reference color is determined by weighted summation, which can comprehensively consider the color characteristics of the image itself and the influence of ambient light on color, making the determined reference color more in line with the color that the actual image should present under the current ambient light, providing a more accurate basis for subsequent color compensation. The reference color is determined by integrating two factors: the original image data and the ambient light, which improves the accuracy and adaptability of the reference color, helps to enhance the effect of subsequent color compensation, enables the original image data to restore colors more accurately under different ambient lights, and the accurate reference color is the basis for subsequent color compensation, which has a decisive impact on determining the compensation parameter matrix for each pixel point and the color accuracy of the final output RGB image adapted to the target device.

[0038] Ambient light data is the measured value of ambient light, and the original image data has a specific resolution, that is, the number of pixels in the horizontal and vertical directions of the image. Mapping the ambient light data into the original image data means making the ambient light data correspond to each pixel point in the original image data, so as to analyze the influence of ambient light on the color of each pixel. Matching the image resolution means reasonably distributing the ambient light data to each pixel position according to the pixel distribution of the image, ensuring that the ambient light data is spatially aligned with the image data.

[0039] Ambient light data is obtained through one or more ambient light sensors, which are located near the image acquisition device or at specific positions in the scene. First, the relative positional relationship between the ambient light sensors and the image acquisition device is determined. If there are multiple sensors, bilinear interpolation is used to estimate the ambient light data at the position of each pixel in the original image data. For a certain pixel in the original image data, find its positional relationship in the grid of sensor measurement values. For example, let the spacing of sensor measurement values in the horizontal direction be , and the spacing in the vertical direction be . The distance of this pixel from the nearest measurement point on the left in the horizontal direction is , and the distance from the nearest measurement point below in the vertical direction is . Then, the ambient light data of this pixel is calculated by bilinear interpolation using the ambient light data of the four surrounding measurement points. In this way, the ambient light data is mapped to each pixel of the original image data, achieving the matching of the ambient light data and the original image data in terms of resolution.

[0040] As Figure 2 shown, the output steps of the light influence factor for each pixel include: Map the ambient light data to the pixels in the original image data through the ray tracing algorithm. For each pixel, output the light propagation path from the ambient light source to the pixel; Judge whether there are occluders on each light propagation path to determine the occlusion coefficient of each pixel; Determine the influence of the light intensity and color temperature in the ambient light data on each pixel to obtain the influence coefficient of each pixel; Combine the occlusion coefficient and influence coefficient of each pixel to output the light influence factor of each pixel.

[0041] Before the ambient light reaches the image pixel, its propagation path will be affected by various objects in the scene, including reflection, refraction, and occlusion, etc. Only by clarifying the specific path of the light from the light source to the pixel can the actual effect of the ambient light on each pixel be accurately analyzed, and thus provide a key basis for calculating the light influence factor; through the ray tracing algorithm, such as the path tracing algorithm, first build a scene model, model the ambient light source as a luminous body with specific position, intensity, and spectral distribution, and define the objects in the scene where the original image data is located according to their geometric shapes and material properties. The geometric shape refers to the polygonal object represented by the triangular mesh, and the material properties include characteristics such as diffuse reflection, specular reflection, and refraction. For each pixel of the original image data, emit a ray from the pixel towards the ambient light source.

[0042] In the process of light propagation, spatial data structures (such as octrees) are used to accelerate the intersection detection between light and objects. When light intersects with an object, the reflection and refraction directions of the light are calculated according to the material properties of the object and the corresponding optical model. The optical model can be used to calculate the reflection and refraction ratios through the Fresnel equation, and continue to track the newly generated light. At each intersection, the intersection position, surface normal, and change in the direction of light propagation are recorded until the light leaves the scene or reaches the maximum tracking depth. For example, in an indoor scene, the light starts from the light source located on the ceiling and reaches a pixel in the original image data after a diffuse reflection of the wall. The optical model will record detailed information such as the position of the reflection point of the light on the wall and the direction of the light before and after the reflection. This accurately simulates the propagation process of light in complex scenes, takes into account a variety of optical phenomena, and provides real and reliable light propagation path information for the subsequent accurate calculation of the occlusion coefficient and the influence coefficient, significantly improving the accuracy of the calculation of the light influence factor, thereby improving the accuracy of color compensation, which is especially suitable for scenes with complex lighting environments.

[0043] The presence of obstructions will change the intensity and color of ambient light reaching the pixel point, which is an important factor affecting the light impact factor. Accurately judging whether there are obstructions on the light propagation path and quantifying the degree of obstruction, that is, determining the obstruction coefficient, can more realistically reflect the actual effect of ambient light on the pixel point, thereby improving the accuracy of light impact factor calculation and making the color compensation result more consistent with the actual scene; in the ray tracing process, when the light intersects with an object in the scene, it is determined whether the object is an obstruction. For opaque objects, it is directly determined that the light is completely blocked. For translucent objects, the light transmission ratio is calculated based on the transmittance of its material and the angle between the light and the surface of the object. The transmittance can be obtained through the optical property parameters of the object material. For example, the transmittance of glass material is usually between 0.8-0.95. Assuming the transmittance of the object is , the cosine of the angle between the light and the surface of the object is , is the angle between the light and the normal line of the object surface, then the calculation formula of the occlusion coefficient is For example, for a piece of glass with a transmittance of 0.9 and light incident at an angle of 30° to the surface normal, cos30°≈0.866, then the occlusion coefficient z=1−0.9×0.866≈0.22; thus, the effect of the obstruction on the propagation of light is taken into account, and the degree of attenuation of the ambient light reaching the pixel point can be accurately quantified according to different occlusion conditions, so that the light impact factor more truly reflects the actual lighting conditions, and the accuracy and reliability of color compensation in complex scenes are improved.

[0044] Illumination intensity and color temperature are the core elements of ambient light affecting image color. Quantifying their influence on each pixel to obtain the influence coefficient can accurately reflect the effect of ambient light on pixel color in these two key aspects, thereby precisely calculating the light influence factor and providing the necessary data support for achieving accurate color compensation. Determining the influence of illumination intensity and color temperature in ambient light data on each pixel to obtain the influence coefficient of each pixel, and the influence coefficient of each pixel = illumination intensity × (1 + (color temperature - standard color temperature) / (maximum color temperature - standard color temperature)), where the standard color temperature refers to the color temperature of the standard light source, and the maximum color temperature refers to the maximum color temperature on the target device. Thus, it comprehensively considers the effect of the key characteristics of ambient light on image color, enabling the influence coefficient to accurately reflect the degree of influence of ambient light on pixel color, providing reliable data for precisely calculating the light influence factor, and significantly improving the accuracy and scientificity of color compensation.

[0045] The occlusion coefficient and the influence coefficient are quantified from two aspects: the occlusion situation during the light propagation process and the influence of the characteristics of ambient light itself on pixel color. Combining them can comprehensively and accurately reflect the comprehensive influence of ambient light on each pixel, thereby obtaining the light influence factor and providing the core data for subsequent color compensation based on ambient light factors. Multiply the occlusion coefficient of each pixel by the influence coefficient of each pixel to obtain the light influence factor, and store the calculated light influence factor in one-to-one correspondence with each pixel in the original image data to form a matrix of light influence factors for subsequent use in the color compensation process. Thus, it comprehensively considers the occlusion factor in light propagation and the ambient light characteristic factor, comprehensively and accurately quantifies the influence of ambient light on each pixel, and the obtained light influence factor can provide accurate ambient light-related parameters for color compensation, effectively improving the accuracy and adaptability of color compensation, enabling the image to more accurately restore color under different ambient light conditions. The light influence factor is one of the important bases for calculating the compensation parameter matrix subsequently, and together with factors such as the deviation from the reference color and the regional sensitivity, it determines the compensation intensity and method for each pixel, playing a crucial role in finally outputting an RGB image with accurate color adapted to the target device.

[0046] S3. Divide the sensitive regions in the original image data through a semantic segmentation model, output the regional sensitivity of each pixel in each sensitive region, and at the same time compare the reference color with the target reference color to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor, and regional sensitivity of each pixel, combine with the compensation intensity of each pixel to output the compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix.

[0047] The output steps of the regional sensitivity of each pixel include: The semantic segmentation model is used to divide the sensitive areas in the original image data, and the attention mechanism is used to determine the semantic category of each pixel in each sensitive area; Extract the modal feature vector of the convolutional neural network combined with the long short-term memory network for each pixel, and set the initial sensitivity for each semantic category based on the modal feature vector to obtain the initial regional sensitivity of each pixel; The spatial distance and feature similarity between pixels are determined based on bilateral filtering, and the regional sensitivity of each pixel is output according to the spatial distance and feature similarity between pixels.

[0048] Different regions have different importance and requirements for color accuracy in the original image data. The semantic segmentation model can divide the original image data into regions with different semantic meanings, such as people, sky, and ground. The attention mechanism can further accurately focus on the uniqueness and importance of each pixel in its semantic category, providing a semantic basis for the subsequent setting of regional sensitivity, so that color compensation can give priority to important areas and key pixels. CNN is used as a semantic segmentation model. During the training process, a large-scale image dataset is used. These images not only mark the bounding boxes of objects, but also mark the semantic categories of each pixel in detail. In the feature extraction stage of the semantic segmentation model, the self-attention module is introduced to allow the semantic segmentation model to fully consider the original image when processing each pixel. The information of pixel points at other positions in the image data can be used to better understand the role of pixel points in the overall semantics. When the original image data is semantically segmented, the semantic segmentation model outputs the semantic category to which each pixel point belongs, such as "person face" and "background building". For example, for a person landscape photo, the semantic segmentation model can accurately mark the pixel points on the person's face as the "person face" category, and mark the pixel points of the distant mountains as the "natural landscape mountain range" category; thereby, the sensitive areas in the original image data can be accurately divided and the semantic category of each pixel point can be determined, which provides a reliable basis for the subsequent setting of regional sensitivity according to semantic importance, improves the pertinence and effectiveness of color compensation, and can highlight the color compensation effect of important areas, especially in complex scene images.

[0049] The features of the pixels in the original image data contain not only spatial information, but also time or context-related information. Combining these two features can more comprehensively describe the characteristics of the pixels. According to the different requirements of different semantic categories for color compensation, these features are combined to set the initial sensitivity for each semantic category, which can preliminarily distinguish the importance of different areas and provide a basis for subsequent optimization. A joint model consisting of a convolutional neural network and a long short-term memory network is constructed. For each pixel, CNN is first used to extract features from the local image block (such as a 3×3 or 5×5 pixel block centered on the pixel). After multiple layers of convolution and pooling operations, CNN obtains a vector containing spatial features. These local image blocks are then input into LSTM in a certain order (such as from left to right or from top to bottom). LSTM captures the contextual dependencies between pixels and outputs a vector containing contextual features. The modal feature vector of each pixel is obtained by concatenating the vector and the vector containing the contextual features. According to the semantic category, for example, for the "human face" semantic category, due to its extremely high requirements for color accuracy, the initial sensitivity is set to 0.8-1.0, and for the "background grass" semantic category, the initial sensitivity is set to 0.3-0.5. For each pixel, according to its semantic category, fine-tuning is performed in combination with its modal feature vector. For example, for a pixel belonging to the "human face" semantic category, if its modal feature vector shows that the pixel is located in a key part of the face, such as near the eyes or lips, its initial sensitivity is set to a value close to 1.0, such as 0.95; thereby, the spatial and contextual features of the pixel are comprehensively considered, and the initial sensitivity is set in combination with the semantic category, so that the sensitivity setting is more scientific and reasonable, which provides a good starting point for subsequent optimization, helps to improve the accuracy of regional sensitivity, and thus improves the effect of color compensation.

[0050] Adjacent pixel points usually have similarities in terms of space and features, and such similarities are related to the regional sensitivity. Based on bilateral filtering, considering both the spatial distance and feature similarity between pixel points, it is possible to smooth and optimize the initial regional sensitivity, making it more in line with the actual characteristics of the image, avoiding sudden changes in sensitivity, and improving the accuracy and stability of the regional sensitivity. For each pixel point, the spatial distance between adjacent pixel points is determined according to the image resolution and actual effect, and the feature similarity between pixel points is calculated through the cosine similarity algorithm. The regional sensitivity of each pixel point is obtained by weighted summing the initial sensitivities between pixel points. For example, for a pixel point located at the edge of the original image data, some of the pixel points in its neighborhood belong to different semantic categories. Through bilateral filtering, combining the spatial distance and feature similarity, and integrating the initial sensitivities of the neighboring pixel points, a more reasonable regional sensitivity of this pixel point is obtained. Thus, the initial regional sensitivity is optimized using bilateral filtering, fully considering the spatial and feature relationships between pixel points, making the regional sensitivity more smoothly and accurately reflect the image characteristics, improving the quality of the regional sensitivity, providing a more reliable basis for accurately calculating the compensation parameter matrix and performing color compensation later. Accurate regional sensitivity is one of the important factors for calculating the compensation parameter matrix, jointly determining the compensation intensity with the deviation from the reference color and the light influence factor, directly affecting the accuracy of the compensation parameter matrix, and further affecting the accuracy and effect of color compensation.

[0051] The output steps of the compensation parameter matrix include: Convert the reference color and the target reference color to the perceptual hash space, and obtain the deviation of the reference color for each pixel point by calculating the Hamming distance between the perceptual hash values of the reference color and the target reference color. Construct a reinforcement learning, determine the state space as the deviation of the reference color, the light influence factor, and the regional sensitivity for each pixel point, the action space as the weight combination of the deviation of the reference color, the light influence factor, and the regional sensitivity for each pixel point, and continuously iteratively train the reinforcement learning. Calculate the compensation intensity for each pixel point according to the trained weight combination, and output the compensation parameter matrix in combination with the color correction parameters.

[0052] Traditional color space distance metrics cannot accurately reflect the human eye's perception of color differences. The perceptual hash space can convert colors into a perception-based hash value. By calculating the Hamming distance between hash values, it can more conform to the human eye's visual perception to measure the difference between the reference color and the target reference color, providing more reasonable color deviation data for determining the compensation intensity in the subsequent process. Through the perceptual hash algorithm, the reference color and the target reference color are converted from the RGB color space to the CIELAB color space to better correlate with the human eye's visual perception. For the colors in the CIELAB color space, they are represented as an image block, such as an 8×8 pixel block. Assuming that the representations of the reference color and the target reference color in the image can be approximated by such small blocks, the DCT transform is performed on this image block to obtain frequency domain coefficients. The low-frequency coefficients are retained because they contain the main structural information of the image and have a greater impact on color perception. Then, the low-frequency coefficients are quantized and binarized to generate a perceptual hash value. For example, the low-frequency coefficients are compared with a threshold, those greater than the threshold are set to 1, and those less than the threshold are set to 0, thus obtaining a binary hash value sequence. For each pixel point, the Hamming distance between the perceptual hash values of its reference color and the target reference color is calculated, that is, the number of different bits in the two hash values. For example, if the perceptual hash value of the reference color is "10101010" and the perceptual hash value of the target reference color is "10001010", then the Hamming distance is 2. Therefore, by calculating the reference color deviation through the perceptual hash space and the Hamming distance, it can more accurately reflect the human eye's perception of color differences, providing color deviation data that better meets the visual needs for color compensation and improving the visual effect of color compensation.

[0053] Different pixel points vary in terms of reference color deviation, light influence factor, and regional sensitivity. The traditional fixed-weight method cannot adaptively adjust according to the characteristics of each pixel point. Reinforcement learning can automatically find the optimal weight combination through continuous trial and error and learning based on the information in the state space to achieve the best color compensation effect. The Q-learning algorithm is used as the basis of reinforcement learning. The state space consists of the reference color deviation, light influence factor, and regional sensitivity of each pixel point. The action space consists of different weight combinations, corresponding to the weights of the reference color deviation, light influence factor, and regional sensitivity respectively. The reward function is defined as the similarity metric between the compensated image and the reference high-quality image, such as the structural similarity index. During the training process, starting from an initial state, an action (weight combination) is selected, and the compensation intensity is calculated according to the compensation intensity function where and are preset constants, represents the th pixel point's regional sensitivity, represents the th pixel point's reference color deviation, Indicates the light influence factor of the nth pixel point, compensates the reference color to obtain the compensated image, calculates the structural similarity index between the compensated image and the reference image as the reward, updates the Q-value table according to the Q-learning algorithm, continuously iterates this process, traverses a large number of pixel point states, makes the Q-value table converge, and finds the optimal action (weight combination) in each state; thus, automatically finds the optimal weight combination through reinforcement learning, can perform adaptive adjustment according to the unique characteristics of each pixel point, significantly improves the effect and accuracy of color compensation, and is more flexible and intelligent than the traditional fixed weight method.

[0054] According to the optimal weight combination for each pixel point obtained by reinforcement learning, combines with the compensation intensity function to calculate the compensation intensity, can achieve precise compensation for each pixel point, forms a compensation parameter matrix in combination with the color correction parameters, and provides a complete parameter basis for the subsequent compensation of the reference color; for each pixel point, substitutes the trained weight combination into the compensation intensity function to calculate the compensation intensity, and at the same time determines the correction parameters of each pixel point on the RGB color channels according to the requirements of color compensation, organizes the compensation intensity and color correction parameters into a compensation parameter matrix, and each row of the matrix corresponds to the compensation parameter information of a pixel point. Assuming the image has N pixel points, the compensation parameter matrix is an N×4 matrix; thus, realizes precise compensation according to the characteristics of each pixel point, the compensation parameter matrix provides comprehensive and accurate parameters for color compensation, can effectively improve the accuracy and effect of color compensation, makes the output image more in line with expectations in terms of color, the compensation parameter matrix is the direct basis for compensating the reference color, determines the compensation method and degree, plays a decisive role in the final output of a high-quality RGB image adapted to the target device, and at the same time affects the evaluation of the compensation effect and parameter update in the hierarchical re-compensation.

[0055] S4. Maps the compensated reference color to the target device color gamut, outputs an RGB image adapted to the target device, and predicts the color difference data of the sensitive area in the RGB image through a deep residual network, and performs hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0056] The compensated reference color is the color benchmark of the image after color compensation in the previous steps, and the target device color gamut is the color range that the target device can display. Mapping the compensated reference color to the target device color gamut is to adjust the color values obtained by compensation to the color range that the target device can display, so as to ensure that the output RGB image can be accurately displayed on the target device and avoid color distortion or exceeding the device display ability.

[0057] As Figure 3 shown, the execution steps of the hierarchical re-compensation include: The RGB image is stratified based on the visual saliency and image complexity of the RGB image, and the compensation priority of each layer is set; The color difference data of sensitive regions in the RGB image is predicted through a deep residual network, the average color difference of each layer is calculated according to the predicted color difference data, and the trigger threshold of each layer is set. The average color difference of each layer is compared with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer; For the stratified layer where re-compensation is triggered, the weight combination is adjusted to re-output the compensation parameter matrix, and the sampling frequency is updated according to the execution effect of the re-compensation.

[0058] The visual saliency and image complexity of different regions in the RGB image are different. Through stratification, different compensation strategies can be adopted for regions with different characteristics, giving priority to compensating regions with high visual saliency and high image complexity, improving the overall color compensation effect, while improving the calculation efficiency and avoiding resource waste caused by treating all regions equally; The RGB image is converted to the Lab color space, the luminance channel is Gaussian blurred to obtain the background image, the visual saliency of the background image is obtained according to the aforementioned method, and then the image complexity of the RGB image is obtained according to the aforementioned method. The image is divided into four layers according to the visual saliency and image complexity: high visual saliency and high complexity region (high-saliency high-complexity region), such as the close-up of a person's face with a complex background, high visual saliency but low complexity region (high-saliency low-complexity region), such as a bright object in front of a simple background, low visual saliency but high complexity region (low-saliency high-complexity region), such as a background with complex texture but not eye-catching, low visual saliency and low complexity region (low-saliency low-complexity region), such as a large-area single-color background. The high-saliency high-complexity region layer is set as the highest priority, followed by the high-saliency low-complexity region layer, then the low-saliency high-complexity region layer, and the low-saliency low-complexity region layer has the lowest priority. For example, when performing re-compensation, the high-saliency high-complexity region layer is processed first to ensure the color accuracy of important and complex regions; thus, different regions of the RGB image can be processed targeted, improving the effect and efficiency of color compensation, highlighting the color quality of important regions, while reasonably allocating computing resources and avoiding over-compensating low-importance regions.

[0059] The deep residual network has powerful feature learning capabilities and can accurately predict the color difference data of sensitive regions. By calculating the average color difference of each layer and comparing it with the trigger threshold, it can determine which regions have color deviations beyond the acceptable range, thereby deciding whether re-compensation is needed, and achieving dynamic monitoring and optimization of the color compensation effect. A deep residual network with multiple residual blocks is constructed. A large number of RGB images containing different scenes and color deviations are collected as the training set, and at the same time, the corresponding accurate color images are collected as labels. During the training process, the RGB images are input into the deep residual network, and the deep residual network outputs the predicted color difference data. The mean squared error is used as the loss function, and the parameters of the deep residual network are updated through the backpropagation algorithm to minimize the error between the predicted color difference data and the true color difference data. For the RGB image to be processed, the color difference data of the sensitive regions in the RGB image are predicted through the trained deep residual network. For each layer, the average color difference of the pixel points in all sensitive regions within the layer is calculated.

[0060] Then, through experiments and the analysis of a large number of RGB images, different trigger thresholds are set for each layer. Since the high-visibility and high-complexity region layer is of high importance, the trigger threshold is set to 3, with the unit being the color difference unit in the CIELAB color space. The trigger threshold for the high-visibility and low-complexity region layer is set to 5, the trigger threshold for the low-visibility and high-complexity region layer is set to 7, and the trigger threshold for the low-visibility and low-complexity region layer is set to 10. The average color difference of each layer is compared with the corresponding trigger threshold. If the average color difference is greater than the trigger threshold, re-compensation for that layer is triggered. Thus, it can accurately monitor the color deviation of each layer of the RGB image, automatically determine whether re-compensation is needed, improve the accuracy and self-adaptability of color compensation, and ensure that the RGB image can achieve better color effects in different regions.

[0061] When re-compensation is triggered for a certain layer, it indicates that the current compensation effect has not reached the ideal state. Adjusting the weight combination can re-distribute the roles of the reference color deviation, light influence factor, and regional sensitivity in the calculation of the compensation intensity to better adapt to the characteristics of that layer and improve the compensation effect. Updating the sampling frequency according to the execution effect of re-compensation can dynamically adjust the data sampling frequency, increasing the sampling frequency in cases of large color changes to obtain more accurate data, and vice versa to reduce the sampling frequency to save resources. For the stratified layer where re-compensation is triggered, the Q-learning reinforcement learning algorithm is used to re-find the optimal weight combination. Using the reference color deviation, light influence factor, and regional sensitivity of the pixel points within that layer as the state space, different weight combinations as the action space, and the reward function is defined as the improvement value of the structural similarity index between the RGB image of that layer after compensation and the reference high-quality image. Through multiple iterative trainings, the weight combination that maximizes the reward is found.

[0062] According to the new weight combination, combined with the compensation intensity function, recalculate the compensation intensity of each pixel point in this layer, and combine color correction parameters, such as the correction values of the RGB channels, to generate a new compensation parameter matrix. If the quality of the RGB image of this layer is significantly improved after re-compensation, such as the structural similarity index is increased by more than 0.1, and the number of layers triggering re-compensation is large, such as more than half of the total number of layers, then increase the sampling frequency by 20%. If the image quality improvement after re-compensation is not obvious and the number of layers triggering re-compensation is small, then reduce the sampling frequency by 10%. For example, after re-compensation, both the high-display high-recovery area layer and the high-display low-recovery area layer have obvious quality improvement and trigger re-compensation. At this time, increase the sampling frequency from 10 Hz to 12 Hz. Thus, it is possible to dynamically adjust the compensation strategy according to the re-compensation requirements, improve the color compensation effect, and at the same time reasonably adjust the sampling frequency to balance the relationship between the resource consumption of data acquisition and processing and the color compensation quality. The updated compensation parameter matrix is used to compensate the reference color again to improve the image color quality. The updated sampling frequency affects subsequent data acquisition, providing a more suitable data basis for continuously optimizing color compensation.

[0063] Embodiment 2 The embodiment of the present application provides a color compensation system based on adaptive color adjustment, and the system includes: Obtain the original image data, ambient light data, and the color gamut of the target device through the sampling frequency, and align the time stamps of the original image data and the ambient light data; Identify the color clusters with a proportion greater than 15% in the original image data through a clustering algorithm as the main colors of the image, calculate the offset of the color temperature under the illumination intensity according to the ambient light data to determine the color difference data, determine the reference color based on the main colors of the image and the color difference data, and map the ambient light data into the original image data to output the light influence factor of each pixel point in the original image data; Divide the sensitive areas in the original image data through a semantic segmentation model to output the regional sensitivity of each pixel point in each sensitive area. At the same time, compare the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and regional sensitivity of each pixel point, combine the compensation intensity of each pixel point to output a compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix; Map the compensated reference color to the color gamut of the target device to output an RGB image adapted to the target device, and predict the color difference data of the sensitive areas in the RGB image through a deep residual network, and perform hierarchical re-compensation according to the change situation of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0064] For the solution of the above system, refer to the content of the embodiment of the method, which will not be elaborated here one by one.

Claims

1. A color compensation method based on adaptive color matching, characterized in that: include: Acquire the original image data, ambient light data and the color configuration of the target device through the sampling frequency, and align the timestamps of the original image data and the ambient light data, wherein the ambient light data includes the color temperature and the light intensity, and the color configuration of the target device includes the color gamut of the target device; A clustering algorithm is used to identify color clusters that account for more than 15% of the original image data as the main color tone of the image. The offset of the color temperature under the light intensity is calculated according to the ambient light data, and the color difference data is determined. The reference color is determined based on the main color tone and the color difference data of the image. The ambient light data is mapped to the original image data to match the image resolution in the original image data, and the light impact factor of each pixel in the original image data is output. The semantic segmentation model is used to divide the sensitive areas in the original image data, and the regional sensitivity of each pixel in each sensitive area is output. At the same time, the reference color and the target reference color are compared to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor and regional sensitivity of each pixel, the compensation parameter matrix is ​​output in combination with the compensation intensity of each pixel, and the reference color is compensated according to the compensation parameter matrix. The compensated reference color is mapped to the color gamut of the target device, and an RGB image adapted to the target device is output. The color difference data of the sensitive areas in the RGB image is predicted through a deep residual network. According to the changes in the color difference data, hierarchical re-compensation is performed to update the compensation parameter matrix and sampling frequency.

2. The color compensation method based on adaptive color matching according to claim 1, characterized in that: The step of determining the color difference data comprises: The historical ambient light data is trained through a recurrent neural network to predict the target color temperature under the light intensity; Calculate the deviation between the actual color temperature and the target color temperature to obtain the color temperature offset under the light intensity; The light intensity influence coefficient is set according to the light intensity, and the color difference data is determined according to the light intensity influence coefficient and the offset of the color temperature under the light intensity.

3. The color compensation method based on adaptive color matching according to claim 2, characterized in that: The step of determining the reference color comprises: According to the proportion of the main hue of each image in the original image data and the visual significance of the main hue of each image, the hue weight of the main hue of each image is calculated, and the cluster center color of the main hue of each image is determined; Calculate the standard deviation of ambient light data and historical ambient light data to determine the ambient light weight and set the calibration color; The hue weight and cluster center color as well as the ambient light weight and calibration color are weighted and summed to determine the reference color.

4. The color compensation method based on adaptive color matching according to claim 3, characterized in that: The step of outputting the light impact factor of each pixel point comprises: The ambient light data is mapped to the pixels in the original image data through a ray tracing algorithm, and for each pixel, the light propagation path from the ambient light source to the pixel is output; Determine whether there is an obstruction on each light propagation path to determine the occlusion coefficient of each pixel; Determine the influence of light intensity and color temperature in ambient light data on each pixel, and obtain the influence coefficient of each pixel; Combine the occlusion coefficient and influence coefficient of each pixel to output the light influence factor of each pixel.

5. The color compensation method based on adaptive color matching according to claim 4, characterized in that: The step of outputting the regional sensitivity of each pixel point comprises: The semantic segmentation model is used to divide the sensitive areas in the original image data, and the attention mechanism is used to determine the semantic category of each pixel in each sensitive area; Extract the modal feature vector of the convolutional neural network combined with the long short-term memory network for each pixel, and set the initial sensitivity for each semantic category based on the modal feature vector to obtain the initial regional sensitivity of each pixel; The spatial distance and feature similarity between pixels are determined based on bilateral filtering, and the regional sensitivity of each pixel is output according to the spatial distance and feature similarity between pixels.

6. The color compensation method based on adaptive color matching according to claim 5, characterized in that: The output step of the compensation parameter matrix comprises: The reference color and the target reference color are converted into the perceptual hash space, and the reference color deviation of each pixel is obtained by calculating the Hamming distance between the perceptual hash values ​​of the reference color and the target reference color; Construct reinforcement learning, determine the state space as the base color deviation, light impact factor and regional sensitivity of each pixel, the action space as the weighted combination of the base color deviation, light impact factor and regional sensitivity of each pixel, and continuously iterate and train reinforcement learning; The compensation intensity of each pixel is calculated according to the weight combination obtained through training, and the compensation parameter matrix is ​​output in combination with the color correction parameters.

7. The color compensation method based on adaptive color matching according to claim 6, characterized in that: The step of determining the sampling frequency comprises: Determine the load factor of the target device based on the CPU computing power and memory bandwidth of the target device, set a baseline sampling rate based on the type of the target device, and determine the sampling rate based on the load factor of the target device; Based on the change gradient of color temperature and light intensity in historical ambient light data, the change rate of ambient light is calculated, and the sampling rate is corrected according to the change rate of ambient light; The image complexity is determined according to the clustering discreteness of the main color tone of the image and the edge density of the original image data, and the sampling rate is optimized according to the image complexity to obtain the final determined sampling frequency.

8. The color compensation method based on adaptive color matching according to claim 7, characterized in that: The steps of executing the layered recompensation include: Layer the RGB image based on its visual saliency and image complexity, and set the compensation priority of each layer; The color difference data of sensitive areas in the RGB image are predicted through a deep residual network. The average color difference of each layer is calculated based on the predicted color difference data, and the trigger threshold of each layer is set. The average color difference of each layer is compared with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer. For the layer that triggers re-compensation, the weight combination is adjusted to re-output the compensation parameter matrix, and the sampling frequency is updated according to the execution effect of the re-compensation.

9. A color compensation system based on adaptive color matching, characterized in that: include: Obtain the original image data, ambient light data, and target device color gamut through sampling frequency, and align the timestamps of the original image data and ambient light data; A clustering algorithm is used to identify color clusters that account for more than 15% of the original image data as the main color tone of the image. The offset of the color temperature under the light intensity is calculated according to the ambient light data, and the color difference data is determined. The reference color is determined based on the main color tone and color difference data of the image, and the ambient light data is mapped to the original image data. The light impact factor of each pixel in the original image data is output; The semantic segmentation model is used to divide the sensitive areas in the original image data, and the regional sensitivity of each pixel in each sensitive area is output. At the same time, the reference color and the target reference color are compared to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor and regional sensitivity of each pixel, the compensation parameter matrix is ​​output in combination with the compensation intensity of each pixel, and the reference color is compensated according to the compensation parameter matrix. The compensated reference color is mapped to the color gamut of the target device, and an RGB image adapted to the target device is output. The color difference data of the sensitive areas in the RGB image is predicted through a deep residual network. According to the changes in the color difference data, hierarchical re-compensation is performed to update the compensation parameter matrix and sampling frequency.

Images