Color calibration method, device and storage medium

Abstract

The present disclosure relates to a color calibration method, device and storage medium. The method comprises: obtaining the position information and initial color corresponding to the pixel to be calibrated in the image to be displayed on the screen, the position information comprising the position of the camera when shooting the screen, the coordinates of the pixel to be calibrated on the screen and the number of the screen; based on the position, coordinates and screen number of the camera corresponding to the pixel to be calibrated, adjusting the initial color of the pixel to be calibrated to determine the target color. According to the embodiment of the present application, the color calibration method of the present application can be made unaffected by different system coordinate systems, and can solve the problem of decreased color calibration effect caused by changes in camera position, and can solve the problem of significant decrease in color calibration accuracy in different areas of the screen under the same viewport, maintain consistency of color calibration effects in different areas, and improve color calibration accuracy. The above-mentioned color calibration process is not constrained by screens of different shapes and positions, meets the needs of different scenes to the greatest extent, and is more versatile.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence (AI) technology, and in particular to a color correction method, device, and storage medium. Background Art

[0002] With the development of AI technology and the video production industry, virtual filming, as an emerging filming technique, has been gaining increasing attention and adoption in recent years. It has broad application prospects in a variety of scenarios, including filming of movies, TV series, and variety shows, product launches, and live video broadcasts. Virtual filming allows actors to be easily placed within virtual scenes, making them appear as if they were actually in the real scene. This technology allows for the capture of scenes previously impossible with traditional methods. Compared to traditional green screens, cameras can capture the final image directly, helping to improve filming and production efficiency and save costs.

[0003] In virtual filming scenarios, current color calibration solutions typically only work when the camera moves within a small range, centered around the calibration position. However, when the camera moves over a large range, the accuracy and effectiveness of the calibration decrease significantly, and calibration may even fail. Furthermore, on large screens, colors may appear inconsistent between nearby and distant areas. Therefore, a new color calibration method is urgently needed to improve image color calibration and ensure consistency across different areas. Summary of the Invention

[0004] In view of this, the present disclosure proposes a color calibration method, device, and storage medium.

[0005] According to one aspect of the present disclosure, a color calibration method is provided. The method includes:

[0006] Obtain the position information and initial color of the pixel to be calibrated in the image to be displayed on the screen. The position information includes the position of the camera when shooting the screen, the coordinates of the pixel to be calibrated on the screen, and the screen number.

[0007] Based on the position and coordinates of the camera and the screen number corresponding to the pixel to be calibrated, the initial color of the pixel to be calibrated is adjusted to determine the target color.

[0008] The coordinates may be UV coordinates.

[0009] In one possible implementation, based on the position and coordinates of the camera corresponding to the pixel to be calibrated and the screen number, the initial color of the pixel to be calibrated is adjusted to determine the target color, including:

[0010] Using the color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position, coordinates and initial color of the camera corresponding to the pixel to be calibrated to determine the target color;

[0011] The LUT tree is determined based on the inverse LUT and position information corresponding to one or more sampled pixels on the screen. The inverse LUT is used to map the initial color of the pixel to be calibrated to the target color. The sampled pixels can be arranged at equal distances on the screen at predetermined coordinate intervals.

[0012] In a possible implementation, the reverse LUTs corresponding to the sampled pixel points captured at the same camera position are under the same first child node of the LUT tree;

[0013] Under the same first subnode, the reverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than the predetermined threshold is under the same second subnode under the same first subnode;

[0014] Each third sub-node under the same second sub-node corresponds to an inverse LUT.

[0015] In one possible implementation, the first child node value of the LUT tree is the position coordinate of the camera, the second child node value is the average of the first coordinate axis coordinate values of each sampling pixel point under the same second child node, and the third child node value is the second coordinate axis coordinate value of the corresponding sampling pixel point.

[0016] In a possible implementation, the position information corresponding to the sampling pixel point includes positions of multiple cameras corresponding to the sampling pixel point.

[0017] In one possible implementation, a color lookup table (LUT) tree corresponding to the screen number of the pixel to be calibrated is used to interpolate based on the position, coordinates, and initial color of the camera corresponding to the pixel to be calibrated to determine the target color, including:

[0018] Based on the position of the camera corresponding to the pixel to be calibrated, using the first child node in the LUT tree, determine the first interpolation weight of the pixel to be calibrated;

[0019] Based on the first coordinate axis coordinate value of the pixel point to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determine the second interpolation weight of the pixel point to be calibrated;

[0020] Based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determine the third interpolation weight of the pixel to be calibrated;

[0021] A target color is determined based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node.

[0022] In one possible implementation, determining a target color based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node includes:

[0023] determining a plurality of first intermediate colors based on the initial colors and the inverse LUT corresponding to the corresponding third child nodes;

[0024] weighting the first intermediate color based on a third interpolation weight to determine a second intermediate color;

[0025] weighting the second intermediate color based on the second interpolation weight to determine a third intermediate color;

[0026] The third intermediate color is weighted based on the first interpolation weight to determine a target color.

[0027] In one possible implementation, based on the position of the camera corresponding to the pixel to be calibrated, a first interpolation weight of the pixel to be calibrated is determined using the first child node in the LUT tree, including:

[0028] Based on the position of the camera corresponding to the pixel to be calibrated, determine one or two first child nodes adjacent to the position of the camera in the LUT tree;

[0029] Based on the position of the camera and one or two first child node values, determine the interpolation weights of each coordinate axis of the world coordinate system respectively;

[0030] Determine weights for each coordinate axis of the world coordinate system based on one or two first child node values;

[0031] A first interpolation weight is determined based on the interpolation weights and the weighted weights under each coordinate axis.

[0032] In one possible implementation, based on the first coordinate axis coordinate value of the pixel to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determining the second interpolation weight of the pixel to be calibrated includes:

[0033] Based on the coordinate value of the first coordinate axis, for one or two first child nodes, determine one or two second child nodes adjacent to the coordinate value of the first coordinate axis under each first child node;

[0034] For one or two first child nodes, a second interpolation weight is determined based on the first coordinate axis coordinate value and one or two second child node values.

[0035] In one possible implementation, based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determining the third interpolation weight of the pixel to be calibrated includes:

[0036] Based on the coordinate value of the second coordinate axis, for one or two second subnodes adjacent to the coordinate value of the first coordinate axis under each first subnode, determine one or two third subnodes adjacent to the coordinate value of the second coordinate axis under each second subnode;

[0037] For one or two second child nodes, a third interpolation weight is determined based on the second coordinate axis coordinate value and one or two third child node values.

[0038] In a possible implementation, the area where the sampling pixel points are set may also include one or more of the following: the intersection area between screens, the area with surface contamination, and the area with quality control problems.

[0039] According to another aspect of the present disclosure, a color calibration device is provided. The device includes:

[0040] An acquisition module is used to obtain the position information and initial color of the pixel to be calibrated in the image to be displayed on the screen. The position information includes the position of the camera when shooting the screen, the coordinates of the pixel to be calibrated on the screen, and the screen number;

[0041] The adjustment module is used to adjust the initial color of the pixel to be calibrated based on the position, coordinates of the camera and the screen number corresponding to the pixel to be calibrated to determine the target color.

[0042] The coordinates may be UV coordinates.

[0043] In a possible implementation, the adjustment module is configured to:

[0044] Using the color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position, coordinates and initial color of the camera corresponding to the pixel to be calibrated to determine the target color;

[0045] The LUT tree is determined based on the inverse LUT and position information corresponding to one or more sampled pixels on the screen. The inverse LUT is used to map the initial color of the pixel to be calibrated to the target color. The sampled pixels can be arranged at equal distances on the screen at predetermined coordinate intervals.

[0046] In a possible implementation, the reverse LUTs corresponding to the sampled pixel points captured at the same camera position are under the same first child node of the LUT tree;

[0047] Under the same first subnode, the reverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than the predetermined threshold is under the same second subnode under the same first subnode;

[0048] Each third sub-node under the same second sub-node corresponds to an inverse LUT.

[0049] In one possible implementation, the first child node value of the LUT tree is the position coordinate of the camera, the second child node value is the average of the first coordinate axis coordinate values of each sampling pixel point under the same second child node, and the third child node value is the second coordinate axis coordinate value of the corresponding sampling pixel point.

[0050] In a possible implementation, the position information corresponding to the sampling pixel point includes positions of multiple cameras corresponding to the sampling pixel point.

[0051] In one possible implementation, a color lookup table (LUT) tree corresponding to the screen number of the pixel to be calibrated is used to interpolate based on the position, coordinates, and initial color of the camera corresponding to the pixel to be calibrated to determine the target color, including:

[0052] Based on the position of the camera corresponding to the pixel to be calibrated, using the first child node in the LUT tree, determine the first interpolation weight of the pixel to be calibrated;

[0053] Based on the first coordinate axis coordinate value of the pixel point to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determine the second interpolation weight of the pixel point to be calibrated;

[0054] Based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determine the third interpolation weight of the pixel to be calibrated;

[0055] A target color is determined based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node.

[0056] In one possible implementation, determining a target color based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node includes:

[0057] determining a plurality of first intermediate colors based on the initial colors and the inverse LUT corresponding to the corresponding third child nodes;

[0058] weighting the first intermediate color based on a third interpolation weight to determine a second intermediate color;

[0059] weighting the second intermediate color based on the second interpolation weight to determine a third intermediate color;

[0060] The third intermediate color is weighted based on the first interpolation weight to determine a target color.

[0061] In one possible implementation, based on the position of the camera corresponding to the pixel to be calibrated, a first interpolation weight of the pixel to be calibrated is determined using the first child node in the LUT tree, including:

[0062] Based on the position of the camera corresponding to the pixel to be calibrated, determine one or two first child nodes adjacent to the position of the camera in the LUT tree;

[0063] Based on the position of the camera and one or two first child node values, determine the interpolation weights of each coordinate axis of the world coordinate system respectively;

[0064] Determine weights for each coordinate axis of the world coordinate system based on one or two first child node values;

[0065] A first interpolation weight is determined based on the interpolation weights and the weighted weights under each coordinate axis.

[0066] In one possible implementation, based on the first coordinate axis coordinate value of the pixel to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determining the second interpolation weight of the pixel to be calibrated includes:

[0067] Based on the coordinate value of the first coordinate axis, for one or two first child nodes, determine one or two second child nodes adjacent to the coordinate value of the first coordinate axis under each first child node;

[0068] For one or two first child nodes, a second interpolation weight is determined based on the first coordinate axis coordinate value and one or two second child node values.

[0069] In one possible implementation, based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determining the third interpolation weight of the pixel to be calibrated includes:

[0070] Based on the coordinate value of the second coordinate axis, for one or two second subnodes adjacent to the coordinate value of the first coordinate axis under each first subnode, determine one or two third subnodes adjacent to the coordinate value of the second coordinate axis under each second subnode;

[0071] For one or two second child nodes, a third interpolation weight is determined based on the second coordinate axis coordinate value and one or two third child node values.

[0072] In a possible implementation, the area where the sampling pixel points are set may also include one or more of the following: the intersection area between screens, the area with surface contamination, and the area with quality control problems.

[0073] According to another aspect of the present disclosure, a color correction device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0074] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0075] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0076] According to an embodiment of the present application, by calibrating the color of the pixels to be calibrated in the image displayed on the screen based on the position of the camera when shooting the screen, the problem of reduced color calibration effect caused by changes in the camera position can be solved, and the accuracy of color calibration can be improved. By calibrating the pixels to be calibrated based on the coordinates of the pixels to be calibrated on the screen, the color calibration method of the present application is not affected by different system coordinate systems, and can solve the problem of significant reduction in color calibration accuracy in different areas of the screen under the same viewport (i.e., the same camera position), thereby maintaining the consistency of color calibration effects in different areas and improving color calibration accuracy. At the same time, by calibrating the pixels to be calibrated based on the above coordinates, the color calibration process can also be unconstrained by screens of different shapes and positions, and there is no requirement for the screen to be in a standing or lying position, making the solution universal for screens of various shapes and postures, meeting the needs of different scenes to the greatest extent, and having greater versatility. By calibrating based on the screen number and determining the target color (i.e., the color after calibration), the differences in luminous characteristics between different screens can be taken into account, thereby improving the overall color calibration effect.

[0077] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0079] Figure 1 Schematic diagram showing the introduction of chromatic aberration in a virtual scene.

[0080] Figure 2A schematic diagram showing system color calibration in a virtual scene.

[0081] Figure 3 A schematic diagram illustrating an application scenario according to an embodiment of the present application.

[0082] Figure 4 A schematic diagram illustrating an application scenario according to an embodiment of the present application.

[0083] Figure 5 A flowchart of a color calibration method according to an embodiment of the present application is shown.

[0084] Figure 6 Schematic diagram showing different coordinate systems.

[0085] Figure 7 A schematic diagram showing screens of different shapes according to an embodiment of the present application.

[0086] Figure 8 A schematic diagram showing screens in different positions according to an embodiment of the present application.

[0087] Figure 9 A schematic diagram of a UV coordinate system according to an embodiment of the present application is shown.

[0088] Figure 10 A schematic diagram showing UV coordinates of pixel points in an LED screen according to an embodiment of the present application is shown.

[0089] Figure 11 A schematic diagram illustrating color acquisition for sampling pixel points according to an embodiment of the present application is shown.

[0090] Figure 12 A schematic diagram of a LUT tree according to an embodiment of the present application is shown.

[0091] Figure 13 A flowchart of a color calibration method according to an embodiment of the present application is shown.

[0092] Figure 14 FIG. 4 shows a structural diagram of a color calibration device according to an embodiment of the present application.

[0093] Figure 15 is a block diagram showing a device 800 for color calibration according to an exemplary embodiment.

[0094] Figure 16 is a block diagram showing a device 1900 for color calibration according to an exemplary embodiment. DETAILED DESCRIPTION

[0095] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0096] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0097] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0098] With the development of AI technology and the video shooting industry, virtual shooting, as an emerging shooting technology, has received increasing attention and application in the industry in recent years. It has broad application prospects in various scenarios such as filming of movies, TV series, product launches, and live video broadcasts. Through virtual shooting, actors can be easily placed in the virtual scene, making them feel as if they are in the real scene. This technology can capture some scenes that were previously impossible to capture with traditional methods. Compared with the original green screen, the camera can directly capture the final image. It helps to improve the efficiency of shooting and production and save costs. Figure 1 , which shows a schematic diagram of introducing color difference into a virtual scene. Figure 1 As shown, the dotted frame portion may indicate the cause of the introduction of chromatic aberration. In a virtual shooting scene, for example, the image of the virtual scene may be displayed in real time on the LED wall through the broadcast control software (e.g. Figure 1 However, due to the influence of broadcast control software, LED wall, camera, ambient light, etc. on the color during the camera shooting process, when the camera shoots the actor and the background image on the LED wall together, the image obtained (such as Figure 1 There will be color differences between the output image (in the image) and the input image, resulting in differences between the captured color and the scene material. Therefore, to calibrate the color of the output image, a black box model is usually used. The influence of the dotted box is used as the transformation function f(x). By offsetting f(x), the image color is calibrated.

[0099] Here, f(x) is usually offset by a transformation g(x), which is the inverse color lookup table (LUT). Figure 2 , which shows a schematic diagram of system color calibration in a virtual scene. Figure 2As shown, in an XR or internal view virtual filming scene, the original image (i.e., the background image to be broadcast, controlled, and displayed on screen, and used for filming) can be obtained, and a reverse LUT can be generated to process the original image to obtain a processed image. This processed image can be used as the real-time display of the virtual scene on the LED wall for broadcast, controlled, and filming. When the camera captures the actor and the processed image on the LED wall, the resulting image eliminates the color effects of the aforementioned factors. In other words, the final image obtained by capturing the processed image eliminates color differences compared to the image obtained by capturing the original image.

[0100] However, in the above virtual shooting scene, the current color calibration solution is usually only applicable to the situation where the camera moves within a small range centered on the color calibration position. When the camera moves over a large range, the accuracy and effect of color calibration will be greatly reduced, and color calibration may even fail. Figure 3 , which shows a schematic diagram of an application scenario according to an embodiment of the present application. Figure 3 As shown, the camera may correspond to different camera positions (camera positions A, B, and C as shown in the figure) during the mobile shooting process. If a reverse LUT is generated by collecting the color of the pixels on the LED screen at camera position A (assuming that camera position A is the color calibration camera for color calibration), the effect of color calibration near camera position A using the reverse LUT may be as expected. However, when the camera moves to camera positions B and C, if the reverse LUT is still used for color calibration, the color calibration effect will deteriorate.

[0101] Also, colors may appear inconsistent between near and far areas on a large screen. Figure 4 , which shows a schematic diagram of an application scenario according to an embodiment of the present application. Figure 4 As shown in the figure, even if the camera is shooting at the same camera position B, the colors of pixel 1 and pixel 2 in different areas of the LED screen are inconsistent (assuming the LED screen displays a solid color image). Therefore, using the generated inverse LUT to calibrate the colors of these two pixels will result in inconsistent color calibration results.

[0102] The above problems are all caused by the LED screen's changing light emission angle, which leads to unstable luminous performance. Therefore, a new color calibration method is urgently needed to improve the color calibration effect of the image and ensure consistency of the color calibration effect in different areas.

[0103] In view of this, the present application proposes a color calibration method. The color calibration method of the embodiment of the present application calibrates the color of the pixel to be calibrated in the image displayed on the screen based on the position of the camera when shooting the screen. This can solve the problem of reduced color calibration effect caused by changes in the camera position and improve the accuracy of color calibration. Since the coordinate system of the screen in the actual scene is not fixed, by calibrating the pixel to be calibrated based on the coordinates of the pixel to be calibrated on the screen, the color calibration method of the present application is not affected by different system coordinate systems, and can solve the problem of significant reduction in color calibration accuracy in different areas of the screen under the same viewport, thereby maintaining the consistency of color calibration effects in different areas and improving color calibration accuracy. At the same time, by calibrating the pixel to be calibrated based on the above-mentioned coordinates, the color calibration process can also be unconstrained by screens of different shapes and positions, and there is no requirement for the screen to be in a standing or lying position. This makes the solution universal for screens of various shapes and postures, meeting the needs of different scenes to the greatest extent, and is more universal. By performing color calibration based on the screen number and determining the target color (i.e. the color after calibration), the differences in luminous characteristics between different screens can be taken into account, thereby improving the overall color calibration effect.

[0104] The color calibration method of the present application can be applied to a terminal device or a server. The terminal device can be any one or more of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), or an in-vehicle device. The embodiments of the present application do not impose any particular restrictions on the specific type of terminal device, and the terminal device can have wired or wireless communication capabilities.

[0105] The server can be located locally or in the cloud, and can be a physical device or a virtual device, such as a virtual machine or container, and has wireless communication capabilities, wherein the wireless communication capabilities can be set in the chip (system) or other parts or components of the server. The wireless communication function can be implemented, for example, through mobile communication technologies such as 2G / 3G / 4G / 5G, as well as Wi-Fi, Bluetooth, frequency modulation (FM), digital radio, satellite communication, etc. Communication can also be carried out through wired connections to achieve interaction with other devices.

[0106] The color correction method of the embodiment of the present application can also be applied to other scenarios where image color correction may be required, and the present application does not limit this.

[0107] The following Figure 5-Figure 13 , the color calibration method of the embodiment of the present application is introduced.

[0108] Figure 5 A flow chart of a color calibration method according to an embodiment of the present application is shown. The method can be used in a terminal device or a server, such as Figure 5 As shown, the method may include:

[0109] Step S501: Obtain the position information and initial color corresponding to the pixel to be calibrated in the image to be displayed on the screen.

[0110] The position information may include the camera's position when capturing the screen, the coordinates of the pixel to be calibrated on the screen, and the screen ID. The initial color of the pixel to be calibrated may refer to the RGB color of the pixel to be calibrated in the pre-calibration image. This color is the color of the pixel to be displayed on the screen and has not yet been calibrated.

[0111] The position of the camera is the position of the camera when shooting the screen. The position can be the spatial coordinate position of the virtual camera. The spatial coordinate can be set to any coordinate system form according to the different calibration equipment. Figure 6 , showing schematic diagrams of different coordinate systems. Figure 6 As shown, it can be a left-handed coordinate system or a right-handed coordinate system, or a rotated coordinate system obtained by arbitrarily transforming the left-handed / right-handed coordinate system. The camera position can also be the world coordinate system position of the physical camera. This application does not impose any restrictions.

[0112] The number of the screen can be used to determine a corresponding screen from at least one screen. These screens (such as the above-mentioned LED screens) can be screens of any shape, see Figure 7 , showing a schematic diagram of screens of different shapes according to an embodiment of the present application. Figure 7 As shown, the LED screen can be a straight screen, a curved screen, etc., and this application does not limit this.

[0113] These screens can also be placed anywhere in the physical space, see Figure 8 , showing a schematic diagram of screens at different positions according to an embodiment of the present application. Figure 8 As shown, the LED screen can be a ceiling screen (placed above the shooting screen), a vertical screen (placed at any position of the shooting screen), a floor screen (placed below the shooting screen), etc., and this application does not impose any restrictions on this.

[0114] The coordinates of the pixel to be calibrated on the screen may be the UV coordinates of the pixel to be calibrated on the screen corresponding to the number. The UV coordinates may represent the texture coordinates of the 3D model surface corresponding to the LED screen. Figure 9 , shows a schematic diagram of a UV coordinate system according to an embodiment of the present application. Figure 9 The LED plane shown can represent the Figure 7The curved screen shown in the figure is expanded in a plane, where the UV coordinate system shown in the figure can represent the texture coordinate system of the 3D model surface corresponding to the curved screen. Any vertex of the LED plane can be used as the origin of the coordinate system (the upper left vertex is used as the origin in the figure). The value range of the U axis and the V axis can be 0 to 1. The position of any pixel in the image displayed on the LED screen can be determined by the UV coordinates. The coordinate system and the expression of the coordinates can be selected as needed. By making the coordinates of the pixel to be calibrated on the screen UV coordinates, color calibration can be performed more conveniently, making the solution more universal.

[0115] For example, if you want to calibrate a pixel to be calibrated in an image to be displayed, and then display it on the screen after calibration, you can get a target color for each combination of the screen where the pixel is to be displayed, the position of the camera that is to shoot the screen, and the coordinates of the pixel on the screen. For example, for the same screen and the same coordinates, you can get target color 1 for camera position 1 and target color 2 for camera position 2, so that target color 1 is displayed when shooting at position 1, and target color 2 is displayed when shooting at position 2. Or for the same camera position, pixel 1 to be displayed on screen 1 corresponds to target color 3, and pixel 2 to be displayed on screen 2 corresponds to target color 4, so as to achieve the purpose of more accurate color calibration. The above target colors 1-4 are colors adjusted using a reverse LUT. The target colors can be used as real-time display colors on LED walls, for broadcast control and shooting, so that the color captured by the camera is the color of the original material, that is Figure 1 The colors in the input image in .

[0116] Step S502 : Based on the position and coordinates of the camera and the screen number corresponding to the pixel to be calibrated, the initial color of the pixel to be calibrated is adjusted to determine the target color.

[0117] Taking the UV coordinate system as an example, since different screens can correspond to different 3D models, different screens have independent UV coordinate systems. Therefore, the corresponding screen can be determined according to the screen number, and the position of the pixel to be calibrated on the image can be determined according to the UV coordinates in the UV coordinate system corresponding to the screen. This facilitates color calibration of the pixel to be calibrated. Figure 10 , shows a schematic diagram of the UV coordinates of pixels in an LED screen according to an embodiment of the present application. Figure 10As shown, the ceiling screen, vertical screen, and floor screen can be represented by three different 3D LED models, and therefore correspond to three different UV coordinate systems. Pixels 1 and 2 on the vertical screen are in the same UV coordinate system but have different UV coordinates. Pixel 3 on the ceiling screen and pixel 4 on the floor screen are in different UV coordinate systems. During color calibration, each of the three screens will be independently calibrated. The following describes one method for implementing S502 in detail.

[0118] According to an embodiment of the present application, by calibrating the color of the pixels to be calibrated in the image displayed on the screen based on the position of the camera when shooting the screen, the problem of reduced color calibration effect caused by changes in the camera position can be solved, and the accuracy of color calibration can be improved. By calibrating the pixels to be calibrated based on the coordinates of the pixels to be calibrated on the screen, the color calibration method of the present application is not affected by different system coordinate systems, and can solve the problem of significant reduction in color calibration accuracy in different areas of the screen under the same viewport (i.e., the same camera position), thereby maintaining the consistency of color calibration effects in different areas and improving color calibration accuracy. At the same time, by calibrating the pixels to be calibrated based on the above coordinates, the color calibration process can also be unconstrained by screens of different shapes and positions, and there is no requirement for the screen to be in a standing or lying position, making the solution universal for screens of various shapes and postures, meeting the needs of different scenes to the greatest extent, and having greater versatility. By calibrating based on the screen number and determining the target color (i.e., the color after calibration), the differences in luminous characteristics between different screens can be taken into account, thereby improving the overall color calibration effect.

[0119] Optionally, step S502 may include:

[0120] Using the LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position, coordinates and initial color of the camera corresponding to the pixel to be calibrated to determine the target color.

[0121] The LUT tree can be determined based on the reverse LUT and position information corresponding to one or more sampled pixels on the corresponding screen. Figure 2 , used to map the initial color of the pixel to be calibrated to the target color. The sampling pixels can be set at equal distances on the screen according to predetermined coordinate intervals.

[0122] The LUT tree can be created in advance before the pixels to be calibrated are calibrated. In the process of establishing the LUT tree, the positions of the sampling pixels can be determined in advance, and the LUT tree can be established based on the sampling pixels.

[0123] Among them, in order to solve the problem of reduced color correction effect caused by large-scale camera movement, pixel points under different camera positions can be sampled (they can be the same or different pixel points under different camera positions), so that the position information corresponding to the sampled pixel points includes the positions of multiple cameras corresponding to the sampled pixel points, and the sampling pixel points corresponding to different camera positions can be different.

[0124] In order to solve the problem of color inconsistency between near and far areas on large screens, sampling pixels can be set in different areas of the screen. Since the colors of the intersection areas between LED screens will be inconsistent due to mutual illumination, and some accidental factors such as contamination on the surface of some LED screen splicing small units and quality control differences between different production batches may also cause color difference problems, in order to more specifically alleviate the above problems, the area where the sampling pixels are set can be selected so that the area where the sampling pixels are set includes one or more of the intersection areas between screens, areas with surface contamination, and areas with quality control problems.

[0125] One or more sampling pixels can be determined on different screens. The sampling pixels on the same screen can correspond to multiple camera positions, and the sampling pixels set for different camera positions can be different. After the sampling pixels are determined, a corresponding reverse LUT can be determined for each sampling pixel.

[0126] See also Figure 11 , shows a schematic diagram of color acquisition for sampling pixel points according to an embodiment of the present application. Figure 11 As shown in the figure, p1_1, p1_2, p1_3 and p1_4 are the sampling pixels on the vertical screen, p2_1 is the sampling pixel on the ground screen, and p3_1 and p3_2 are the sampling pixels on the sky screen. You can first fix the camera shooting position (such as camera position A, position (x1, y1, z1)), display the color card X (R, G, B) on each screen at the same time, and switch different colors in sequence. If you want to collect n-level LUT, you can display n 3 When displaying each color, for the vertical screen, the colors of p1_1, p1_2, p1_3 and p1_4 captured by the camera of the sampling pixel point can be recorded to determine 4 LUTs (i.e. Figure 1 f(x)), the sky screen and the ground screen can also be recorded simultaneously, so that the LUT of each screen can be obtained at camera position A. Based on the existing technology, the reverse LUT corresponding to the LUT on each screen can be obtained (i.e. Figure 2 's g(x)).

[0127] The camera can then be moved to the next camera position (such as camera position B, position (x2, y2, z2), or camera position C, position (x3, y3, z3)), and the above steps can be repeated. The camera position during the sampling process can be set within the commonly required shooting range. Ultimately, a color calibration method for a sampled pixel can be uniquely identified by the combination of three positional information: camera position, sampled pixel coordinates, and screen number. For each color calibration method for each sampled pixel, a corresponding reverse LUT can be obtained.

[0128] Therefore, a LUT tree may be created based on the inverse LUT and position information corresponding to one or more sampling pixel points on the corresponding screen. The process of creating the LUT tree is introduced below.

[0129] First, based on the position information of the sampled pixels, the sampled pixels under the same screen number can be extracted, and different LUT trees can be created for different LED screens, so that one LUT tree is created for each screen. The root node on the first layer of the LUT tree can correspond to the screen number.

[0130] For each LUT tree, the inverse LUT corresponding to the sampling pixel points shot at the same camera position is under the same first child node of the LUT tree; under the same first child node, the inverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than a predetermined threshold is under the same second child node under the same first child node; each third child node under the same second child node corresponds to a reverse LUT.

[0131] The sampled pixel points at the same camera position may be taken out to determine the child nodes (ie, first child nodes) on the second layer of the LUT tree, and each first child node corresponds to a camera position.

[0132] The first coordinate axis can be any coordinate axis in the coordinate system, and the second coordinate axis can be another coordinate axis. For example, taking UV coordinates as an example, clustering can be performed based on the UV coordinates of the sampled pixels, such as based on the V value (the coordinate value of the first coordinate axis), to determine the child nodes (i.e., second child nodes) on the third level of the LUT tree. Each second child node corresponds to one or more sampled pixels with similar V values at the same camera position.

[0133] Taking UV coordinates as an example, in order to facilitate clustering based on UV coordinates to establish a LUT tree, before clustering, the U-axis coordinate value and the V-axis coordinate value in the UV coordinate value can be mapped from [0, 1] to [0, k1] and [0, k2] respectively. For example, u =k1*x u ,y v =k1*x v , where (yu ,y v ) is the UV coordinate value after mapping, (x u , x v ) is the UV coordinate value before mapping. k1 and k2 can take empirical values, for example, both are 180.

[0134] Taking the first coordinate axis as the V axis as an example, for each first child node, the sampling pixel points under the node can be clustered based on the V value in the UV coordinate. For example, when the V values of the two sampling pixel points satisfy abs(V1-V2)≤th, the two reverse LUTs corresponding to V1 and V2 are classified into a second child node, and the value of the second child node is v=(V1+V2) / 2; otherwise, if abs(V1-V2)>th, V1 and V2 can be considered as two second child nodes respectively.

[0135] Wherein, V1 and V2 may respectively represent the V values in the UV coordinates of two sampling pixel points under the same first child node, th is the above-mentioned predetermined threshold, which may take the empirical value 2, and abs(·) is used to take the absolute value.

[0136] After the second child nodes are determined, the second child nodes may be sorted according to the magnitude of the second child node values to obtain the third layer of the LUT tree.

[0137] For each second child node, each second coordinate axis coordinate value (here, U value) under the node can be used as a third child node, and the U values can be sorted by size to obtain the fourth layer of the LUT tree. Each third child node in the fourth layer can correspond to a sampled pixel point, that is, each third child node can correspond to an inverse LUT.

[0138] In this way, the LUT tree corresponding to screen i can be constructed (i is the number of the screen).

[0139] See also Figure 12 , shows a schematic diagram of a LUT tree according to an embodiment of the present application. Figure 12 As shown, the first layer of the LUT tree is the root node, corresponding to the screen numbered i (LED i), the second layer of the LUT tree is the first child node, the third layer is the second child node, and the fourth layer is the third child node. Each third child node can correspond to a sampled pixel point, that is, corresponds to a reverse LUT

[0140] Among them, the first child node value of the LUT tree is the position coordinate of the camera (such as loc1, loc2, locn), the second child node value is the average value of the first coordinate axis (such as V axis) coordinate value of each sampling pixel point under the same second child node (such as v1, v2, v3), and the third child node value is the second coordinate axis (such as U axis) coordinate value of the corresponding sampling pixel point (such as u1-u7).

[0141] It should be noted that the third and fourth layers of the above LUT tree are interchangeable. That is, the third layer can be clustered based on the V value, in which case the fourth layer corresponds to the U value. Alternatively, the third layer can be clustered based on the U value, in which case the fourth layer corresponds to the V value. In other words, the U axis can also be used as the first coordinate axis, and the V axis as the second coordinate axis.

[0142] After the LUT tree is established, you can use the LUT tree to calibrate the pixels to be calibrated. Figure 13 , shows a flow chart of a color calibration method according to an embodiment of the present application. Figure 13 As shown, the above method uses the LUT tree corresponding to the screen number of the pixel to be calibrated, interpolates based on the position, coordinates and initial color of the camera corresponding to the pixel to be calibrated, and determines the target color, including:

[0143] Step S1301 : Based on the position of the camera corresponding to the pixel to be calibrated, a first interpolation weight of the pixel to be calibrated is determined using the first child node in the LUT tree.

[0144] The LUT tree is a LUT tree corresponding to the screen number of the pixel to be calibrated. Optionally, step S1301 may include:

[0145] Based on the position of the camera corresponding to the pixel to be calibrated, determine one or two first child nodes adjacent to the position of the camera in the LUT tree; based on the position of the camera and one or two first child node values, determine the interpolation weights under each coordinate axis of the world coordinate system respectively; based on one or two first child node values, determine the weighted weights under each coordinate axis of the world coordinate system respectively; based on the interpolation weights and weighted weights under each coordinate axis, determine the first interpolation weight.

[0146] For example, the camera position corresponding to the pixel to be calibrated is loc(x,y,z), which can be a position in the world coordinate system. The Euclidean distance between loc(x,y,z) and each first child node can be calculated to determine one or two adjacent first child nodes. If there is only one first child node adjacent to loc, such as loc1(x1,y1,z1), then the first child node adjacent to loc is loc1. If there are two or more first child nodes adjacent to loc, the two first child nodes with the closest Euclidean distance to loc, such as loc1(x1,y1,z1) and loc2(x2,y2,z2), can be used as the first child node adjacent to loc. loc can overlap with the position of loc1 or loc2.

[0147] If there is only one first child node, there may be only one first child node adjacent to the camera position, which is the first child node.

[0148] The interpolation weights under the x-axis, y-axis, and z-axis can be determined separately. Taking the determination of the interpolation weight under the x-axis as an example:

[0149] In the case where x1 ≤ x2, if x ≥ x1 and x < x2, w_x can be set to (x - x1) / (x2 - x1); if x < x1, w_x can be set to 0; if x ≥ x2, w_x can be set to 1.

[0150] In the case where x2 < x1, if x ≥ x2 and x < x1, w_x can be set to (x - x2) / (x1 - x2); if x < x2, w_x can be set to 1; if x ≥ x1, w_x can be set to 0.

[0151] The above w_x can represent the interpolation weight under the x-axis. The methods for determining the interpolation weights under the y-axis and z-axis are the same, and thus the interpolation weight w_y under the y-axis and the interpolation weight w_z under the z-axis can be obtained.

[0152] Based on one or two first child node values, the weighted weights under the x-axis, y-axis, and z-axis can be determined separately. Taking the determination of the weighted weight under the x-axis as an example, wt_x can be set to abs(x2 - x1) / (abs(x2 - x1)+abs(y2 - y1)+abs(z2 - z1)). wt_x can represent the weighted weight under the x-axis. The methods for determining the weighted weights under the y-axis and z-axis are the same (changing the numerator to abs(y2 - y1) and abs(z2 - z1)), and thus the weighted weight wt_y under the y-axis and the weighted weight wt_z under the z-axis can be obtained.

[0153] Thus, one way to determine the first interpolation weight can be to make the first interpolation weight w_loc = wt_x * w_x + wt_y * w_y + wt_z * w_z.

[0154] Step S1302: Based on the first coordinate axis coordinate value of the pixel to be color-corrected, use the second child nodes under the corresponding first child nodes in the LUT tree to determine the second interpolation weight of the pixel to be color-corrected.

[0155] The second interpolation weight can be calculated for one or two first child nodes respectively. Optionally, this step S1302 may include:

[0156] Based on the first coordinate axis coordinate value of the pixel to be color-corrected, for one or two first child nodes, respectively determine one or two second child nodes adjacent to the first coordinate axis coordinate value under each first child node; for one or two first child nodes, based on the first coordinate axis coordinate value and one or two second child node values, determine the second interpolation weight.

[0157] For example, the first coordinate axis is the V axis, the V-axis coordinate value of the pixel to be color-corrected is v0, and there are three second child nodes v1, v2, and v3 (where v1 < v2 < v3) under the first child node loc2. If v0 is between two second child nodes, for example, between the second child node v1 and the second child node v2, that is, v1 ≤ v0 < v2, there can be two second child nodes adjacent to v0, namely v1 and v2; if v0 is at either end, that is, v0 < v1 or v0 ≥ v3, there can be one second child node adjacent to v0, which is v1 when v0 < v1 and v3 when v0 ≥ v3. Among them, v0 can coincide with the position of v1 or v2 or v3.

[0158] Taking the calculation of the second interpolation weight for the first child node loc2 as an example:

[0159] If v1 ≤ v0 < v2, w_v_loc2 can be set to (v0 - v1) / (v2 - v1); if v0 < v1, w_v_loc2 can be set to 0; if v0 ≥ v2, w_v_loc2 can be set to 1.

[0160] The above w_v_loc2 can represent the second interpolation weight of the first child node loc2. The method for determining the second interpolation weight of the first child node loc1 is the same, and thus the second interpolation weight w_v_loc1 of the first child node loc1 can be obtained.

[0161] In the case where there is only one second child node adjacent to v0, when calculating the second interpolation weight, v1 and v2 above can be equal values, that is, both are the values of the second child node adjacent to v0.

[0162] Step S1303: Based on the second coordinate axis coordinate value of the pixel to be color-corrected, use the third child nodes under the corresponding second child nodes in the LUT tree to determine the third interpolation weight of the pixel to be color-corrected.

[0163] The third interpolation weight can be calculated separately for each second child node adjacent to the first coordinate axis coordinate value under each first child node.

[0164] For example, the first coordinate axis is the V axis, the second coordinate axis is the U axis. Under the first child node loc2, there are two second child nodes adjacent to the V-axis coordinate value of the pixel to be color-corrected, namely v1 and v2, and under the first child node loc1, there are two second child nodes adjacent to the V-axis coordinate value of the pixel to be color-corrected, namely v2 and v3. The third interpolation weights of the second child nodes v1 and v2 under the first child node loc2, and the third interpolation weights of v2 and v3 under the first child node loc2 can be calculated separately.

[0165] When there is only one second child node under the first child node, there can be only one adjacent second child node, which is this second child node.

[0166] Optionally, this step S1303 may include:

[0167] Based on the second coordinate axis coordinate value, for one or two second child nodes adjacent to the first coordinate axis coordinate value under each first child node, respectively determine one or two third child nodes adjacent to the second coordinate axis coordinate value under each second child node; for one or two second child nodes, determine the third interpolation weight based on the second coordinate axis coordinate value and the value of one or two third child nodes.

[0168] For example, the first coordinate axis is the V axis, the second coordinate axis is the U axis, the U axis coordinate value of the pixel to be color-corrected is u0, and there are three third child nodes u1, u2, u3 (where u1 < u2 < u3) under the second child node v1 of the first child node loc2. If u0 is between two third child nodes, for example, between the third child node u1 and the third child node u2, that is, u1 ≤ u0 < u2, there can be two third child nodes adjacent to u0, which are u1 and u2 respectively; if u0 is at either end, that is, u0 < u1 or u0 ≥ u3, there can be one third child node adjacent to u0, which is u1 when u0 < u1 and u3 when u0 ≥ u3. Among them, u0 can coincide with the position of u1 or u2 or u3.

[0169] Taking the calculation of the third interpolation weight for the second child node v1 under the first child node loc2 as an example:

[0170] If u1 ≤ u0 < u2, it can be made that w_u1 = (u0 - u1) / (u2 - u1); if u0 < u1, it can be made that w_u1 = 0; if u0 ≥ u2, it can be made that w_u1 = 1.

[0171] The above w_u1 can represent the third interpolation weight of the second child node v1. The method of determining the third interpolation weights of other second child nodes is the same, so that the third interpolation weight w_u2 of the second child node v2, the third interpolation weight w_u3 of the second child node v3, and the third interpolation weight w_u4 of the second child node v4 can be obtained.

[0172] When there is only one third child node under the second child node, there can be only one adjacent third child node, which is this third child node. For example, at this time when calculating the third interpolation weight, u1 and u2 above can be equal values, that is, both are the value of this third child node adjacent to u0.

[0173] After determining the first interpolation weight, the second interpolation weight, and the third interpolation weight, the initial color of the pixel to be color-corrected can be color-corrected to determine the target color, as described below.

[0174] Step S1304 : determining a target color based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node.

[0175] For example, the coordinate value of the second coordinate axis (such as the U axis) of the pixel to be calibrated is u0. For the first child node loc2, if under the second child node v1, u0 is located between the third child nodes u1 and u2; under the second child node v2, u0 is located between the third child nodes u4 and u5. For the first child node loc1, if under the second child node v2, u0 is larger than u4 (see Figure 12 , at this time v2 has only one third child node u4); under the second child node v3, u0 is located between the third child nodes u6 and u7.

[0176] The following describes a possible method for determining the target color based on the above example. Optionally, step S1304 may include:

[0177] determining a plurality of first intermediate colors based on the initial colors and the inverse LUT corresponding to the corresponding third child nodes;

[0178] weighting the first intermediate color based on a third interpolation weight to determine a second intermediate color;

[0179] weighting the second intermediate color based on the second interpolation weight to determine a third intermediate color;

[0180] The third intermediate color is weighted based on the first interpolation weight to determine a target color.

[0181] The first intermediate colors lut1 (color_in), lut2 (color_in), lut4 (color_in), lut5 (color_in), lut6 (color_in), and lut7 (color_in) can be obtained based on the initial color color_in (including RGB values) and the reverse LUTs (lut1, lut2, lut4, and lut5) corresponding to the third child nodes u1, u2, u4, and u5 under the first child node loc2, as well as the reverse LUTs (lut4, lut6, and lut7) corresponding to the third child nodes u4, u6, and u7 under the first child node loc1. The RGB values of the initial color can be input into the reverse LUTs corresponding to the respective third child nodes, and the RGB values of the first intermediate colors output by the respective reverse LUTs are determined based on the mapping relationships in the reverse LUTs.

[0182] The first intermediate color can be weighted based on the third interpolation weights w_u1, w_u2, w_u3 and w_u4. In the above example, four second intermediate colors value_loc2_v1, value_loc2_v2, value_loc1_v2 and value_loc1_v3 can be obtained, wherein:

[0183] value_loc2_v1 = mix(lut1(color_in), lut2(color_in), w_u1), where value_loc2_v1 represents the second intermediate color of the second child node v1 under the first child node loc2, and mix(x, y, a) = x*(1-a)+y*a;

[0184] value_loc2_v2 = mix(lut4(color_in), lut5(color_in), w_u2), where value_loc2_v2 represents the second intermediate color corresponding to the second child node v2 under the first child node loc2;

[0185] value_loc1_v2=mix(lut4(color_in),lut4(color_in),w_u3), value_loc1_v2 represents the second intermediate color corresponding to the second child node v2 under the first child node loc1. Figure 12 , at this time there is only one third child node u4 under v2;

[0186] value_loc1_v3=mix(lut6(color_in),lut7(color_in),w_u4), where value_loc1_v3 represents the second intermediate color corresponding to the second child node v3 under the first child node loc1.

[0187] The second intermediate color can be weighted based on the second interpolation weights w_v_loc2 and w_v_loc1. In the above example, two third intermediate colors value_loc2 and value_loc1 can be obtained, wherein:

[0188] value_loc2=mix(value_loc2_v1, value_loc2_v2, w_v_loc2), where value_loc2 represents the third intermediate color corresponding to the first child node loc2;

[0189] value_loc1=mix(value_loc1_v2, value_loc1_v3, w_v_loc1), where value_loc1 represents the third intermediate color corresponding to the first child node loc1.

[0190] The third intermediate color may be weighted based on the first interpolation weight w_loc. In the above example, the target color color_out may be obtained, where: color_out=mix(value_loc1, value_loc2, w_loc).

[0191] Thus, the RGB value of the pixel to be calibrated after color calibration can be obtained.

[0192] Figure 14 FIG. 1 shows a structural diagram of a color calibration device according to an embodiment of the present application. Figure 14 As shown, the device may include:

[0193] An acquisition module 1401 is configured to acquire the position information and initial color of the pixel to be calibrated in the image to be displayed on the screen. The position information includes the position of the camera when the screen was captured, the coordinates of the pixel to be calibrated on the screen, and the screen number.

[0194] The adjustment module 1402 is configured to adjust the initial color of the pixel to be calibrated based on the position and coordinates of the camera and the screen number corresponding to the pixel to be calibrated, and determine the target color.

[0195] In one possible implementation, the adjustment module 1402 is configured to:

[0196] Using the color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position, coordinates and initial color of the camera corresponding to the pixel to be calibrated to determine the target color;

[0197] The LUT tree is determined based on the inverse LUT and position information corresponding to one or more sampled pixels on the screen. The inverse LUT is used to map the initial color of the pixel to be calibrated to the target color. The sampled pixels can be arranged at equal distances on the screen at predetermined coordinate intervals.

[0198] In a possible implementation, the area where the sampling pixel points are set may also include one or more of the following: the intersection area between screens, the area with surface contamination, and the area with quality control problems.

[0199] In a possible implementation, the reverse LUTs corresponding to the sampled pixel points captured at the same camera position are under the same first child node of the LUT tree;

[0200] Under the same first subnode, the reverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than the predetermined threshold is under the same second subnode under the same first subnode;

[0201] Each third sub-node under the same second sub-node corresponds to an inverse LUT.

[0202] In one possible implementation, the first child node value of the LUT tree is the position coordinate of the camera, the second child node value is the average of the first coordinate axis coordinate values of each sampling pixel point under the same second child node, and the third child node value is the second coordinate axis coordinate value of the corresponding sampling pixel point.

[0203] In a possible implementation, the position information corresponding to the sampling pixel point includes positions of multiple cameras corresponding to the sampling pixel point.

[0204] In one possible implementation, a color lookup table (LUT) tree corresponding to the screen number of the pixel to be calibrated is used to interpolate based on the position, coordinates, and initial color of the camera corresponding to the pixel to be calibrated to determine the target color, including:

[0205] Based on the position of the camera corresponding to the pixel to be calibrated, using the first child node in the LUT tree, determine the first interpolation weight of the pixel to be calibrated;

[0206] Based on the first coordinate axis coordinate value of the pixel point to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determine the second interpolation weight of the pixel point to be calibrated;

[0207] Based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determine the third interpolation weight of the pixel to be calibrated;

[0208] A target color is determined based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node.

[0209] In one possible implementation, determining a target color based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the inverse LUT corresponding to the corresponding third child node includes:

[0210] determining a plurality of first intermediate colors based on the initial colors and the inverse LUT corresponding to the corresponding third child nodes;

[0211] weighting the first intermediate color based on a third interpolation weight to determine a second intermediate color;

[0212] weighting the second intermediate color based on the second interpolation weight to determine a third intermediate color;

[0213] The third intermediate color is weighted based on the first interpolation weight to determine a target color.

[0214] In one possible implementation, based on the position of the camera corresponding to the pixel to be calibrated, a first interpolation weight of the pixel to be calibrated is determined using the first child node in the LUT tree, including:

[0215] Based on the position of the camera corresponding to the pixel to be calibrated, determine one or two first child nodes adjacent to the position of the camera in the LUT tree;

[0216] Based on the position of the camera and one or two first child node values, determine the interpolation weights of each coordinate axis of the world coordinate system respectively;

[0217] Determine weights for each coordinate axis of the world coordinate system based on one or two first child node values;

[0218] A first interpolation weight is determined based on the interpolation weights and the weighted weights under each coordinate axis.

[0219] In one possible implementation, based on the first coordinate axis coordinate value of the pixel to be calibrated, using the second child node under the corresponding first child node in the LUT tree, determining the second interpolation weight of the pixel to be calibrated includes:

[0220] Based on the coordinate value of the first coordinate axis, for one or two first child nodes, determine one or two second child nodes adjacent to the coordinate value of the first coordinate axis under each first child node;

[0221] For one or two first child nodes, a second interpolation weight is determined based on the first coordinate axis coordinate value and one or two second child node values.

[0222] In one possible implementation, based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree, determining the third interpolation weight of the pixel to be calibrated includes:

[0223] Based on the coordinate value of the second coordinate axis, for one or two second subnodes adjacent to the coordinate value of the first coordinate axis under each first subnode, determine one or two third subnodes adjacent to the coordinate value of the second coordinate axis under each second subnode;

[0224] For one or two second child nodes, a third interpolation weight is determined based on the second coordinate axis coordinate value and one or two third child node values.

[0225] According to an embodiment of the present application, by calibrating the color of the pixels to be calibrated in the image displayed on the screen based on the camera's position when capturing the screen, the problem of decreased color calibration due to camera position changes can be solved, thereby improving color calibration accuracy. By calibrating the pixels to be calibrated based on their coordinates on the screen, the color calibration method of the present application is not affected by different coordinate systems and can solve the problem of significant decreases in color calibration accuracy in different areas of the screen under the same viewport (i.e., the same camera position). This ensures consistency in color calibration across different areas and improves color calibration accuracy. Furthermore, by calibrating the pixels to be calibrated based on these coordinates, the color calibration process is not constrained by screens of different shapes or positions, and does not require the screen to be in an upright or lying position. This makes the solution universally applicable to screens of various shapes and positions, meeting the needs of different scenarios to the greatest extent possible and providing greater versatility. By calibrating the target color (i.e., the color after calibration) based on the screen number, differences in luminous properties between different screens can be taken into account, improving the overall color calibration effect.

[0226] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0227] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0228] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0229] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0230] Figure 15 FIG1 is a block diagram of an apparatus 800 for color calibration according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0231] Reference Figure 15, the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 (I / O interface), a sensor component 814 , and a communication component 816 .

[0232] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0233] The memory 804 is configured to store various types of data to support the operations of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0234] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0235] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0236] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0237] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0238] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0239] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0240] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0241] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by the processor 820 of the apparatus 800 to perform the above method.

[0242] Figure 16 1 is a block diagram of a device 1900 for color calibration according to an exemplary embodiment. For example, the device 1900 can be provided as a server or a terminal device. Figure 16 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0243] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSDTM or similar.

[0244] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0245] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0246] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0247] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0248] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0249] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0250] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0251] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0252] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0253] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A color calibration method, characterized in that: The method comprises: Obtaining position information and initial colors corresponding to pixels to be calibrated in an image to be displayed on a screen, the position information including the position of a camera when shooting the screen, the coordinates of the pixels to be calibrated on the screen, and the screen number; Adjusting the initial color of the pixel to be calibrated based on the position of the camera corresponding to the pixel to be calibrated, the coordinates, and the number of the screen to determine the target color includes: Using a color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position of the camera corresponding to the pixel to be calibrated, the coordinates, and the initial color to determine the target color; The LUT tree is established based on sampling pixel points. The sampling pixel points are set in different areas of the screen. Different camera positions correspond to different sampling pixel points. The areas where the sampling pixel points are set also include one or more of the intersection areas between screens, areas with surface contamination, and areas with quality control problems. The reverse LUT corresponding to the sampled pixel points captured at the same camera position is under the same first child node of the LUT tree; Under the same first subnode, the reverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than the predetermined threshold is under the same second subnode under the same first subnode; Each third sub-node under the same second sub-node corresponds to an inverse LUT.

2. The method according to claim 1, characterized in that The LUT tree is determined based on an inverse LUT and position information corresponding to one or more sampled pixels on the corresponding screen, and the inverse LUT is used to map the initial color of the pixel to be calibrated to the target color.

3. The method according to claim 1, characterized in that The first child node value of the LUT tree is the position coordinate of the camera, the second child node value is the average of the first coordinate axis coordinate values of each sampling pixel point under the same second child node, and the third child node value is the second coordinate axis coordinate value of the corresponding sampling pixel point.

4. The method according to claim 2 or 3, characterized in that The position information corresponding to the sampling pixel point includes the positions of multiple cameras corresponding to the sampling pixel point.

5. The method according to any one of claims 1 to 4, characterized in that The method of determining the target color by using a color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated and interpolating based on the position of the camera corresponding to the pixel to be calibrated, the coordinates, and the initial color, includes: Determining a first interpolation weight of the pixel to be calibrated using a first child node in the LUT tree based on a position of a camera corresponding to the pixel to be calibrated; Determine a second interpolation weight of the pixel to be calibrated based on the first coordinate axis coordinate value of the pixel to be calibrated, using the second child node under the corresponding first child node in the LUT tree; Determine a third interpolation weight of the pixel to be calibrated based on the second coordinate axis coordinate value of the pixel to be calibrated, using the third child node under the corresponding second child node in the LUT tree; The target color is determined based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and an inverse LUT corresponding to the corresponding third child node.

6. The method according to claim 5, characterized in that The determining the target color based on the initial color, the first interpolation weight, the second interpolation weight, the third interpolation weight, and the reverse LUT corresponding to the corresponding third child node includes: determining a plurality of first intermediate colors based on the initial colors and the inverse LUT corresponding to the corresponding third child nodes; weighting the first intermediate color based on the third interpolation weight to determine a second intermediate color; weighting the second intermediate color based on the second interpolation weight to determine a third intermediate color; The third intermediate color is weighted based on the first interpolation weight to determine the target color.

7. The method according to claim 5 or 6, characterized in that The determining, based on the position of the camera corresponding to the pixel to be calibrated, using the first child node in the LUT tree, a first interpolation weight of the pixel to be calibrated includes: Based on the position of the camera corresponding to the pixel to be calibrated, determining one or two first child nodes adjacent to the position of the camera in the LUT tree; Determine interpolation weights for each coordinate axis of a world coordinate system based on the position of the camera and the one or two first child node values; Determine weighted values for each coordinate axis of the world coordinate system based on the one or two first child node values; The first interpolation weight is determined based on the interpolation weight and the weighted weight under each coordinate axis.

8. The method according to any one of claims 5 to 7, characterized in that: Determining a second interpolation weight of the pixel to be calibrated based on the first coordinate axis coordinate value of the pixel to be calibrated and using a second child node under the corresponding first child node in the LUT tree includes: Based on the first coordinate axis coordinate value, for the one or two first sub-nodes, determine one or two second sub-nodes adjacent to the first coordinate axis coordinate value under each first sub-node; For the one or two first sub-nodes, the second interpolation weight is determined based on the first coordinate axis coordinate value and the one or two second sub-node values.

9. The method according to any one of claims 5 to 8, characterized in that: Determining a third interpolation weight of the pixel to be calibrated based on the second coordinate axis coordinate value of the pixel to be calibrated and using a third subnode under the corresponding second subnode in the LUT tree includes: Based on the second coordinate axis coordinate value, for one or two second subnodes adjacent to the first coordinate axis coordinate value under each first subnode, determine one or two third subnodes adjacent to the second coordinate axis coordinate value under each second subnode; For the one or two second sub-nodes, the third interpolation weight is determined based on the second coordinate axis coordinate value and the one or two third sub-node values.

10. A color calibration device, characterized in that: The device comprises: An acquisition module is used to obtain position information and initial colors corresponding to pixels to be calibrated in an image to be displayed on a screen, wherein the position information includes the position of a camera when shooting the screen, the coordinates of the pixels to be calibrated on the screen, and the screen number; An adjustment module, configured to adjust the initial color of the pixel to be calibrated based on the position of the camera corresponding to the pixel to be calibrated, the coordinates, and the screen number, to determine a target color, including: Using a color lookup table LUT tree corresponding to the screen number of the pixel to be calibrated, interpolation is performed based on the position of the camera corresponding to the pixel to be calibrated, the coordinates, and the initial color to determine the target color; The LUT tree is established based on sampling pixel points. The sampling pixel points are set in different areas of the screen. Different camera positions correspond to different sampling pixel points. The areas where the sampling pixel points are set also include one or more of the intersection areas between screens, areas with surface contamination, and areas with quality control problems. The reverse LUT corresponding to the sampled pixel points captured at the same camera position is under the same first child node of the LUT tree; Under the same first subnode, the reverse LUT corresponding to each sampling pixel point whose coordinate values of the first coordinate axis in the coordinates are less than the predetermined threshold is under the same second subnode under the same first subnode; Each third sub-node under the same second sub-node corresponds to an inverse LUT.

11. A color calibration device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 9 when executing the instructions stored in the memory.

12. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images