Color calibration method, device and storage medium

Abstract

The present disclosure relates to a color calibration method, device and storage medium. The method is used for a first server, and the method includes: controlling a display device to display at least one frame of a first picture, and controlling multiple cameras to synchronously capture the first picture; based on the second picture captured by multiple cameras for the first picture, analyzing the difference between the color of the second picture and the color of the first picture projected; based on the difference between the color of the second picture and the color of the first picture projected, color calibrating multiple cameras. According to the embodiment of the present disclosure, the tedious process of shooting calibration pictures one by one with a single camera can be avoided, and multiple cameras can be controlled to complete data collection at the same time, further improving work efficiency, and significantly shortening the calibration time through parallel calibration, greatly improving the efficiency of multi-camera color calibration, avoiding the repeated operation of calibrating a single camera one by one, and meeting the requirements of fast response and high consistency of multi-camera systems in specific scenarios.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual shooting, and in particular to a color calibration method, device, and storage medium. Background Art

[0002] With the rapid development of virtual filming technology, the application of multi-camera systems has become particularly important in fields such as film and television production, Virtual Reality (VR), and Augmented Reality (AR). Virtual filming scenes usually require multiple cameras to work together, capturing images of the same scene from different perspectives and synthesizing and rendering these images in real time to construct a virtual image that highly reproduces the real world. However, due to differences in camera hardware characteristics, sensor sensitivity, lens optical characteristics, and lighting conditions, different cameras often produce color deviations when capturing the same scene. This color inconsistency affects the naturalness of the virtual filming image splicing, resulting in noticeable seams in the image fusion, which undermines the immersion and authenticity of the virtual scene.

[0003] Traditional color calibration solutions typically involve calibrating each camera independently. However, calibrating each camera individually is not only cumbersome and inefficient, but also unable to guarantee consistent color calibration across multiple cameras within a short period of time. Furthermore, calibrating each camera individually requires repeated setup and data collection, which doesn't guarantee accurate color calibration. It's time-consuming, labor-intensive, and error-prone, making it difficult to meet the demands of real-time applications. Summary of the Invention

[0004] In view of this, the present disclosure proposes a color calibration method, device, and storage medium.

[0005] According to one aspect of the present disclosure, a color calibration method is provided. The method is used on a first server and includes:

[0006] Controlling a display device to display at least one frame of the first image, and controlling multiple cameras to synchronously capture the first image;

[0007] Analyzing, based on a second image captured by multiple cameras for the first image, a difference between a color of the second image and a color of the first image projected;

[0008] Based on the difference between the colors of the second screen and the colors of the first screen, multiple cameras are color calibrated.

[0009] In one possible implementation, controlling a display device to display at least one frame of a first image and controlling multiple cameras to simultaneously capture the first image includes:

[0010] Controlling the display device to display the first picture of the current frame, and controlling multiple cameras to synchronously capture the first picture of the current frame;

[0011] Analyze the difference between the color of the second image captured by multiple cameras and the color of the first image, including:

[0012] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected corresponding frame first screens are obtained.

[0013] In one possible implementation, controlling a display device to display at least one frame of a first image and controlling multiple cameras to synchronously capture the first image includes:

[0014] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset duration, and controlling multiple cameras to synchronously capture each frame of the first image;

[0015] Analyze the difference between the color of the second image captured by multiple cameras and the color of the first image, including:

[0016] In response to the plurality of cameras capturing the corresponding second images for each frame of the first image, differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding first images projected are analyzed respectively.

[0017] In a possible implementation, the method further includes:

[0018] In response to a camera that fails to capture a corresponding second image for one or more first images, the display device is controlled to redisplay the one or more first images, and the camera that fails to capture a corresponding second image is controlled to recapture the one or more first images.

[0019] In one possible implementation, controlling a display device to display at least one first frame includes:

[0020] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0021] In a possible implementation, when multiple cameras all capture second images including the same identifier, it is determined that the multiple cameras all capture corresponding second images for the first image including the same identifier.

[0022] In one possible implementation, before controlling the display device to display at least one frame of the first image and controlling the multiple cameras to synchronously capture the first image, the method further includes:

[0023] Determine desired values ​​of camera parameters and set the camera parameters of multiple cameras to the same desired values.

[0024] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface (SDI) signal format, color temperature, aperture, and focal length.

[0025] In one possible implementation, color calibration is performed on multiple cameras based on a difference between the color of the second image and the color of the projected first image, including:

[0026] For each camera, a lookup table (LUT) corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0027] According to another aspect of the present disclosure, a color calibration device is provided. The device is used for a first server and includes:

[0028] A control module, configured to control a display device to display at least one frame of a first image, and to control multiple cameras to synchronously capture the first image;

[0029] An analysis module is configured to analyze, based on a second image captured by multiple cameras for the first image, a difference between the color of the second image and the color of the first image projected;

[0030] The color calibration module is used to perform color calibration on multiple cameras based on the difference between the color of the second picture and the color of the first picture projected.

[0031] In a possible implementation, the control module is configured to:

[0032] Controlling the display device to display the first picture of the current frame, and controlling multiple cameras to synchronously capture the first picture of the current frame;

[0033] Analysis modules for:

[0034] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected corresponding frame first screens are obtained.

[0035] In a possible implementation, the control module is configured to:

[0036] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset duration, and controlling multiple cameras to synchronously capture each frame of the first image;

[0037] Analysis modules for:

[0038] In response to the plurality of cameras capturing the corresponding second images for each frame of the first image, differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding first images projected are analyzed respectively.

[0039] In a possible implementation, the device further includes:

[0040] The redisplay module is used to control the display device to redisplay the one or more frames of the first image in response to the existence of a camera that has not captured the corresponding second image for one or more frames of the first image, and to control the camera that has not captured the corresponding second image to recapture the one or more frames of the first image.

[0041] In a possible implementation, the control module is configured to:

[0042] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0043] In a possible implementation, when multiple cameras all capture second images including the same identifier, it is determined that the multiple cameras all capture corresponding second images for the first image including the same identifier.

[0044] In a possible implementation, the device further includes:

[0045] The determination module is used to determine expected values ​​of camera parameters and set the camera parameters of the multiple cameras to the same expected values ​​before controlling the display device to display at least one frame of the first image and controlling multiple cameras to synchronously capture the first image.

[0046] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface (SDI) signal format, color temperature, aperture, and focal length.

[0047] In one possible implementation, the color calibration module is configured to:

[0048] For each camera, a lookup table (LUT) corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0049] According to another aspect of the present disclosure, a color calibration device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0050] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0051] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0052] According to an embodiment of the present disclosure, by controlling the display device to display at least one frame of the first image through the first server, and controlling multiple cameras to synchronously capture the first image, the tedious process of a single camera shooting calibration images one by one can be avoided, and multiple cameras can be controlled to complete data acquisition at the same time, which greatly reduces the color calibration time. The automated synchronous operation can reduce the workload of human intervention, further improving work efficiency. In addition, synchronous acquisition can also reduce the light changes and color shift problems caused by the time difference, making subsequent color difference calculations more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second image and the color of the first image projected based on the second image captured by multiple cameras for the first image, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of the multi-camera system in specific scenarios.

[0053] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0055] Figure 1 A structural diagram of a color calibration system according to an embodiment of the present disclosure is shown.

[0056] Figure 2 A flowchart of a color calibration method according to an embodiment of the present disclosure is shown.

[0057] Figure 3 A structural diagram of a color calibration device according to an embodiment of the present disclosure is shown.

[0058] Figure 4 is a block diagram of a device 1900 for color calibration according to an exemplary embodiment. DETAILED DESCRIPTION

[0059] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0060] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0061] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0062] With the rapid development of virtual filming technology, the application of multi-camera systems has become particularly important in fields such as film and television production, Virtual Reality (VR), and Augmented Reality (AR). Virtual filming scenes usually require multiple cameras to work together, capturing images of the same scene from different perspectives and synthesizing and rendering these images in real time to construct a virtual image that highly reproduces the real world. However, due to differences in camera hardware characteristics, sensor sensitivity, lens optical characteristics, and lighting conditions, different cameras often produce color deviations when capturing the same scene. This color inconsistency affects the naturalness of the virtual filming image splicing, resulting in noticeable seams in the image fusion, which undermines the immersion and authenticity of the virtual scene.

[0063] Traditional color calibration solutions typically involve calibrating each camera independently. However, calibrating each camera individually is not only cumbersome and inefficient, but also unable to guarantee consistent color calibration across multiple cameras within a short period of time. Furthermore, calibrating each camera individually requires repeated setup and data collection, which doesn't guarantee accurate color calibration. It's time-consuming, labor-intensive, and error-prone, making it difficult to meet the demands of real-time applications.

[0064] In view of this, the present disclosure proposes a color calibration method, device and storage medium. The method can be used for the first server. By controlling the display device to display at least one frame of the first picture and controlling multiple cameras to synchronously capture the first picture, the tedious process of shooting the calibration pictures one by one with a single camera can be avoided, and multiple cameras can be controlled to complete data acquisition at the same time, which greatly reduces the color calibration time. The workload of human intervention can be reduced through automated synchronous operation, further improving work efficiency. Moreover, the light change and color shift problems caused by the time difference can be reduced through synchronous acquisition, making the subsequent color difference calculation more accurate, which helps to improve the overall calibration accuracy. Thus, by analyzing the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of the multi-camera system in specific scenarios.

[0065] Figure 1 FIG. 1 shows a structural diagram of a color calibration system according to an embodiment of the present disclosure. Figure 1As shown, the color calibration system may include a first server, a second server, a display device, and multiple cameras (such as cameras 1 to 3 in the figure). The first server can be a master computer, used to control multiple cameras and the second server; the second server can be a screen-mounting machine, used to receive images sent by the first server and send them to the display device for display; the display device can be used to display the images sent by the second server; and cameras 1 to 3 can be used to capture the images displayed by the display device. The present disclosure does not limit the number of cameras.

[0066] Among them, the server in this disclosure generally refers to a device or system that provides data, resources or services. The first server and the second server can be different physical servers, and this disclosure does not limit the type of server. The first server and the second server can also be the same server. When the first server and the second server are different servers, they can be connected by wired or wireless means. The second server and the display device can be connected by wired or wireless means. The display device can be a light emitting diode (LED) display, a liquid crystal display (LCD), etc., and this disclosure does not limit this. The first server and cameras 1 to 3 can be connected by wired or wireless means. Cameras 1 to 3 can be set at any position. This disclosure does not display the types of cameras 1 to 3. Multiple cameras can be image acquisition devices of the same type or different types.

[0067] The color calibration system disclosed in the present invention can be used in application scenarios of multi-camera shooting. For example, in a virtual shooting scenario, the first server can control the second server to render the picture to be shot and display it on the display device, and control multiple cameras to work together to shoot different perspectives of the same scene, and synthesize and render these pictures in real time, thereby constructing a virtual picture that highly restores the real world. In such scenarios of multi-camera collaborative shooting, different cameras often produce color deviations when shooting the same scene, and this color inconsistency will affect the shooting effect. The color calibration method disclosed in the present invention can be used to perform color calibration on multiple cameras before formal shooting to ensure that multiple cameras will not have color differences when capturing the same picture during the formal shooting process.

[0068] Figure 2 FIG. 1 is a flow chart showing a color calibration method according to an embodiment of the present disclosure. The method can be used for the first server as described above. Figure 2 As shown, the method may include:

[0069] Step S201 : controlling a display device to display at least one frame of a first image, and controlling a plurality of cameras to synchronously capture the first image.

[0070] Before multiple cameras capture images, camera parameters of each camera may be uniformly configured first. The method may further include:

[0071] Determine desired values ​​of camera parameters and set the camera parameters of multiple cameras to the same desired values.

[0072] The camera parameters may include any one or more of camera color gamut, white balance, frame rate, Serial Digital Interface (SDI) signal format, color temperature, aperture, and focal length.

[0073] The expected value corresponding to each camera parameter can be pre-configured as needed. The first server can send the pre-configured expected value corresponding to each camera parameter to each camera, so that the same camera parameter on each camera is configured as the same expected value.

[0074] In this way, the camera parameters of multiple cameras can be automatically set, reducing color calibration failures caused by missed settings or incorrect settings during manual settings, thereby improving the accuracy and efficiency of the color calibration process.

[0075] The first server may determine the color to be projected on the first screen, wherein one frame of the first screen may include one color. During the color calibration process, multiple colors are usually used, so these colors may be displayed separately through multiple frames of the first screen. In order to distinguish the colors of different frames of the first screen, the first server may also generate different identifiers for the colors to be projected. These identifiers may be used to distinguish different colors to be projected. In step S201, the following may be performed:

[0076] The first server sends the color of the screen to be projected and the identifier corresponding to the color to be projected to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color to be projected, and then sends it to the display device to display the first picture on the display device.

[0077] Among them, the identifier corresponding to the color to be projected can be in any form. For example, the identifier can be the RGB (red, green, blue) value of the color to be projected, or in the form of numbers, letters, etc., as long as different colors to be projected can be distinguished.

[0078] The second server can generate a first screen by rendering the color to be projected and the corresponding logo. The rendering process can be implemented based on existing technology, so that the corresponding color and logo are displayed on the first screen. In the first screen obtained after rendering, the logo can be located at any position in the first screen.

[0079] After the first picture is displayed on the display device, the first server can control each camera to synchronously capture the first picture. Each camera can be set at any relative position of the display device (including any relative angle and any relative distance). The position of each camera can also be set to a symmetrical layout, for example, multiple cameras are distributed at equal angles around the display device and maintain a consistent distance from the display device to ensure that the pictures captured by all cameras are consistent.

[0080] After the display device displays the first image, it can feedback the current display status (for example, a status signal indicating that the first image has been displayed) to the second server through the control interface. The second server can feedback the display status of the display device to the first server, so that the first server can know that the first image has been displayed on the display device. The display device can also communicate synchronously with the second server through a synchronization mechanism (for example, sending a synchronization signal to the second server) to feedback to the second server that the first image has been displayed. The second server can synchronize the synchronization signal fed back by the display device to the first server, so that the first server can know that the first image has been displayed on the display device.

[0081] After the multiple cameras capture the corresponding second images for the first images, they can send the captured second images to the first server so that the first server can perform subsequent analysis.

[0082] Step S202 , based on the second picture captured by multiple cameras for the first picture, analyze the difference between the color of the second picture and the color of the first picture projected.

[0083] The color of the first screen projection is used to generate the color of the screen to be projected for the first screen.

[0084] Since color calibration usually requires displaying multiple frames of the first image, one possible implementation method is that after multiple cameras complete the acquisition of a frame of the first image, the first server performs a difference analysis on the frame of the first image. After the analysis is completed, multiple cameras capture the next frame of the first image, and the first server performs the analysis again, until the differences of all frames of the first image are analyzed. In this case, in step S201, you can:

[0085] Controlling the display device to display the first picture of the current frame, and controlling multiple cameras to synchronously capture the first picture of the current frame;

[0086] In step S202, you can:

[0087] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected corresponding frame first screens are obtained.

[0088] For the first picture of the current frame, if the first server does not receive the corresponding second picture captured by a certain camera within a preset time, for example, the second pictures captured by camera 1 and camera 3 are received within the preset time, but the second picture captured by camera 2 is not captured, the camera that has not captured the second picture (i.e., camera 2) can be controlled to re-capture the first picture of the current frame to obtain the second pictures captured by all cameras for the first picture of the current frame.

[0089] When the first server performs analysis, it can analyze the difference between the color of the second picture captured by the camera and the color of the current frame of the first picture for each camera to obtain the analysis result of the camera. After analyzing each camera, the color of the next picture to be projected and the corresponding identifier are sent to the second server, so that the second server generates the next frame of the first picture and displays the next frame of the first picture on the display device, and controls multiple cameras to synchronously capture and analyze based on the captured second picture. By iteratively executing the above process, the analysis results for multiple frames of the first picture corresponding to each camera can be obtained. For example, if there are N frames of the first picture, N analysis results can be obtained for each camera, each analysis result corresponds to 1 frame of the first picture, and each analysis result represents the difference between the color of the second picture captured by the camera and the color of the first picture projected for the frame of the first picture.

[0090] In one possible implementation, in response to multiple cameras capturing a second image for the current first image, the first server can update the next first image as the current first image without waiting for the analysis of the color difference between the second images captured by the multiple cameras and the color of the projected first image. In other words, the first server can also perform the step of updating the first image and the step of analyzing the color difference asynchronously.

[0091] In another possible implementation, after multiple cameras have completed capturing the first images of each frame, the first server can then perform a difference analysis on the first images of each frame. In this case, in step S201, the following can be done:

[0092] The display device is controlled to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time length, and multiple cameras are controlled to synchronously capture each frame of the first image.

[0093] In step S202, you can:

[0094] In response to the plurality of cameras capturing the corresponding second images for each frame of the first image, differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding first images projected are analyzed respectively.

[0095] The multiple frames of first images displayed continuously can be all the first images to be collected and used in the color calibration process. The first server can control the display device through the second server to continuously display all the first images to be collected, where each frame of the first image is displayed for a preset time length (such as 20ms), and can control multiple cameras to collect the first image at the same frequency, for example, once every 20ms, and multiple cameras can send the collected second images to the first server.

[0096] During continuous acquisition, one or more cameras may miss capturing one or more first frames. The first server may determine whether there is a camera that fails to capture the corresponding second frame for one or more first frames.

[0097] When the plurality of cameras all capture the second images including the same identifier, it can be determined that the plurality of cameras all capture the corresponding second images for the first images including the same identifier.

[0098] By determining whether multiple cameras have captured the corresponding second image for each first image based on the identifier, the uniqueness and comparability of color data in the color calibration process can be achieved, further improving the accuracy of color calibration.

[0099] The method may further include:

[0100] In response to a camera that fails to capture a corresponding second image for one or more first images, the display device is controlled to redisplay the one or more first images, and the camera that fails to capture a corresponding second image is controlled to recapture the one or more first images.

[0101] By recapturing the missed images, the occurrence of missed images during the color calibration process can be prevented, thereby further improving the accuracy of the color calibration.

[0102] For example, the display device continuously displays three frames of the first image, and the identifiers included in these three frames of the first image are (255, 0, 0), (0, 255, 0) and (0, 0, 255) respectively. When the first server does not receive the second image including the identifier (0, 0, 255) sent back by camera 2, and does not receive the second image including the identifiers (0, 255, 0) and (0, 0, 255) sent back by camera 3, it can be considered that camera 2 missed capturing the corresponding second image for the third frame of the first image, and camera 3 missed capturing the corresponding second images for the second and third frames of the first image. At this time, the first server can control the display device through the second server to continuously display the first images of the 2nd and 3rd frames that were missed (that is, the first images including the identifiers (0, 255, 0) and (0, 0, 255)), and when the display device displays the first image of the 2nd frame, control the camera 3 that missed capturing the image to re-capture the corresponding second image for the first image, and when the display device displays the first image of the 3rd frame, control the cameras 2 and 3 that missed capturing the image to re-capture the corresponding second images for the first image.

[0103] Through the above process, the first server can receive multiple frames of second images captured by M cameras (M is the total number of cameras), where each camera captures at least N frames of second images (N is the total number of displayed first images), and each of the N frames of second images is associated with one frame of the first image.

[0104] When the first server performs analysis, it can analyze the difference between the color of each second image captured by each camera and the color of the corresponding first image projected, thereby obtaining analysis results for multiple frames of the first image corresponding to each camera. The analysis results for multiple frames of the first image corresponding to each camera can be found in the examples in the above embodiments.

[0105] In the above process of controlling multiple cameras to synchronously capture the first image, the first server can also control multiple cameras to capture multiple frames of the first image for one frame, where the number of captures can be set as needed, so that for one frame of the first image, each camera can capture multiple frames of the corresponding second image.

[0106] In this case, when the first server analyzes the difference between the color of the second image and the color of the first image projected based on the second image captured by multiple cameras for the first image, for a certain frame of the first image, the first server can analyze the difference between the color of the second image and the color of the first image projected based on the second image captured multiple times. For example, for the third frame of the first image, if camera 2 captures four frames of the second image, the first server can first average the colors of the four frames of the second image, and then analyze the difference between the averaged color of the four frames of the second image and the color of the corresponding third frame of the first image projected.

[0107] In this way, multiple cameras can be calibrated synchronously, which can improve work efficiency and picture quality in corresponding shooting scenarios and provide users with a more realistic and smooth experience.

[0108] In the analysis results for the multiple frames of the first image corresponding to each of the above cameras, the difference between the color of the second image and the color of the projected first image can include the difference between the RGB color of the second image and the RGB color of the projected first image. The RGB color of the projected first image is the RGB color used to generate the color to be projected of the first image.

[0109] You can calculate the difference between the color of the second image and the color of the first image by subtracting the value of each RGB component (red, green, and blue) in the RGB color of the first image from the corresponding color component in the RGB color of the second image. This difference in color is used as the difference between the colors of the second image and the first image. For example, if the color of the first image is (0, 255, 255), and the camera captures the color of the second image as (10, 220, 220) for (0, 255, 255), then the difference between the colors of the second image and the first image is (-10, 5, 5).

[0110] Step S203: color calibrate the multiple cameras based on the difference between the color of the second picture and the color of the first picture.

[0111] According to an embodiment of the present disclosure, by controlling the display device to display at least one frame of the first image through the first server, and controlling multiple cameras to synchronously capture the first image, the tedious process of a single camera shooting calibration images one by one can be avoided, and multiple cameras can be controlled to complete data acquisition at the same time, which greatly reduces the color calibration time. The automated synchronous operation can reduce the workload of human intervention, further improving work efficiency. In addition, synchronous acquisition can also reduce the light changes and color shift problems caused by the time difference, making subsequent color difference calculations more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second image and the color of the first image projected based on the second image captured by multiple cameras for the first image, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of the multi-camera system in specific scenarios.

[0112] Color calibration may be performed for each camera separately. In step S203, the following steps may be performed:

[0113] For each camera, a lookup table (LUT) corresponding to the camera is generated based on the difference between the color of the projected first image of each frame and the color of the corresponding second image captured by the camera.

[0114] LUT can be used to perform color calibration on the camera. LUT can represent the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0115] For example, the LUT corresponding to each camera may include N pairs of mapping relationships, each of which is determined based on the difference between the color of the N frames of the first screen projected and the color of the corresponding second screen captured by the camera. Each pair of mapping relationships associates the color captured by the camera with the color value that needs to be adjusted (i.e., the calculated color difference used to compensate for the color deviation captured by the camera). For example, for the standard color (0, 255, 255), if the color of the second screen captured by the camera for this color is (10, 220, 220) based on the above, and the corresponding difference from the standard color (i.e., the color of the first screen projected) is (-10, 5, 5), then a pair of mapping relationships can be generated, including the corresponding RGB color (10, 220, 220) captured by the camera and the color value (-10, 5, 5) that needs to be adjusted for each color component therein.

[0116] During the actual shooting process, the color captured by the camera can be added to the corresponding color value that needs to be adjusted in the LUT (for example, for the color (10,220,220) captured by the camera, adjustment can be made according to the corresponding color value (-10,5,5) that needs to be adjusted in the LUT) to obtain the output color after color calibration (for example, adding (10,220,220) to (-10,5,5) will obtain (0,255,255)).

[0117] In this way, the color differences between different cameras during the actual shooting process can be eliminated, ensuring the consistency of the final output image color and the effect of image splicing and synthesis.

[0118] Figure 3 The structure diagram of the color calibration device according to the embodiment of the present disclosure is shown. The device is used for the first server, such as Figure 3 As shown, the device includes:

[0119] The control module 301 is used to control the display device to display at least one frame of the first image, and control multiple cameras to synchronously capture the first image;

[0120] An analysis module 302 is configured to analyze, based on a second image captured by multiple cameras for the first image, a difference between the color of the second image and the color of the first image projected;

[0121] The color calibration module 303 is used to perform color calibration on multiple cameras based on the difference between the color of the second picture and the color of the first picture.

[0122] In a possible implementation, the control module 301 is configured to:

[0123] Controlling the display device to display the first picture of the current frame, and controlling multiple cameras to synchronously capture the first picture of the current frame;

[0124] The analysis module 302 is used to:

[0125] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected corresponding frame first screens are obtained.

[0126] In a possible implementation, the control module 301 is configured to:

[0127] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset duration, and controlling multiple cameras to synchronously capture each frame of the first image;

[0128] The analysis module 302 is used to:

[0129] In response to the plurality of cameras capturing the corresponding second images for each frame of the first image, differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding first images projected are analyzed respectively.

[0130] In a possible implementation, the device further includes:

[0131] The redisplay module is used to control the display device to redisplay the one or more frames of the first image in response to the existence of a camera that has not captured the corresponding second image for one or more frames of the first image, and to control the camera that has not captured the corresponding second image to recapture the one or more frames of the first image.

[0132] In a possible implementation, the control module 301 is configured to:

[0133] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0134] In a possible implementation, when multiple cameras all capture second images including the same identifier, it is determined that the multiple cameras all capture corresponding second images for the first image including the same identifier.

[0135] In a possible implementation, the device further includes:

[0136] The determination module is used to determine expected values ​​of camera parameters and set the camera parameters of the multiple cameras to the same expected values ​​before controlling the display device to display at least one frame of the first image and controlling multiple cameras to synchronously capture the first image.

[0137] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface (SDI) signal format, color temperature, aperture, and focal length.

[0138] In one possible implementation, the color calibration module 303 is configured to:

[0139] For each camera, a lookup table (LUT) corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0140] According to an embodiment of the present disclosure, by controlling the display device to display at least one frame of the first image through the first server, and controlling multiple cameras to synchronously capture the first image, the tedious process of a single camera shooting calibration images one by one can be avoided, and multiple cameras can be controlled to complete data acquisition at the same time, which greatly reduces the color calibration time. The automated synchronous operation can reduce the workload of human intervention, further improving work efficiency. In addition, synchronous acquisition can also reduce the light changes and color shift problems caused by the time difference, making subsequent color difference calculations more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second image and the color of the first image projected based on the second image captured by multiple cameras for the first image, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of the multi-camera system in specific scenarios.

[0141] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0142] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0143] An embodiment of the present disclosure further provides a color calibration device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0144] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0145] Figure 4 1 is a block diagram of a device 1900 for color calibration according to an exemplary embodiment. For example, the device 1900 can be provided as a server. Figure 4 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0146] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0147] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0148] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0149] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0150] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0151] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0152] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0153] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0154] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0155] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0156] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A color calibration method, characterized in that: The method is used for a first server, and the method includes: Determining expected values ​​of camera parameters, and setting the camera parameters of the plurality of cameras to the same expected values; Controlling a display device to display at least one frame of a first image, and controlling multiple cameras to synchronously capture the first image, wherein the multiple cameras are configured to collaborate and capture images of the same scene from different perspectives during virtual filming, and the images from different perspectives are configured to be synthesized and rendered in real time; Analyzing, based on a second image captured by the multiple cameras for the first image, a difference between a color of the second image and a color of the first image projected; performing color calibration on the multiple cameras based on a difference between a color of the second image and a color of the first image; The controlling the display device to display at least one frame of the first picture and controlling the multiple cameras to synchronously capture the first picture includes: Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset duration, and controlling multiple cameras to synchronously capture each frame of the first image; In response to a camera not capturing a corresponding second image for one or more first image frames, controlling the display device to redisplay the one or more first image frames, and controlling the camera not capturing the corresponding second image to recapture the one or more first image frames; When the plurality of cameras all capture the second images including the same identifier, it is determined that the plurality of cameras all capture the corresponding second images for the first images including the same identifier.

2. The method according to claim 1, characterized in that The controlling the display device to display at least one frame of the first picture and controlling multiple cameras to simultaneously capture the first picture includes: Controlling the display device to display the first picture of the current frame, and controlling multiple cameras to synchronously capture the first picture of the current frame; The analyzing, based on the second picture captured by the multiple cameras for the first picture, a difference between a color of the second picture and a color of the first picture projected, includes: In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected corresponding frame first screens are obtained.

3. The method according to claim 1, characterized in that The analyzing, based on the second picture captured by the multiple cameras for the first picture, a difference between a color of the second picture and a color of the first picture projected, includes: In response to the plurality of cameras capturing the corresponding second images for each frame of the first image, differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding first images projected are analyzed respectively.

4. The method according to claim 1, wherein The controlling the display device to display at least one frame of the first picture includes: The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

5. The method according to claim 1, wherein The camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface (SDI) signal format, color temperature, aperture, and focal length.

6. The method according to claim 1, wherein The color calibrating the multiple cameras based on a difference between a color of the second picture and a color of the first picture projection includes: For each camera, a lookup table LUT corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

7. A color calibration device, characterized in that: The device is used for a first server, and the device includes: a determination module, configured to determine expected values ​​of camera parameters and set the camera parameters of multiple cameras to the same expected values; a control module, configured to control a display device to display at least one frame of a first image, and to control multiple cameras to synchronously capture the first image, wherein the multiple cameras are configured to work together during virtual shooting and capture images of the same scene from different perspectives, and the images from different perspectives are configured to be synthesized and rendered in real time; an analysis module, configured to analyze, based on a second picture captured by the multiple cameras for the first picture, a difference between a color of the second picture and a color of the first picture projected; A color calibration module, configured to perform color calibration on the multiple cameras based on a difference between a color of the second picture and a color of the first picture projection; The control module is used to: Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset duration, and controlling multiple cameras to synchronously capture each frame of the first image; a redisplay module for controlling the display device to redisplay the one or more first frames in response to a camera not capturing a corresponding second frame for the one or more first frames, and controlling the camera not capturing the corresponding second frame to recapture the one or more first frames; When the plurality of cameras all capture the second images including the same identifier, it is determined that the plurality of cameras all capture the corresponding second images for the first images including the same identifier.

8. A color calibration device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 6 when executing the instructions stored in the memory.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer-readable code, wherein when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 6.

Images