Color calibration method and device and storage medium

Abstract

The invention relates to a color calibration method and device and a storage medium. The method is applied to a first server, and comprises the following steps: controlling a display device to display at least one frame of first picture, and controlling a plurality of cameras to synchronously collect the first picture; based on a second picture acquired by the plurality of cameras for the first picture, analyzing the difference between the color of the second picture and the projection color of the first picture; and performing color calibration on the plurality of cameras based on the difference between the color of the second picture and the color projected by the first picture. According to the embodiment of the invention, the tedious process that a single camera shoots calibration pictures one by one can be avoided, the plurality of cameras are controlled to complete data acquisition at the same time, the working efficiency is further improved, the calibration time can be remarkably shortened through a parallel calibration mode, the efficiency of multi-camera color calibration is greatly improved, and the user experience is improved. The repeated operation of one-by-one calibration of a single camera is avoided, and the requirements for quick response and high consistency of a multi-camera system in a specific scene are met.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual shooting, and in particular to a color calibration method, device and storage medium. Background Art

[0002] With the rapid development of virtual shooting technology, the application of multi-camera systems has become particularly important in film and television production, virtual reality (VR) and augmented reality (AR). Virtual shooting scenes usually require multiple cameras to work together to build a virtual picture that highly restores the real world by shooting pictures of the same scene from different perspectives and synthesizing and rendering these pictures in real time. However, due to differences in camera hardware characteristics, sensor sensitivity, lens optical characteristics and lighting conditions, different cameras often produce color deviations when shooting the same scene. This color inconsistency will affect the naturalness of the splicing of virtual shooting pictures, resulting in obvious seams in the picture fusion, which will destroy the immersion and authenticity of the virtual scene.

[0003] At present, the traditional color calibration solution is usually to calibrate a single camera independently. However, the method of calibrating a single camera one by one is not only cumbersome, but also inefficient, and cannot guarantee that multiple cameras can complete consistent color calibration in a short time. In addition, calibrating each camera individually requires repeated settings and data collection operations, which cannot guarantee the accuracy of color calibration, is time-consuming and labor-intensive, and prone to errors, making it difficult to meet application scenarios with high real-time requirements. Summary of the invention

[0004] In view of this, the present disclosure proposes a color calibration method, device and storage medium.

[0005] According to one aspect of the present disclosure, a color calibration method is provided. The method is used for a first server, and the method includes:

[0006] Controlling a display device to display at least one frame of the first picture, and controlling multiple cameras to synchronously capture the first picture;

[0007] Analyze the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by the multiple cameras for the first picture;

[0008] Based on the difference between the color of the second picture and the color of the first picture, color calibration is performed on multiple cameras.

[0009] In a possible implementation, controlling a display device to display at least one frame of a first picture, and controlling a plurality of cameras to simultaneously capture the first picture, includes:

[0010] Control the display device to display the first picture of the current frame, and control multiple cameras to synchronously capture the first picture of the current frame;

[0011] Based on the second picture captured by multiple cameras for the first picture, the difference between the color of the second picture and the color of the first picture is analyzed, including:

[0012] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected screen of the current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected screen of the corresponding frames of the first screen are obtained.

[0013] In a possible implementation, controlling a display device to display at least one frame of a first picture, and controlling a plurality of cameras to synchronously capture the first picture, includes:

[0014] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time, and controlling multiple cameras to synchronously capture each frame of the first image;

[0015] Based on the second picture captured by multiple cameras for the first picture, the difference between the color of the second picture and the color of the first picture is analyzed, including:

[0016] In response to the fact that the multiple cameras capture the corresponding second images for each frame of the first image, the differences between the colors of the second images captured by the multiple cameras and the colors of the corresponding frame of the first image are analyzed respectively.

[0017] In a possible implementation, the method further includes:

[0018] In response to the presence of a camera that has not captured a corresponding second picture for one or more frames of a first picture, the display device is controlled to redisplay the one or more frames of the first picture, and the camera that has not captured the corresponding second picture is controlled to recapture the one or more frames of the first picture.

[0019] In a possible implementation, controlling a display device to display at least one frame of a first picture includes:

[0020] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0021] In a possible implementation, when the plurality of cameras all capture the second pictures including the same identifier, it is determined that the plurality of cameras all capture the corresponding second pictures for the first pictures including the same identifier.

[0022] In a possible implementation, before controlling the display device to display at least one frame of the first picture and controlling the multiple cameras to synchronously capture the first picture, the method further includes:

[0023] Desired values ​​of camera parameters are determined, and camera parameters of multiple cameras are set to the same desired values.

[0024] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface SDI signal format, color temperature, aperture, and focal length.

[0025] In a possible implementation, color calibration is performed on multiple cameras based on a difference between a color of the second picture and a color of the first picture projection, including:

[0026] For each camera, a lookup table LUT corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0027] According to another aspect of the present disclosure, a color calibration device is provided. The device is used for a first server, and the device includes:

[0028] A control module, used to control a display device to display at least one frame of a first image, and to control multiple cameras to synchronously capture the first image;

[0029] An analysis module, configured to analyze the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by the multiple cameras for the first picture;

[0030] The color calibration module is used to perform color calibration on multiple cameras based on the difference between the color of the second picture and the color of the first picture projection.

[0031] In a possible implementation, the control module is used to:

[0032] Control the display device to display the first picture of the current frame, and control multiple cameras to synchronously capture the first picture of the current frame;

[0033] Analysis modules for:

[0034] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected screen of the current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected screen of the corresponding frames of the first screen are obtained.

[0035] In a possible implementation, the control module is used to:

[0036] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time, and controlling multiple cameras to synchronously capture each frame of the first image;

[0037] Analysis modules for:

[0038] In response to the fact that the multiple cameras capture the corresponding second images for each frame of the first image, the differences between the colors of the second images captured by the multiple cameras and the colors of the corresponding frame of the first image are analyzed respectively.

[0039] In a possible implementation, the device further includes:

[0040] A redisplay module is used to control the display device to redisplay the one or more frames of the first picture in response to the existence of a camera that has not captured the corresponding second picture for one or more frames of the first picture, and to control the camera that has not captured the corresponding second picture to re-capture the one or more frames of the first picture.

[0041] In a possible implementation, the control module is used to:

[0042] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0043] In a possible implementation, when the plurality of cameras all capture the second pictures including the same identifier, it is determined that the plurality of cameras all capture the corresponding second pictures for the first pictures including the same identifier.

[0044] In a possible implementation, the device further includes:

[0045] The determination module is used to determine the expected values ​​of the camera parameters and set the camera parameters of the multiple cameras to the same expected values ​​before controlling the display device to display at least one frame of the first picture and controlling the multiple cameras to synchronously capture the first picture.

[0046] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface SDI signal format, color temperature, aperture, and focal length.

[0047] In a possible implementation, the color calibration module is used to:

[0048] For each camera, a lookup table LUT corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0049] According to another aspect of the present disclosure, a color calibration device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0050] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0051] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0052] According to the embodiment of the present disclosure, by controlling the display device to display at least one frame of the first picture through the first server, and controlling multiple cameras to synchronously capture the first picture, the tedious process of a single camera shooting calibration pictures one by one can be avoided, and multiple cameras can be controlled to complete data collection at the same time, which greatly reduces the color calibration time. The workload of human intervention can be reduced through automated synchronization operations, further improving work efficiency. In addition, the light change and color shift problems caused by the time difference can be reduced through synchronous acquisition, making the subsequent color difference calculation more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of multi-camera system in specific scenarios.

[0053] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0055] Figure 1 A structural diagram of a color calibration system according to an embodiment of the present disclosure is shown.

[0056] Figure 2 A flow chart of a color calibration method according to an embodiment of the present disclosure is shown.

[0057] Figure 3 A structural diagram of a color calibration device according to an embodiment of the present disclosure is shown.

[0058] Figure 4 is a block diagram of a device 1900 for color calibration according to an exemplary embodiment. DETAILED DESCRIPTION

[0059] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0060] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0061] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0062] With the rapid development of virtual shooting technology, the application of multi-camera systems has become particularly important in film and television production, virtual reality (VR) and augmented reality (AR). Virtual shooting scenes usually require multiple cameras to work together to build a virtual picture that highly restores the real world by shooting pictures of the same scene from different perspectives and synthesizing and rendering these pictures in real time. However, due to differences in camera hardware characteristics, sensor sensitivity, lens optical characteristics and lighting conditions, different cameras often produce color deviations when shooting the same scene. This color inconsistency will affect the naturalness of the splicing of virtual shooting pictures, resulting in obvious seams in the picture fusion, which will destroy the immersion and authenticity of the virtual scene.

[0063] At present, the traditional color calibration solution is usually to calibrate a single camera independently. However, the method of calibrating a single camera one by one is not only cumbersome, but also inefficient, and cannot guarantee that multiple cameras can complete consistent color calibration in a short time. In addition, calibrating each camera individually requires repeated settings and data collection operations, which cannot guarantee the accuracy of color calibration, is time-consuming and labor-intensive, and prone to errors, making it difficult to meet application scenarios with high real-time requirements.

[0064] In view of this, the present disclosure proposes a color calibration method, device and storage medium. The method can be used for the first server. By controlling the display device to display at least one frame of the first picture and controlling multiple cameras to synchronously capture the first picture, the tedious process of a single camera shooting calibration pictures one by one can be avoided, and multiple cameras can be controlled to complete data acquisition at the same time, which greatly reduces the color calibration time. The workload of human intervention can be reduced through automated synchronization operations, further improving work efficiency. In addition, the light change and color shift problems caused by the time difference can be reduced through synchronous acquisition, making the subsequent color difference calculation more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of multi-camera systems in specific scenarios.

[0065] Figure 1 FIG. 2 shows a structural diagram of a color calibration system according to an embodiment of the present disclosure. Figure 1As shown, the color calibration system may include a first server, a second server, a display device, and multiple cameras (such as camera 1 to camera 3 in the figure). The first server may be a main control machine, used to control multiple cameras and a second server; the second server may be a screen-mounting machine, used to receive the picture sent by the first server and send it to the display device for display; the display device may be used to display the picture sent by the second server; and cameras 1 to camera 3 may be used to capture the picture displayed by the display device. The present disclosure does not limit the number of cameras.

[0066] Among them, the server in the present disclosure generally refers to a device or system that provides data, resources or services. The first server and the second server can be different physical servers, and the present disclosure does not limit the type of server. The first server and the second server can also be the same server. When the first server and the second server are different servers, they can be connected by wire or wireless. The second server and the display device can be connected by wire or wirelessly. The display device can be a light emitting diode display (Light Emitting Diode, LED) display, a liquid crystal display (Liquid Crystal Display, LCD), etc., and the present disclosure does not limit this. The first server and cameras 1 to 3 can be connected by wire or wirelessly. Cameras 1 to 3 can be set at any position. The present disclosure does not display the types of cameras 1 to 3. Multiple cameras can be image acquisition devices of the same type or different types.

[0067] The color calibration system disclosed in the present invention can be used in application scenarios of multi-camera shooting. For example, in a virtual shooting scenario, the first server can control the second server to render the picture to be shot and display it on the display device, and control multiple cameras to work together to shoot different perspectives of the same scene, and synthesize and render these pictures in real time, so as to construct a virtual picture that highly restores the real world. In such scenarios of multi-camera collaborative shooting, different cameras often produce color deviations when shooting the same scene, and this color inconsistency will affect the shooting effect. The color calibration method disclosed in the present invention can be used to perform color calibration on multiple cameras before formal shooting to ensure that multiple cameras will not have color differences when capturing the same picture during the formal shooting process.

[0068] Figure 2 FIG. 4 is a flow chart showing a color calibration method according to an embodiment of the present disclosure. The method can be used for the first server as described above. Figure 2 As shown, the method may include:

[0069] Step S201, controlling a display device to display at least one frame of a first picture, and controlling a plurality of cameras to synchronously capture the first picture.

[0070] Before multiple cameras capture images, camera parameters of each camera may be uniformly configured first. The method may further include:

[0071] Desired values ​​of camera parameters are determined, and camera parameters of multiple cameras are set to the same desired values.

[0072] The camera parameters may include any one or more of camera color gamut, white balance, frame rate, Serial Digital Interface (SDI) signal format, color temperature, aperture, and focal length.

[0073] The expected value corresponding to each camera parameter can be pre-configured as needed, and the first server can send the pre-configured expected value corresponding to each camera parameter to each camera, so that the same camera parameter on each camera is configured as the same expected value.

[0074] Thus, the camera parameters of multiple cameras can be automatically set, and color calibration failures caused by missed settings and incorrect settings during manual settings can be reduced, thereby improving the accuracy and efficiency of the color calibration process.

[0075] The first server may determine the color to be projected on the first screen, wherein one frame of the first screen may include one color. In the process of color calibration, multiple colors are usually used, so these colors can be displayed separately through multiple frames of the first screen. In order to distinguish the colors of different frames of the first screen, the first server may also generate different identifiers for the colors to be projected, and these identifiers can be used to distinguish different colors to be projected. In step S201, it is possible to:

[0076] The first server sends the color of the screen to be projected and the identifier corresponding to the color to be projected to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color to be projected, and then sends it to the display device to display the first picture on the display device.

[0077] Among them, the identifier corresponding to the color to be projected can be in any form. For example, the identifier can be the RGB (red, green, blue) value of the color to be projected, or in the form of numbers, letters, etc., as long as the different colors to be projected can be distinguished.

[0078] The second server can generate the first picture by rendering the color to be projected and the corresponding logo, and the rendering process can be implemented based on the existing technology, so that the corresponding color and logo are displayed on the first picture. In the first picture obtained after rendering, the logo can be located at any position in the first picture.

[0079] After the first picture is displayed on the display device, the first server can control each camera to synchronously capture the first picture. Each camera can be set at any relative position of the display device (including any relative angle and any relative distance). The position of each camera can also be set to a symmetrical layout, for example, multiple cameras are distributed at equal angles around the display device and maintain a consistent distance from the display device to ensure that the pictures captured by all cameras are consistent.

[0080] After the display device displays the first picture, the current display status (for example, a status signal indicating that the first picture has been displayed) can be fed back to the second server through the control interface, and the second server can feed back the display status of the display device to the first server, so that the first server can know that the first picture has been displayed on the display device. The display device can also perform synchronous communication with the second server through a synchronization mechanism (for example, sending a synchronization signal to the second server) to feed back to the second server that the first picture has been displayed, and the second server can synchronize the synchronization signal fed back by the display device to the first server, so that the first server can know that the first picture has been displayed on the display device.

[0081] After the multiple cameras have captured the corresponding second images for the first images, the captured second images may be sent to the first server so that the first server can perform subsequent analysis.

[0082] Step S202, based on the second picture captured by multiple cameras for the first picture, analyze the difference between the color of the second picture and the color of the first picture projected.

[0083] The color of the first screen projection is used to generate the color of the screen to be projected for the first screen.

[0084] Among them, since it is usually necessary to display multiple frames of the first picture during color calibration, in this process, a possible implementation method can be that after multiple cameras complete the acquisition of a frame of the first picture, the first server performs a difference analysis on the frame of the first picture, and after the analysis is completed, multiple cameras acquire the next frame of the first picture, and the first server performs the analysis again, until the differences of all frames of the first picture are analyzed. In this case, in step S201, it is possible to:

[0085] Control the display device to display the first picture of the current frame, and control multiple cameras to synchronously capture the first picture of the current frame;

[0086] In step S202, it is possible to:

[0087] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected screen of the current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected screen of the corresponding frames of the first screen are obtained.

[0088] For the first picture of the current frame, if the first server does not receive the corresponding second picture captured by a certain camera within a preset time, for example, it receives the second pictures captured by camera 1 and camera 3 within the preset time but does not capture the second picture captured by camera 2, it can control the camera that has not captured the second picture (i.e., camera 2) to re-capture the first picture of the current frame to obtain the second pictures captured by all cameras for the first picture of the current frame.

[0089] When the first server performs analysis, for each camera, the difference between the color of the second picture captured by the camera and the color of the first picture of the current frame projected on the screen can be analyzed to obtain the analysis result of the camera. After analyzing each camera, the next color to be projected and the corresponding identifier are sent to the second server, so that the second server generates the next frame of the first picture and displays the next frame of the first picture on the display device, and controls multiple cameras to synchronously capture and analyze based on the captured second picture again. By iteratively executing the above process, the analysis results for multiple frames of the first picture corresponding to each camera can be obtained. For example, there are N frames of the first picture, and N analysis results can be obtained for each camera, each analysis result corresponds to 1 frame of the first picture, and each analysis result indicates the difference between the color of the second picture captured by the camera and the color of the first picture projected on the screen for the frame of the first picture.

[0090] In a possible implementation, in response to multiple cameras capturing the second picture for the current frame first picture, the first server can update the next frame first picture as the current frame first picture without waiting for the analysis of the difference between the color of the second picture captured by the multiple cameras and the color of the current frame first picture projected. In other words, the step of the first server updating the first picture and the step of the first server analyzing the color difference can also be performed asynchronously.

[0091] In another possible implementation, after multiple cameras have completed capturing the first images of each frame, the first server can then perform a difference analysis on the first images of each frame. In this case, in step S201, it is possible to:

[0092] The display device is controlled to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time length, and multiple cameras are controlled to synchronously capture each frame of the first image.

[0093] In step S202, it is possible to:

[0094] In response to the fact that the multiple cameras capture the corresponding second images for each frame of the first image, the differences between the colors of the second images captured by the multiple cameras and the colors of the corresponding frame of the first image are analyzed respectively.

[0095] The multiple frames of first images displayed continuously may be all the first images to be captured used in the color calibration process. The first server may control the display device through the second server to continuously display all the first images to be captured, wherein each frame of the first image is displayed for a preset duration (e.g., 20ms), and multiple cameras may be controlled to capture the first image at the same frequency, for example, once every 20ms, and multiple cameras may send the captured second images to the first server.

[0096] During continuous acquisition, one or more cameras may miss acquiring one or more first frames. The first server may determine whether there is a camera that fails to acquire the corresponding second frame for one or more first frames.

[0097] When the plurality of cameras all capture the second images including the same identifier, it can be determined that the plurality of cameras all capture the corresponding second images for the first images including the same identifier.

[0098] By determining whether the multiple cameras have captured the corresponding second images for each first image based on the identifier, the uniqueness and comparability of color data in the color calibration process can be achieved, and the accuracy of color calibration can be further improved.

[0099] The method may further include:

[0100] In response to the presence of a camera that has not captured a corresponding second picture for one or more frames of a first picture, the display device is controlled to redisplay the one or more frames of the first picture, and the camera that has not captured the corresponding second picture is controlled to recapture the one or more frames of the first picture.

[0101] By re-capturing the missed images, the missed images can be prevented from occurring during the color calibration process, thereby further improving the accuracy of the color calibration.

[0102] For example, the display device continuously displays three frames of first images, and the identifiers included in these three frames of first images are respectively (255, 0, 0), (0, 255, 0) and (0, 0, 255). When the first server does not receive the second image including the identifier (0, 0, 255) sent back by camera 2, and does not receive the second image including the identifier (0, 255, 0) and (0, 0, 255) sent back by camera 3, it can be considered that camera 2 missed capturing the corresponding second image for the third frame of the first image, and camera 3 missed capturing the corresponding second images for the second and third frames of the first image. At this time, the first server can control the display device through the second server to continuously display the first images of the 2nd and 3rd frames that were missed (i.e., the first images including the identifiers (0, 255, 0) and (0, 0, 255)), and when the display device displays the first image of the 2nd frame, control the camera 3 that missed capturing the image to re-capture the corresponding second image for the first image; when the display device displays the first image of the 3rd frame, control the cameras 2 and 3 that missed capturing the image to re-capture the corresponding second images for the first image.

[0103] Through the above process, the first server can receive multiple frames of second pictures captured by M cameras (M is the total number of cameras), wherein each camera captures at least N frames of second pictures (N is the total number of displayed first pictures), and each of the N frames of second pictures is associated with one frame of first picture.

[0104] When the first server performs analysis, the difference between the color of each second picture captured by the camera and the color of the corresponding first picture projection can be analyzed for each camera, so as to obtain the analysis results of the multiple frames of the first picture corresponding to each camera. The analysis results of the multiple frames of the first picture corresponding to each camera can be referred to the examples in the above embodiments.

[0105] In the above process of controlling multiple cameras to synchronously capture the first picture, for one frame of the first picture, the first server can also control multiple cameras to capture multiple times respectively, where the number of captures can be set as needed, so that for one frame of the first picture, each camera can capture multiple frames of corresponding second pictures.

[0106] In this case, when the first server analyzes the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, for a certain frame of the first picture, the first server can analyze the difference between the color of the first picture projected based on the second picture captured multiple times. For example, for the third frame of the first picture, if camera 2 captures 4 frames of the second picture, the first server can first average the colors of the 4 frames of the second picture, and then analyze the difference between the averaged colors of the 4 frames of the second picture and the corresponding color of the third frame of the first picture projected.

[0107] In this way, multiple cameras can be calibrated synchronously, which can improve work efficiency and picture quality in corresponding shooting scenarios and provide users with a more realistic and smooth experience.

[0108] In the analysis results of the multiple frames of the first screen corresponding to each of the above cameras, the difference between the color of the second screen and the color of the first screen projection may include the difference between the RGB color of the second screen and the RGB color of the first screen projection. The RGB color of the first screen projection is the RGB color used to generate the color to be projected of the first screen.

[0109] The value of each RGB component (red, green, blue) in the RGB color of the first screen can be subtracted from the corresponding color component value in the RGB color of the second screen, and the color difference under each RGB component is used as the difference between the color of the second screen and the color of the first screen. For example, the color of the first screen is (0,255,255), if the color of the second screen captured by the camera for (0,255,255) is (10,220,220), then the difference between the color of the second screen and the color of the first screen can be (-10,5,5).

[0110] Step S203, color calibration is performed on multiple cameras based on the difference between the color of the second picture and the color of the first picture.

[0111] According to the embodiment of the present disclosure, by controlling the display device to display at least one frame of the first picture through the first server, and controlling multiple cameras to synchronously capture the first picture, the tedious process of a single camera shooting calibration pictures one by one can be avoided, and multiple cameras can be controlled to complete data collection at the same time, which greatly reduces the color calibration time. The workload of human intervention can be reduced through automated synchronization operations, further improving work efficiency. In addition, the light change and color shift problems caused by the time difference can be reduced through synchronous acquisition, making the subsequent color difference calculation more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of multi-camera system in specific scenarios.

[0112] Wherein, color calibration may be performed for each camera respectively. In step S203, the following steps may be performed:

[0113] For each camera, a lookup table (LUT) corresponding to the camera is generated based on the difference between the color of the projected first picture of each frame and the color of the corresponding second picture captured by the camera.

[0114] LUT can be used to perform color calibration on the camera. LUT can represent the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0115] For example, the LUT corresponding to each camera may include N pairs of mapping relationships, which are determined according to the difference between the color of the N-frame first screen projected and the color of the corresponding second screen captured by the camera, and each pair of mapping relationships associates the color captured by the camera with the color value that needs to be adjusted (i.e., the calculated color difference, used to compensate for the color deviation captured by the camera). For example, for the standard color (0,255,255), if the color of the second screen captured by the camera for this color is (10,220,220) based on the above, and the corresponding difference with the standard color (i.e., the color of the first screen projected) is (-10,5,5), then a pair of mapping relationships can be generated, including the corresponding RGB color (10,220,220) captured by the camera, and the color value (-10,5,5) that needs to be adjusted corresponding to each color component.

[0116] During the actual shooting process, the color captured by the camera can be added to the corresponding color value that needs to be adjusted in the LUT (for example, for the color (10,220,220) captured by the camera, adjustment can be made according to the corresponding color value (-10,5,5) that needs to be adjusted in the LUT) to obtain the output color after color calibration (for example, add (10,220,220) to (-10,5,5) to get (0,255,255)).

[0117] In this way, the color differences between different cameras during the actual shooting process can be eliminated, ensuring that the final output image color is consistent and the image stitching and synthesis effects are guaranteed.

[0118] Figure 3 FIG. 1 shows a structural diagram of a color calibration device according to an embodiment of the present disclosure. The device is used in a first server, such as Figure 3 As shown, the device comprises:

[0119] The control module 301 is used to control the display device to display at least one frame of the first picture, and control multiple cameras to synchronously capture the first picture;

[0120] An analysis module 302 is used to analyze the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture;

[0121] The color calibration module 303 is used to perform color calibration on multiple cameras based on the difference between the color of the second picture and the color of the first picture.

[0122] In a possible implementation, the control module 301 is configured to:

[0123] Control the display device to display the first picture of the current frame, and control multiple cameras to synchronously capture the first picture of the current frame;

[0124] The analysis module 302 is used to:

[0125] In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screens captured by the multiple cameras and the colors of the projected screen of the current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected screen of the corresponding frames of the first screen are obtained.

[0126] In a possible implementation, the control module 301 is configured to:

[0127] Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time, and controlling multiple cameras to synchronously capture each frame of the first image;

[0128] The analysis module 302 is used to:

[0129] In response to the fact that the multiple cameras capture the corresponding second images for each frame of the first image, the differences between the colors of the second images captured by the multiple cameras and the colors of the corresponding frame of the first image are analyzed respectively.

[0130] In a possible implementation, the device further includes:

[0131] A redisplay module is used to control the display device to redisplay the one or more frames of the first picture in response to the existence of a camera that has not captured the corresponding second picture for one or more frames of the first picture, and to control the camera that has not captured the corresponding second picture to re-capture the one or more frames of the first picture.

[0132] In a possible implementation, the control module 301 is configured to:

[0133] The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

[0134] In a possible implementation, when the plurality of cameras all capture the second pictures including the same identifier, it is determined that the plurality of cameras all capture the corresponding second pictures for the first pictures including the same identifier.

[0135] In a possible implementation, the device further includes:

[0136] The determination module is used to determine the expected values ​​of the camera parameters and set the camera parameters of the multiple cameras to the same expected values ​​before controlling the display device to display at least one frame of the first picture and controlling the multiple cameras to synchronously capture the first picture.

[0137] In a possible implementation, the camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface SDI signal format, color temperature, aperture, and focal length.

[0138] In a possible implementation, the color calibration module 303 is used to:

[0139] For each camera, a lookup table LUT corresponding to the camera is generated based on the difference between the color of the projected first screen of each frame and the color of the corresponding second screen captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

[0140] According to the embodiment of the present disclosure, by controlling the display device to display at least one frame of the first picture through the first server, and controlling multiple cameras to synchronously capture the first picture, the tedious process of a single camera shooting calibration pictures one by one can be avoided, and multiple cameras can be controlled to complete data collection at the same time, which greatly reduces the color calibration time. The workload of human intervention can be reduced through automated synchronization operations, further improving work efficiency. In addition, the light change and color shift problems caused by the time difference can be reduced through synchronous acquisition, making the subsequent color difference calculation more accurate, which helps to improve the overall calibration accuracy. Therefore, by analyzing the difference between the color of the second picture and the color of the first picture projected based on the second picture captured by multiple cameras for the first picture, so as to perform color calibration on multiple cameras, it is possible to significantly shorten the calibration time through parallel calibration, greatly improve the efficiency of multi-camera color calibration, avoid the repeated operation of calibrating a single camera one by one, and meet the requirements of fast response and high consistency of multi-camera system in specific scenarios.

[0141] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0142] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0143] The embodiment of the present disclosure further proposes a color calibration device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0144] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0145] Figure 4 1 is a block diagram of a device 1900 for color calibration according to an exemplary embodiment. For example, the device 1900 may be provided as a server. Figure 4 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0146] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0147] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0148] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0149] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0150] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0151] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0152] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0153] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0154] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0155] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0156] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A color calibration method, characterized in that: The method is used for a first server, and the method comprises: Controlling a display device to display at least one frame of a first picture, and controlling a plurality of cameras to synchronously capture the first picture; Based on the second picture captured by the multiple cameras for the first picture, analyzing the difference between the color of the second picture and the color of the first picture projected; Based on the difference between the color of the second picture and the color of the first picture projection, the multiple cameras are color calibrated.

2. The method according to claim 1, characterized in that The controlling the display device to display at least one frame of the first picture and controlling multiple cameras to simultaneously capture the first picture includes: Control the display device to display the first picture of the current frame, and control multiple cameras to synchronously capture the first picture of the current frame; The analyzing the difference between the color of the second picture captured by the multiple cameras for the first picture and the color of the first picture projected includes: In response to multiple cameras capturing the second screen for the current frame first screen, the differences between the colors of the second screen captured by the multiple cameras and the colors of the projected screen of the current frame first screen are analyzed respectively, and the next frame first screen is updated as the current frame first screen, and the steps of controlling the display device to display the current frame first screen and subsequent steps are repeated until the differences between the colors of all the second screens captured by the multiple cameras and the colors of the projected screen of the corresponding frames of the first screen are obtained.

3. The method according to claim 1, characterized in that The controlling the display device to display at least one frame of the first picture and controlling multiple cameras to synchronously capture the first picture includes: Controlling a display device to continuously display multiple frames of the first image, wherein each frame of the first image is displayed for a preset time, and controlling multiple cameras to synchronously capture each frame of the first image; The analyzing the difference between the color of the second picture captured by the multiple cameras for the first picture and the color of the first picture projected includes: In response to the fact that the plurality of cameras capture the corresponding second images for each frame of the first image, the differences between the colors of the second images captured by the plurality of cameras and the colors of the corresponding frame of the first image are respectively analyzed.

4. The method according to claim 3, characterized in that The method further comprises: In response to the presence of a camera that has not captured a corresponding second picture for one or more frames of a first picture, the display device is controlled to redisplay the one or more frames of the first picture, and the camera that has not captured the corresponding second picture is controlled to recapture the one or more frames of the first picture.

5. The method according to claim 1, characterized in that The controlling the display device to display at least one frame of the first picture includes: The color of the screen to be projected and the identifier corresponding to the color of the screen to be projected are sent to the second server, so that the second server generates a first picture based on the color of the screen to be projected and the identifier corresponding to the color of the screen to be projected, and then sends it to the display device to display the first picture on the display device.

6. The method according to claim 5, characterized in that When the plurality of cameras all capture the second pictures including the same identifier, it is determined that the plurality of cameras all capture the corresponding second pictures for the first pictures including the same identifier.

7. The method according to claim 1, characterized in that Before controlling the display device to display at least one frame of the first picture and controlling the multiple cameras to synchronously capture the first picture, the method further includes: Desired values ​​of camera parameters are determined, and the camera parameters of the plurality of cameras are set to the same desired values.

8. The method according to claim 6, characterized in that The camera parameters include any one or more of camera color gamut, white balance, frame rate, serial digital interface SDI signal format, color temperature, aperture, and focal length.

9. The method according to claim 1, characterized in that: The color calibration of the multiple cameras based on the difference between the color of the second picture and the color of the first picture projection includes: For each camera, a lookup table LUT corresponding to the camera is generated based on the difference between the color of the projected first picture of each frame and the color of the corresponding second picture captured by the camera. The LUT is used to color calibrate the camera. The LUT represents the mapping relationship between the input color captured by the camera and the output color after color calibration.

10. A color calibration device, characterized in that: The device is used for a first server, and the device includes: A control module, used to control a display device to display at least one frame of a first picture, and control multiple cameras to synchronously capture the first picture; An analysis module, configured to analyze, based on a second picture captured by the multiple cameras for the first picture, a difference between a color of the second picture and a color of the first picture projected; The color calibration module is used to perform color calibration on the multiple cameras based on the difference between the color of the second picture and the color of the first picture projection.

11. A color calibration device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 9 when executing the instructions stored in the memory.

12. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 9.

Images