Color lookup table establishment method, system, device, product and storage medium

Abstract

This specification provides a method, system, device, product and storage medium for establishing a color lookup table. The virtual shooting system includes a display screen and multiple shooting devices, including a reference shooting device and a shooting device to be compensated; the original colors of each virtual scene displayed on the display screen by the shooting device are obtained, and the corresponding virtual scene acquisition colors are respectively collected, to establish a virtual scene disparity fitting LUT of the shooting device; the real scene acquisition colors collected after the shooting device shoots each real scene card are obtained, and for the shooting device to be compensated, the real scene disparity fitting LUT of the shooting device to be compensated is established according to the relationship between the real scene acquisition colors of the reference shooting device and the shooting device to be compensated; the virtual scene disparity fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed corresponding to the reference shooting device; the virtual scene disparity fitting LUT and the real scene disparity fitting LUT of the shooting device to be compensated are used to perform color calibration on the content to be displayed corresponding to the shooting device to be compensated.

Description

Technical Field

[0001] This specification relates to the field of virtual photography technology, and in particular to a method, system, device, product, and storage medium for establishing a color lookup table. Background Art

[0002] The key principle of virtual filming is to digitize the background and render it in real time onto a set of large displays. These displays are arranged around the actual performance area, where physical props can be placed and actors can perform within the filming area. The displays can adjust their content in real time as needed to present a realistic virtual background that blends in with the actual performance area.

[0003] In virtual shooting tasks that require capturing both the screen and the real scene at the same time, there are always color differences when different shooting devices capture the same scene, requiring a lot of time to manually align the virtual and real scenes. Summary of the Invention

[0004] To overcome the problems existing in the related art, this specification provides a method, system, device, product and storage medium for establishing a color lookup table.

[0005] According to a first aspect of an embodiment of the present specification, a method for establishing a color lookup table (LUT) is provided. The method is applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The method includes:

[0006] For each camera, after obtaining the virtual scene captured colors corresponding to each virtual scene original color displayed on the display screen by the camera, a virtual scene color difference fitting LUT of the camera is established, where the virtual scene color difference fitting LUT represents a mapping relationship from the virtual scene captured colors to the virtual scene original colors;

[0007] Acquire each real scene captured by each shooting device after shooting each real scene card, and for each shooting device to be compensated, establish a real scene color difference fitting LUT for the shooting device to be compensated based on the relationship between each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated;

[0008] The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen;

[0009] The virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

[0010] According to a second aspect of the embodiments of this specification, a color calibration method is provided. The method is applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The method includes:

[0011] Acquiring a virtual scene to be displayed on the display screen;

[0012] After determining the to-be-displayed content of the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using a virtual color disparity fitting LUT of the reference shooting device;

[0013] After determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color difference fitting LUT and the real color difference fitting LUT of the to-be-compensated shooting device;

[0014] The virtual image disparity fitting LUT and real image disparity fitting LUT of the shooting device to be compensated, and the virtual image disparity fitting LUT of the reference shooting device are obtained through the steps of the method described in the first aspect.

[0015] According to a third aspect of the embodiments of this specification, a virtual shooting system is provided, comprising a main control terminal, a rendering engine, a display screen for displaying a virtual scene, and a plurality of shooting devices, wherein the plurality of shooting devices include a reference shooting device and a shooting device to be compensated;

[0016] The main control end is used to implement the steps of the method described in the first aspect; the rendering engine is used to implement the steps of the method described in the second aspect.

[0017] According to a fourth aspect of the embodiments of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method embodiments described in the first or second aspect are implemented.

[0018] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method embodiments described in the first or second aspect are implemented.

[0019] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0020] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0021] In an embodiment of the present specification, for each camera, after obtaining the virtual scene captured colors corresponding to each virtual scene original color displayed on the display screen by the camera, a virtual scene disparity fitting LUT is established for the camera. The virtual scene disparity fitting LUT represents a mapping relationship from virtual scene captured colors to virtual scene original colors. Therefore, the virtual scene disparity fitting LUT of each camera can fit the color difference between the captured colors of the camera itself and the display colors of the display screen. This embodiment also performs real scene card sampling and is designed to establish a real scene disparity fitting LUT for each camera to be compensated based on the relationship between each real scene captured color of the reference camera and each real scene captured color of each camera to be compensated, taking the reference camera as the standard. Thus, the reference camera uses the virtual scene disparity fitting LUT to perform color calibration on the content to be displayed corresponding to the reference camera. Each camera to be compensated then uses the virtual scene disparity fitting LUT and the real scene disparity fitting LUT of the camera to be compensated to perform color calibration on the content to be displayed in the display area corresponding to the camera to be compensated. In this way, virtual and real alignment can be achieved for all cameras. It can be seen that this embodiment only requires two rounds of color card sampling to achieve virtual-real alignment of all shooting devices, greatly reducing labor costs and uncertainty caused by manual subjective visual modification.

[0022] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1A is a schematic diagram of a virtual shooting scene according to an exemplary embodiment of this specification.

[0024] Figure 1B This is a schematic diagram of a virtual shooting link according to an exemplary embodiment of the present specification.

[0025] Figure 1C This is another schematic diagram of a virtual shooting link according to an exemplary embodiment of this specification.

[0026] Figure 2A This is a flowchart of a method for establishing a color lookup table LUT according to an exemplary embodiment of this specification.

[0027] Figure 2B is a schematic diagram of another virtual shooting scene according to an exemplary embodiment of this specification.

[0028] Figure 2CThis is another schematic diagram of a virtual shooting link according to an exemplary embodiment of this specification.

[0029] Figure 3 This is a flowchart of a method for establishing a color lookup table LUT according to an exemplary embodiment of this specification.

[0030] Figure 4 This is a hardware structure diagram of a computer device where a device for establishing a color lookup table LUT / color calibration device is located according to an exemplary embodiment of this specification.

[0031] Figure 5 This is a block diagram of a device for establishing a color lookup table (LUT) according to an exemplary embodiment of this specification.

[0032] Figure 6 This is a block diagram of a device for establishing a color lookup table (LUT) according to an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0034] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0035] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0036] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0037] Virtual filming is a virtual shooting technology that uses display screens to create a virtual background. Combining real-time rendering and LED (Light Emitting Diode) display technology, it can present a realistic virtual environment in real time on set, replacing traditional green-screen or blue-screen filming. The main principle of virtual filming is to digitize the background and render it in real time onto a set of large display screens. These screens are arranged around the actual performance area, where physical props can be placed and actors can perform. The display screens can adjust the display content in real time as needed to present a realistic virtual background that blends in with the actual performance area.

[0038] like Figure 1A FIG. 1 is a schematic diagram of a virtual filming scene according to an exemplary embodiment of the present specification. The virtual filming scene may include a virtual filming system composed of one or more computer devices. As an example, the virtual filming system may include a combination of one or more of the following devices: one or more host computers 011, one or more rendering devices 021 (also referred to as screen-up devices), one or more broadcast control processing devices 031, a display screen 040 (in actual scenarios, the display screen 040 may be composed of one or more display screens. For example, the figure shows three display screens included in the display screen 040: display screen 041, display screen 042, and display screen 043), and one or more filming devices 051. The number of each type of device can be flexibly configured according to actual needs and is not limited in this embodiment. In actual applications, the virtual filming system may also include other devices, such as mobile terminals or network devices, as needed, which is not limited in this embodiment.

[0039] Optionally, each host computer 011 can be connected to one or more rendering devices 021. The specific connection method can be selected based on actual needs and device compatibility. For example, a wired or wireless connection can be established via a local area network or the Internet, and communication can be performed using a network transmission protocol. For example, the host computer can send various control instructions to the connected rendering device, such as control instructions containing specific image information.

[0040] Optionally, each rendering device 021 can be connected to one or more broadcast control processing devices 031; the specific connection method can be selected according to actual needs and device compatibility. As an example, it can include DP (DisplayPort, a digital display interface standard) connection, which can be used to transmit high-quality audio and video signals. It can also be HDMI (High-Definition Multimedia Interface, a high-definition digital audio and video interface standard), which can combine audio, video and control signals on a single cable for transmission. As an example, the rendering device 021 can send a variety of control instructions to the broadcast control processing device connected to it. For example, the rendering device can act as an image signal source, send control instructions containing rendered images, and so on.

[0041] In actual applications, the broadcast control processing device 031 is optional and may not be configured in some scenarios. Optionally, each broadcast control processing device 031 can be connected to one or more display screens 040; the specific connection method can be selected according to actual needs and device compatibility. As an example, it can also include DP or HDMI connection, and can also include USB (Universal Serial Bus) or network connection, etc. The broadcast control processing device can be used to control and manage the display screen connected to it. As an example, the broadcast control processing device 031 can be used for data transmission and decoding, such as the broadcast control processing device 031 can receive signals from external sources (such as screen display machines, computers, mobile terminals or media players, etc.) and decode them into a format suitable for display on the display screen; it can also be used for display control, such as overall control and scheduling of the display screen, including brightness adjustment, color correction, grayscale control, etc.; it can also be used for partition management of the display screen, and the display screen can be divided into multiple independent areas, each area can display different content.

[0042] Optionally, each host computer 011 can be connected to one or more camera devices 051. The specific connection method can be selected based on actual needs and device compatibility. For example, it can include wired connections such as HDMI or SDI (Serial Digital Interface, a digital video transmission standard), and can also include wireless connections such as Wi-Fi (Wireless Fidelity) or RF (Radio Frequency). The camera device 051 can transmit the captured data to the host computer.

[0043] Optionally, the display screen 040 may be an LED screen, an LCD screen, or other types, and may be a curved screen or a flat screen. It should be understood that those skilled in the art can customize the type, quantity, size, resolution, and other aspects of the display screen in the virtual shooting system according to actual needs, and this embodiment of the specification does not limit this. It should be understood that this embodiment of the specification does not limit the communication connection method between devices.

[0044] In some virtual shooting scenes, multiple shooting devices are required to shoot simultaneously; and in virtual shooting scenes that need to shoot both the screen and the real scene at the same time, there will always be color differences when different shooting devices shoot the same scene, resulting in a lot of time spent on realigning the virtual scene and the real scene each time the shooting device is changed, which relies on subjective manual labor.

[0045] In today's virtual filming workflow, crews typically allocate only a few hours to adjust the equipment and color. Color alignment between the virtual and real scenes takes up the most time. This is because if the virtual background on the screen doesn't match the real foreground in color, it can cause bloopers, necessitating meticulous manual adjustments. Color differences, known as camera chromatic aberration, occur when shooting scenes with different colors using the same camera, as well as when shooting the same scene with different cameras.

[0046] To overcome the inherent color difference between the virtual and real scenes of the shooting equipment itself, the inherent color difference between the virtual scenes of different shooting equipment, and the inherent color difference between the real scenes of different shooting equipment, staff need to perform detailed manual comparison to modify the color palette. However, the color mapping relationship between the virtual scene and the real scene, and between the virtual scene and the virtual asset editor is not a linear relationship, and manual adjustment is difficult. It is difficult to achieve full color gamut alignment using a simple RGB three-channel palette.

[0047] like Figure 1B As shown, it is a schematic diagram of a virtual shooting link in a virtual shooting scene. The shooting equipment directly shoots the virtual scene on the display screen and the real scene in the venue. At this time, the virtual scenes and real scenes captured by different shooting equipment are different. The only way is to manually align the virtual and real scenes of each shooting equipment one by one. In addition, due to the complex spectral response and color relationship between the shooting equipment screen and the color palette of the virtual asset editor and the venue lighting, manual adjustment is difficult to completely align them. It is also necessary to modify the shooting results of each shooting equipment during post-processing, which is quite time-consuming and labor-intensive.

[0048] like Figure 1C As shown, it is another schematic diagram of virtual shooting link in the virtual shooting scene. Figure 1BOn the basis of the previous method, a virtual-real alignment scheme that relies on virtual scene card sampling is added. By sampling the virtual scene card, the virtual scene images captured by the shooting equipment are restored to the same. This can effectively reduce the difficulty of manual virtual-real alignment when the color difference between the shooting devices is not large. However, manual alignment is still difficult when the color difference between the shooting devices is large. Moreover, when there are a large number of shooting devices or the shooting devices need to be replaced frequently, manual adjustment of each shooting device is still a considerable workload.

[0049] Based on this, the present embodiment provides a method for establishing a color lookup table (LUT). This method only requires two rounds of color card sampling to achieve virtual-real alignment for all camera devices, significantly reducing labor costs and the uncertainty caused by subjective visual modification. The following describes the present embodiment in detail.

[0050] like Figure 2A As shown, Figure 2A This is a flowchart of a method for establishing a color lookup table (LUT) according to an exemplary embodiment of this specification. The method can be applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The method may include the following steps:

[0051] In step 202, for each shooting device, after obtaining the virtual scene captured colors corresponding to the original colors of each virtual scene displayed on the display screen by the shooting device, a virtual scene color difference fitting LUT of the shooting device is established.

[0052] The virtual scene disparity fitting LUT represents a mapping relationship from the virtual scene captured color to the virtual scene original color.

[0053] In step 204, each real scene captured color captured by each shooting device after shooting each real scene color card is obtained, and for each shooting device to be compensated, a real scene color difference fitting LUT of the shooting device to be compensated is established based on the relationship between each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated.

[0054] The virtual color disparity fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen.

[0055] The virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

[0056] As an example, multiple shooting devices may be used for virtual shooting in a virtual shooting scene. The number of shooting devices may be configured according to the actual shooting scene, and this embodiment does not limit this. The multiple shooting devices in this embodiment include a reference shooting device and a shooting device to be compensated. The reference shooting device may be one, and the shooting device to be compensated may be one or more. You can arbitrarily select one from the multiple shooting devices as the reference shooting device. When shooting, the framing ranges of different shooting devices may not overlap, that is, different shooting devices may shoot different areas. For example, Figure 2B In the virtual shooting scene diagram shown, camera 1 and camera 2 have different framing ranges, capturing different areas of the display screen. Typically, the display screen is used to render and display the overall virtual shooting scene. Since different cameras capture different areas of the display screen, the corresponding display content is generally different for each camera. However, the color style of different areas of the virtual scene is generally consistent.

[0057] The method of this embodiment can be applied to any computer device with computing capabilities in the virtual shooting system. In some examples, the method of this embodiment can be applied to the main control terminal in the virtual shooting system, which can be a computer running on Figure 1A The software program on the host computer in the illustrated embodiment, wherein the host computer can specifically be a computer device, including but not limited to a server, a cloud server, a server cluster, a tablet computer, a personal digital assistant, a laptop computer, or a desktop computer.

[0058] As an example, colors in this embodiment can be represented using color values, such as color values in the RGB (Red, Green, Blue) color space, where each color value includes color classifications for the three RGB channels. For example, in the color values used by a display screen, each color channel can have a 10-bit value ranging from 0 to 1023. Optionally, the color accuracy of the image captured by the camera will be higher. In other examples, the use of color values in other color spaces, such as Lab or YUV, is also optional and is not limited in this embodiment.

[0059] As an example, the original color of each virtual scene can be predetermined; for example, multiple different values can be uniformly sampled for each color channel to constitute the original color of all virtual scenes. The sampled values can be flexibly configured, for example, they can be values such as 64, which is not limited in this embodiment.

[0060] Each virtual scene original color can be displayed on the display screen in a certain order. Each camera can capture each virtual scene original color displayed on the display screen, and the captured virtual scene color captured by the camera can be obtained from the captured data of the camera. Each virtual scene original color is equivalent to a virtual scene card, which can be obtained through asset editing tools such as rendering engines (such as Unreal Engine (UE)). Therefore, a large number of virtual scene cards can be used, allowing for sampling to obtain higher-level results.

[0061] As an example, the virtual scene colors captured by the shooting device can be arranged according to the display order of the original colors of the virtual scenes displayed on the display screen to obtain a forward LUT from the original colors of the virtual scenes to the captured colors of the virtual scenes; the forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain the virtual scene disparity fitting LUT of the shooting device.

[0062] Specifically, after the camera collects the colors of each virtual scene corresponding to each virtual scene card, it can arrange them in a sampling order, and the arrangement result is equivalent to the high-frequency sampling of the forward color mapping space of the camera;

[0063] For example, the set of original virtual scene colors displayed on the display is M, which contains 262,144 virtual scene original colors from M_1 to M_262144. The camera 1 captures each color in the set M, and the resulting virtual scene acquisition color set is N, which also contains 262,144 virtual scene acquisition colors from N_1 to N_262144. After acquisition, a forward LUT representing the mapping relationship from M to N can be obtained.

[0064] Furthermore, for each actual captured color (i.e., virtual scene acquisition color) that is equal to the input value of the sampling point in the virtual scene disparity fitting LUT to be established, the mapping relationship between the virtual scene acquisition color of the shooting device and the original color of the virtual scene can be determined; for the actual captured color that is not equal to the input value of the sampling point, interpolation calculation can be used to predict the original color of the virtual scene corresponding to the actual captured color.

[0065] For example, the forward LUT is inverted, that is, the mapping relationship from N to M is represented; the virtual scene difference fitting LUT that needs to be established can be used in the rendering engine. The rendering engine has certain format requirements for the LUT. It is assumed that the virtual scene difference fitting LUT to be established is a LUT that represents the mapping relationship from N' to M'. The color set of N' is not necessarily the same as the virtual scene acquisition color set N. For example, some colors in N' exist in the set N. These colors can directly find the corresponding virtual scene original colors from the set M. However, for the virtual scene acquisition colors in N' but not in the set N, the corresponding virtual scene original colors can be calculated by interpolation.

[0066] For example, based on the rendering engine's LUT format requirements, a virtual scene disparity fitting LUT is established to represent the mapping relationship from the virtual scene acquisition color set N' to the virtual scene original color M'. Assume that N' contains 262,144 colors, from n'_1 to n'_262,144. The virtual scene acquisition color set captured by the camera for each virtual scene card is set N, and these two sets are likely different. Each color in set N' can be checked to see if it exists in set N. Assuming n'_1 is the same as a color in set N, such as N_i, the color corresponding to N_i can be determined in set M. Assuming it is M_j, the mapping relationship between n'_1 and M_j is determined. Similarly, for n'_2, this color is different from every color in set N. Therefore, the original virtual scene color mapped by n'_2 needs to be calculated through interpolation. The same applies to other colors in set N'.

[0067] Since the arrangement result of the virtual scene captured colors corresponding to the original colors of each virtual scene by each camera is equivalent to the high-frequency sampling sequence of the forward color mapping space of the camera, the high-frequency sampling sequence can be reversely interpolated to obtain a reverse LUT between the camera and the virtual scene on the display screen. Through this reverse LUT, unified management of the virtual scene color is achieved, so that the virtual scenes in the pictures of all camera devices are consistent.

[0068] By interpolating the high-frequency sampling sequence of the virtual scene, the high-frequency sampling sequence can be reorganized into a virtual scene disparity fitting LUT in a standard format. Specifically, the forward LUT can be inverted using an inverse interpolation algorithm. The inverse operation is equivalent to solving a set of equations that outputs an equal output to the input itself after first mounting the inverse LUT and then the forward LUT for each given input value. The inverse interpolation algorithm can be used to solve the problem by first calculating the local inverse LUT and then gradually updating it to the global level to obtain the global inverse LUT. When there are irreversible points in the color mapping LUT, the theoretical color values of the irreversible points are filled in using an interpolation algorithm. Finally, the global inverse LUT is expanded to a standard format LUT that can be read by the device, that is, a standard format LUT that can be read by the rendering engine.

[0069] like Figure 2CAs shown, this is another schematic diagram of a virtual shooting link shown in this specification according to an exemplary embodiment. The virtual scene disparity fitting LUT obtained in the above embodiment can realize the unified management of virtual scene colors, so that the virtual scenes in the shooting pictures of all shooting devices are consistent, thereby aligning the virtual scene disparity between the shooting devices. For example, assuming that each shooting device does not need to shoot the real scene but only needs to shoot the display screen, that is, only needs to shoot the virtual scene, a virtual asset editor such as Unreal Engine can generate a virtual scene to be displayed on the display screen, and different shooting devices shoot different display areas of the display screen, and each shooting device has corresponding content to be displayed. The virtual scene disparity fitting LUT of each shooting device can be used to perform color calibration on the content to be displayed corresponding to each shooting device. For example, each frame of the virtual scene to be displayed contains multiple pixels, and each pixel has a color (i.e., pixel value). Taking the reference shooting device as an example, the pixel value of each pixel contained in the corresponding content to be displayed is the color of each pixel contained in the picture captured by the reference shooting device. However, there is a color difference between the color displayed on the display screen and the color captured by the reference shooting device. For the color of each pixel in the picture captured by the reference shooting device, the corresponding target color can be obtained from the virtual scene difference fitting LUT. The target color of each pixel in the picture captured by the reference shooting device is used as the color calibration result of the content to be displayed, and is used to display it on the display area corresponding to the reference shooting device on the display screen. For example, let's assume the color of the virtual scene to be displayed (the original virtual scene color) of the reference camera is (60, 100, 150). To achieve the color of the captured image (the captured virtual scene color) of (60, 100, 150), this color needs to be calibrated. Specifically, the color (60, 100, 150) is used as the input of the virtual scene disparity fitting LUT for the reference camera. The corresponding color is (56, 108, 142). The color (56, 108, 142) is then used as the calibration result for the color (60, 100, 150) and displayed in the display area corresponding to the reference camera on the display screen. This results in the color of the reference camera's captured image closer to (60, 100, 150). The same applies to other cameras. Therefore, for the operator of the asset editor, the color of the virtual scene generated by the asset editor is consistent with the color captured by the camera on the display screen.

[0070] Next, the real-world color difference between the cameras needs to be fitted, as different cameras have different color responses. In real-world scenarios, cameras capture both the virtual scene displayed on the display and the real scene of the shooting location. For the reference camera, after the aforementioned processing, the reference camera obtains a virtual-color difference fitting LUT, allowing the virtual scene captured by the reference camera and the virtual scene displayed on the display to be color-aligned. Next, virtual-real alignment can be performed on the reference camera, aligning the colors of the real and virtual scenes in the images captured by the reference camera. For example, after setting up the shooting location and its physical lighting, with the virtual scene displayed on the display, the reference camera captures an image containing the real scene area and the display. The captured image can be manually reviewed to see if there is a color difference between the real and virtual scenes. If so, in most cases, virtual-real alignment can be achieved by adjusting the physical lighting on the scene. After manually completing virtual-real alignment for the reference camera, virtual-real alignment can then be performed for the other cameras to be compensated.

[0071] For the reference shooting device that has completed the virtual-real alignment, the shooting picture of the reference shooting device contains the virtual scene 1 and the real scene 1 with aligned colors; for a certain shooting device to be compensated, its shooting picture also contains the virtual scene 2 and the real scene 2. Since the virtual scene 2 will also be calibrated through the virtual color difference fitting LUT of the shooting device to be compensated, the virtual scene 2, the virtual scene 1 and the real scene 1 are all color-aligned, but the colors of the real scene 2 and the virtual scene 2 are not aligned; in the actual shooting scene, after the on-site layout is completed and the reference shooting device completes the virtual-real alignment, the real scene lighting is usually not changed. Therefore, the color of the virtual scene 2 will be adjusted so that the color of the real scene 2 can be aligned with the color of the virtual scene 2. Therefore, it is necessary to determine the color difference between real scene 1 and real scene 2, that is, to find a LUT that fits the color difference between real scene 1 and real scene 2. Real scene 1 has been color-aligned with virtual scene 1 and virtual scene 2. The LUT that fits the color difference between real scene 1 and real scene 2 has the same meaning as the LUT that fits the color difference between virtual scene 2 and real scene 2. This LUT will be mounted on virtual scene 2 to align the colors between virtual scene 2 and real scene 2.

[0072] Based on this, multiple physical real scene cards can be prepared and set in the actual shooting area. The number of real scene cards can be configured according to actual needs. Usually, the number of real scene cards is too small to reach the number of virtual scene cards.

[0073] For each real scene card, each shooting device can obtain the corresponding real scene collection color by shooting. Wherein, when each shooting device shoots each real scene card, the light received by each real scene card remains consistent.

[0074] In this embodiment, the shooting device shoots the real scene color card to obtain the real scene acquisition colors, and the acquired real scene acquisition colors can be arranged in a sampling order.

[0075] In some examples, the brightness values can be normalized according to the brightness differences in linear space, retaining only the color sampling differences of the real-life scene card caused by inherent color style differences between different shooting devices. Specifically, the process of establishing the real-life color difference fitting LUT for the shooting device to be compensated based on the relationship between the real-life scene captured colors of the reference shooting device and the real-life scene captured colors of the shooting device to be compensated may include:

[0076] Obtaining a first average brightness value of each real scene captured color of the reference shooting device and a second average brightness value of each real scene captured color of the shooting device to be compensated;

[0077] Adjusting each real scene captured color of the shooting device to be compensated based on a ratio of the second brightness average to the first brightness average;

[0078] A real scene disparity fitting LUT of the shooting device to be compensated is established according to the relationship between the real scene captured colors of the reference shooting device and the adjusted real scene captured colors of the shooting device to be compensated.

[0079] For example, the brightness value of each real scene captured color of the reference shooting device can be obtained, and a first brightness average can be calculated; similarly, the brightness value of each real scene captured color of the shooting device to be compensated can be obtained, and a second brightness average can be calculated; for example, the real scene captured colors captured by the reference shooting device for each real scene color card are T_1 to T_48 respectively, and the brightness value of each of the 48 colors can be obtained and the average can be calculated to obtain a first brightness average t; similarly, the same operation is performed on each shooting device to be compensated; for example, the real scene captured colors captured by a certain shooting device for each real scene color card are R_1 to R_48 respectively, and the brightness value of each of the 48 colors can be obtained and the average can be calculated to obtain a second brightness average r. Using the first average brightness value of the reference camera as a reference, the ratio of the second average brightness value to the first average brightness value is calculated. This ratio is used as a coefficient and divided by each real-scene captured color of the camera to be compensated to obtain the adjusted real-scene captured colors of the camera to be compensated. Specifically, the quotient of r divided by t is calculated, and then each color from R_1 to R_48 is divided by this quotient to obtain the adjusted real-scene captured colors of the camera to be compensated. The calculation of the adjusted real-scene captured colors of other cameras to be compensated is similar. In practical applications, in addition to the above-described method of directly dividing by the ratio, other adjustment methods can be set based on the ratio, which is not limited in this embodiment. In this way, the brightness difference between the camera to be compensated and the reference camera can be eliminated, thereby retaining only the color sampling differences of the real-scene image card caused by the inherent color style differences between the cameras. Optionally, in the subsequent process of establishing a real-scene color disparity fitting LUT for the camera to be compensated, the relationship between the adjusted real-scene captured colors of the camera to be compensated can be used to establish the real-scene color disparity fitting LUT for the camera to be compensated. The real-scene color disparity fitting LUT represents a mapping relationship between the real-scene color captured by the reference shooting device and the real-scene color captured by the shooting device to be compensated.

[0080] If a real-world image card is capable of high-frequency sampling in actual applications, a real-world image difference fitting LUT can be constructed using an interpolation algorithm based on the high-frequency sampling results, similar to the construction of a virtual-world image difference fitting LUT in the aforementioned embodiment. However, high-frequency sampling on a real-world image card is more time-consuming than on a virtual-world image card. Therefore, if the sampling order of a real-world image card is lower, numerical fitting rather than high-frequency interpolation can be used to fit the real-world color differences between different capture devices.

[0081] This embodiment designs a numerical fitting method using a machine learning model. As an example, the establishment of a real-life visual disparity fitting LUT for the camera to be compensated may include:

[0082] Obtaining an initial relationship function between the real scene captured color representing the reference shooting device and the real scene captured color of the shooting device to be compensated; the initial relationship function includes a plurality of coefficients to be solved;

[0083] Substituting each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated into the initial relationship function respectively to obtain multiple equation groups;

[0084] Inputting the multiple equation groups into a preset machine learning model, respectively, and training the machine learning model with minimizing the color difference between each real scene captured color of the camera to be compensated and each predicted color as an optimization goal, so as to obtain a target relationship function after obtaining the values of the coefficients to be solved solved by the machine learning model; the predicted colors refer to the predicted colors obtained by substituting the real scene captured colors of the reference camera into the current relationship function after obtaining a current relationship function based on the current values of the coefficients to be solved;

[0085] A real-world visual disparity fitting LUT for the shooting device to be compensated is established according to the target relationship function.

[0086] For example, the initial relationship function can be y=f(x|θ); where x is the real-scene captured color of the reference shooting device, y is the real-scene captured color of the shooting device to be compensated, and θ is the coefficient to be solved; in actual applications, there can be multiple coefficients to be solved, and the multiple coefficients to be solved can include linear coefficients and / or nonlinear coefficients.

[0087] For example, if the baseline camera and the camera to be compensated are of the same type, the coefficients to be solved can be simply linear coefficients. Research has found that for high-quality cameras of the same brand, the color differences in real-life scenes between cameras approach linear deviations. "Same type" here can refer to the same manufacturer or model. For different baseline cameras and cameras to be compensated, additional coefficient correction terms can be introduced in addition to the linear coefficients, so the coefficients to be solved can include both linear and nonlinear coefficients.

[0088] Assume that the colors of the real-life color card are a, b, and c. For the sake of convenience, the color values are represented by simple characters. In actual applications, as mentioned in the above embodiment, the color values can be represented by RGB three-channel values.

[0089] Assume that the reference camera captures the three real-world scene cards, and the captured colors are Da, Db, and Dc respectively; and the captured colors of the camera to be compensated 1 are Ka, Kb, and Kc respectively. We can obtain three equations: Ka = f(Da), Kb = f(Da), Kc = f(Dc);

[0090] The machine learning model needs to solve the numerical values of the coefficients to be solved in the initial relationship function based on these three equations; for example, the solution process of this embodiment is to solve the value of the coefficient θ in the relationship function y = f(x|θ); different values of θ will cause the input x of the function to obtain different y; this embodiment hopes to solve the value of θ as accurately as possible; and whether the value of θ is accurate enough is measured by the value of the loss function.

[0091] Specifically, if there are s real-world color maps, there are s pairs of true mapping relationships: from the real-world colors captured by the reference camera to the real-world colors captured by the camera to be compensated. Different values of θ yield different relationship functions: y = f(x|θ). The training process of a machine learning model involves solving for different values of θ to determine the optimal value.

[0092] Substituting the actual scene colors captured by the s reference cameras into the solved value y = f(x|θ) yields s corresponding y values. The color differences between these s y values and the s actual scene colors captured by the camera to be compensated are the values of the loss function. The greater the color difference between these s y values and the s actual scene colors captured by the camera to be compensated, the less accurate the current value of θ. The smaller the color difference, the more accurate the current value of θ. The color difference is negatively correlated with the current value of θ.

[0093] Therefore, the color difference in this embodiment refers to the color difference between the real scene captured colors of the shooting device to be compensated and the predicted colors; each predicted color refers to the predicted colors obtained by substituting the real scene captured colors of the reference shooting device into the current relationship function after obtaining the current relationship function based on the current value of the coefficient to be solved.

[0094] The specific method for calculating color difference can be customized. For example, a metric such as ΔE (a standard that quantifies the difference between two colors using a distance metric in the ITP color space) that approximates the human eye's true perception of color difference can be used as the color difference. This color difference serves as the loss function for the machine learning model. Training can be stopped when the loss function reaches a preset value or the calculation exceeds a specified number of iterations. The values of the coefficients to be solved are then obtained, and the target relationship function can be derived.

[0095] In practical applications, the initial values of the model parameters of the machine learning model (i.e., the coefficients θ to be solved in the aforementioned relationship function) can be configured as needed. In order to improve the efficiency of solving, in this embodiment, during the training process of the machine learning model, the initial values of the model parameters can be first calculated by a linear regression algorithm, and then the model parameters can be adjusted using an iterative optimization algorithm. For example, the linear regression algorithm can be an algorithm such as the least squares method, and the iterative optimization algorithm can be an algorithm such as the gradient descent method. The aforementioned multiple equation groups can be fitted first by a linear regression algorithm, and a set of initial linear coefficient values can be quickly solved so that the model parameters have a better starting point. After that, the iterative optimization algorithm can be used to gradually adjust all model parameters, which can achieve model convergence more efficiently.

[0096] After obtaining the target relationship function, a standard real-world color disparity fitting LUT can be constructed using the target relationship function. For example, the real-world color disparity fitting LUT to be constructed needs to include a standard input-output color pair, such as a standard input color value d. Using d as the x value in the target relationship function, the y value can be calculated. For example, similar to the format requirements of the LUT for the rendering engine mentioned in the aforementioned embodiment, assuming that the requirements of the real-world color disparity fitting LUT are a mapping relationship from a color set P to a color set W, if a color p1 in the color set P is the same as the real-world color captured by the reference camera for the real-world color card v, then the real-world color wi captured by the camera to be compensated for the real-world color card v can be known, and the mapping relationship between p1 and wi is obtained. If a color p2 in the color set P is not captured by the reference camera, then p2 is substituted into x in the target relationship function to calculate the y value, and the mapping relationship between p2 and the calculated y value can be obtained.

[0097] In practical applications, when there are multiple cameras to be compensated, a real-image disparity fitting LUT for each camera to be compensated can be constructed using the above-described embodiment. In this way, a real-image disparity fitting LUT for each camera can be obtained, thereby fitting the real-image disparity between the cameras.

[0098] Based on the above embodiment, a virtual color disparity fitting LUT for the reference camera can be constructed. Each camera to be compensated has a virtual color disparity fitting LUT and a real color disparity fitting LUT. These LUTs can then be used to perform color calibration on the content to be displayed for each camera during actual shooting.

[0099] In some examples, the virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the display screen for the content to be displayed in the display area corresponding to the shooting device to be compensated.

[0100] For example, for each shooting device to be compensated, the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are connected in series in the same color space, and the color values corresponding to the input and output of the connected LUT are recorded, which can be saved as the color difference fitting LUT of the shooting device to be compensated.

[0101] For example, the real color difference fitting LUT contains multiple pairs of input-output color values, while the virtual color difference fitting LUT also contains multiple pairs of input-output color values. The color difference fitting LUT represents a color value, which is used as the input of the virtual color difference fitting LUT to obtain the corresponding output value, and then used as the input of the real color difference fitting LUT to obtain the output value of the real color difference fitting LUT. The color difference fitting LUT can be established by: for the input color value sequence C1-in of the virtual color difference fitting LUT, the corresponding output color value sequence C2-out in the virtual color difference fitting LUT is obtained; then the output color value sequence C2-out is used as each input color value of the real color difference fitting LUT to obtain the corresponding output color value sequence C3-out in the real color difference fitting LUT. The mapping relationship between C1-in and C3-out is the color difference fitting LUT.

[0102] Since the baseline camera has no color difference with itself, you can directly attach your own virtual scene color difference fitting LUT to achieve color consistency between the virtual scene and the Virtual Asset Editor. For other cameras to be compensated, concatenate their virtual scene color difference fitting LUTs with the real scene color difference fitting LUTs in the same color space. Record the corresponding color values of the input and output of the concatenated LUTs and save them as the color difference fitting LUTs for the camera to be compensated. By attaching the inter-camera color difference fitting LUTs to the corresponding cameras, you can achieve color difference matching between the baseline camera and the camera to be compensated, thus achieving virtual and real alignment for all cameras.

[0103] like Figure 3 FIG2 is a flowchart of a color calibration method according to an exemplary embodiment of the present specification. The method can be applied to a virtual shooting system, wherein the virtual shooting system further includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The method can include the following steps:

[0104] In step 302, a virtual scene to be displayed on the display screen is obtained;

[0105] In step 304, after determining the content to be displayed in the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using a virtual color disparity fitting LUT of the reference shooting device;

[0106] In step 306, after determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color difference fitting LUT and the real color difference fitting LUT of the to-be-compensated shooting device;

[0107] The virtual color disparity fitting LUT and real color disparity fitting LUT of the shooting device to be compensated, and the virtual color disparity fitting LUT of the reference shooting device are obtained through the steps of the aforementioned color lookup table LUT establishment embodiment.

[0108] The method of this embodiment can be applied to any computer device with computing capabilities in a virtual filming system. For example, considering that a rendering engine is used to generate a virtual scene on a display screen, this embodiment can be applied to a computer device equipped with a rendering engine. For example, the program corresponding to the method of this embodiment can be configured in the rendering engine.

[0109] For example, in an actual shooting scene, the rendering engine can obtain a virtual scene to be displayed on the display screen; and in actual shooting, each shooting device will correspond to different display areas on the shooting display screen, such as Figure 2B As shown, the two shooting devices correspond to different display areas. The correspondence between the shooting devices and the display areas can be pre-configured in actual shooting, so that the rendering engine can determine the content to be displayed corresponding to each shooting device based on the configured correspondence and the virtual scene to be displayed. It can be understood that the virtual scene to be displayed contains the content to be displayed corresponding to each shooting device.

[0110] For the content to be displayed on the reference camera, color calibration can be performed using the virtual color disparity fitting LUT of the reference camera. This involves mapping the colors of the content to be displayed to the colors of the color-calibrated content using the virtual color disparity fitting LUT. The color-calibrated content to be displayed is then displayed on the display area corresponding to the reference camera. For the content to be displayed on each camera to be compensated, color calibration can be performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT. This involves first calibrating the colors of the content to be displayed using the virtual color disparity fitting LUT, and then calibrating them using the real color disparity fitting LUT. This results in color-calibrated content to be displayed, which can then be displayed on the display area corresponding to the camera to be compensated.

[0111] In some examples, performing color calibration using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated includes:

[0112] Perform color calibration using a color difference fitting LUT of the shooting device to be compensated;

[0113] The color difference fitting LUT of the shooting device to be compensated is obtained by connecting the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated in series.

[0114] In this embodiment, a color difference fitting LUT can be obtained in advance by connecting the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated in series. During color calibration, the color difference fitting LUT can be directly used to calibrate the color of the content to be displayed of the shooting device to be compensated, that is, the color of the content to be displayed is mapped to the color of the calibrated content to be displayed through the color difference fitting LUT, thereby improving the color calibration efficiency.

[0115] As can be seen from the above embodiments, this embodiment only requires the color response data of the camera to the virtual scene card and the real scene card to generate a color lookup table (LUT) based on the algorithm that can compensate for the inherent color difference between the camera's virtual scenes and the real scenes. Using the color lookup table can conveniently, quickly and stably achieve color consistency between virtual scenes and real scenes across multiple cameras, solving the problem of time-consuming virtual and real alignment of multiple cameras in virtual filming and relying on subjective manual adjustment. This can greatly shorten the time spent on color management in virtual filming and effectively improve the efficiency of the crew.

[0116] The color difference fitting solution between shooting devices in virtual shooting provided in this embodiment can, after using a shooting device to shoot a virtual scene card and a real scene card on a specified display screen, fit the real scene color differences between shooting devices through a machine learning model based on a linear measurement strategy and a parameter iteration algorithm, and align the virtual scene color differences through reverse interpolation mapping of a color lookup table, thereby achieving color difference compensation between multiple shooting devices.

[0117] This embodiment uses a parallel combination of a virtual scene card sampling LUT reverse interpolation algorithm and a real scene card color difference compensation LUT fitting algorithm. Compared to the current solution where experienced visual effects personnel on the crew manually adjust the color palette and shooting equipment parameters for all shooting equipment for each shooting equipment in each scene to compensate for the shooting equipment color difference, this embodiment first eliminates the virtual scene difference between multiple shooting equipment through high-precision reverse interpolation of the virtual scene card sampling LUT. After the virtual and real alignment of the reference shooting equipment, the color difference compensation LUT is fitted using a machine learning algorithm based on iterative optimization theory. This aligns the inherent color difference between the compensation shooting equipment and the reference shooting equipment, and achieves compensation for the color difference between multiple shooting equipment. This allows multiple shooting equipment to achieve virtual and real alignment of all shooting equipment with only two rounds of color card sampling, greatly reducing labor costs and the uncertainty caused by the subjective visual modification of the visual effects director.

[0118] This embodiment uses machine learning to model the real-world scene charts captured by the camera. A multivariate equation system is established in a linear color space. The model's linear parameters are initialized using regression algorithms such as least squares. The model is then iteratively optimized using relevant metrics such as ΔE as a loss function to obtain optimal parameters in a nonlinear color space that better aligns with human perception. This approach effectively matches the inherent color differences between multiple cameras. This allows for virtual-real alignment across all cameras on a virtual shoot, requiring only two color chart captures and a single virtual-real alignment. This significantly reduces manual effort and ensures color consistency across multiple cameras. In other words, virtual-real alignment on one camera is equivalent to virtual-real alignment for all cameras, resulting in a single output for the entire shoot cycle. This significantly reduces the time required for color management and alignment, as well as the reliance on subjective judgment by professionals, providing strong support for the commercialization and promotion of virtual photography.

[0119] Corresponding to the aforementioned embodiments of the method for establishing a color lookup table LUT / the color calibration method, this specification also provides embodiments of an apparatus and a computer device to which the apparatus is applied.

[0120] The embodiments of the color lookup table LUT establishment device / color calibration device in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 3 The figure shows a hardware structure diagram of the computer device where the color lookup table LUT creation device / color calibration device is located in this manual, except Figure 4In addition to the processor 410, network interface 420, memory 430, and non-volatile memory 440 shown, the computer device in which the color lookup table LUT establishment device / color calibration device is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0121] like Figure 5 As shown, Figure 5 This is a block diagram of a device for establishing a color lookup table (LUT) according to an exemplary embodiment of this specification. The device is applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The device includes:

[0122] The first establishing module 51 is configured to: for each camera, obtain the virtual scene captured colors corresponding to the virtual scene original colors displayed by the camera on the display screen, and establish a virtual scene color difference fitting LUT for the camera, wherein the virtual scene color difference fitting LUT represents a mapping relationship from the virtual scene captured colors to the virtual scene original colors;

[0123] The second establishing module 51 is configured to obtain each real scene color captured by each photographing device after photographing each real scene card, and establish, for each photographing device to be compensated, a real scene color difference fitting LUT for the photographing device to be compensated based on the relationship between each real scene color captured by the reference photographing device and each real scene color captured by the photographing device to be compensated;

[0124] The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen;

[0125] The virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

[0126] In some examples, the first establishing module 51 is further configured to:

[0127] Arranging the virtual scene acquisition colors acquired by the shooting device according to the display order of the virtual scene original colors displayed on the display screen to obtain a forward LUT from the virtual scene original colors to the virtual scene acquisition colors;

[0128] The forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain a virtual scene disparity fitting LUT of the shooting device.

[0129] In some examples, the second establishing module 52 is further configured to:

[0130] Obtaining a first average brightness value of each real scene captured color of the reference shooting device and a second average brightness value of each real scene captured color of the shooting device to be compensated;

[0131] Adjusting each real scene captured color of the shooting device to be compensated based on the ratio of the second brightness average to the first brightness average;

[0132] A real scene disparity fitting LUT of the shooting device to be compensated is established according to the relationship between the real scene captured colors of the reference shooting device and the adjusted real scene captured colors of the shooting device to be compensated.

[0133] In some examples, the second establishing module 52 is further configured to:

[0134] Obtaining an initial relationship function between the real scene acquisition color representing the reference shooting device and the real scene acquisition color of the shooting device to be compensated; the initial relationship function includes coefficients to be solved;

[0135] Substituting each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated into the initial relationship function respectively to obtain multiple equation groups;

[0136] Inputting the multiple equation groups into a preset machine learning model, respectively, and training the machine learning model with minimizing the color difference between each real scene captured color of the camera to be compensated and each predicted color as an optimization goal, so as to obtain a target relationship function after obtaining the values of the coefficients to be solved solved by the machine learning model; the predicted colors refer to the predicted colors obtained by substituting the real scene captured colors of the reference camera into the current relationship function after obtaining a current relationship function based on the current values of the coefficients to be solved;

[0137] A real-world visual disparity fitting LUT for the shooting device to be compensated is established according to the target relationship function.

[0138] In some examples, during the training process of the machine learning model, the initial values of the model parameters are first calculated using a linear regression algorithm, and then the model parameters are adjusted using an iterative optimization algorithm.

[0139] In some examples, the second establishing module 52 is further configured to:

[0140] The virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the display screen for the content to be displayed in the display area corresponding to the shooting device to be compensated.

[0141] like Figure 6 As shown, Figure 6 This is a block diagram of another color calibration device according to an exemplary embodiment of this specification. The device is applied to a virtual shooting system, which includes a display screen for displaying a virtual scene and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated. The device includes:

[0142] An acquisition module 61 is configured to: acquire a virtual scene to be displayed on the display screen;

[0143] A first calibration module 61 is configured to: determine, based on the virtual scene, the content to be displayed in the display area corresponding to the reference camera on the display screen, and then perform color calibration using a virtual color disparity fitting LUT of the reference camera;

[0144] A second calibration module 61 is configured to: determine, based on the virtual scene, the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen, and then perform color calibration using the virtual color difference fitting LUT and the real color difference fitting LUT of the to-be-compensated shooting device;

[0145] The virtual color disparity fitting LUT and real color disparity fitting LUT of the shooting device to be compensated, and the virtual color disparity fitting LUT of the reference shooting device are obtained through the steps of the aforementioned color lookup table LUT establishment method embodiment.

[0146] In some examples, the second calibration module 61 is further configured to:

[0147] Perform color calibration using a color difference fitting LUT of the shooting device to be compensated;

[0148] The color difference fitting LUT of the shooting device to be compensated is obtained by connecting the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated in series.

[0149] The implementation process of the functions and effects of each module in the above-mentioned color lookup table LUT establishment device / color calibration device is specifically described in the implementation process of the corresponding steps in the above-mentioned color lookup table LUT establishment method / color calibration method, and will not be repeated here.

[0150] Accordingly, an embodiment of this specification further provides a computer program product, including a computer program, which implements the steps of the aforementioned color lookup table LUT establishment method / color calibration method embodiment when executed by a processor.

[0151] Accordingly, an embodiment of this specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of an embodiment of a method for establishing a color lookup table LUT / a color calibration method are implemented.

[0152] Accordingly, an embodiment of this specification further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the color lookup table LUT establishment method / color calibration method embodiment are implemented.

[0153] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0154] The above embodiments can be applied to one or more computer devices, where the computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0155] The computer device may be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0156] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0157] The network where the computer device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0158] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] The steps of the various methods above are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0160] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of specific embodiments of specific inventions. Certain features described in multiple embodiments within this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of the sub-combination.

[0161] The phrases "specific examples" or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0162] Other embodiments of the present invention will readily occur to those skilled in the art upon consideration of the present invention and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present invention being indicated by the following claims.

[0163] It should be understood that the present description is not limited to the exact structure that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0164] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for establishing a color lookup table (LUT), the method being applied to a virtual filming system comprising a display screen for displaying a virtual scene and multiple filming devices, the multiple filming devices comprising a reference filming device and a filming device to be compensated; the method comprising: For each camera, after obtaining the virtual scene captured colors corresponding to each virtual scene original color displayed on the display screen by the camera, a virtual scene color difference fitting LUT of the camera is established, where the virtual scene color difference fitting LUT represents a mapping relationship from the virtual scene captured colors to the virtual scene original colors; Acquire each real scene captured by each shooting device after shooting each real scene card, and for each shooting device to be compensated, establish a real scene color difference fitting LUT for the shooting device to be compensated based on the relationship between each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated; The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen; The virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

2. The method according to claim 1, wherein establishing the virtual color disparity fitting LUT of the shooting device comprises: Arranging the virtual scene acquisition colors acquired by the shooting device according to the display order of the virtual scene original colors displayed on the display screen to obtain a forward LUT from the virtual scene original colors to the virtual scene acquisition colors; The forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain a virtual scene disparity fitting LUT of the shooting device.

3. The method according to claim 1 , wherein establishing a real-scene color disparity fitting LUT for the to-be-compensated shooting device based on a relationship between each real-scene captured color of the reference shooting device and each real-scene captured color of the to-be-compensated shooting device comprises: Obtaining a first average brightness value of each real scene captured color of the reference shooting device and a second average brightness value of each real scene captured color of the shooting device to be compensated; Adjusting each real scene captured color of the shooting device to be compensated based on the ratio of the second brightness average to the first brightness average; A real scene disparity fitting LUT of the shooting device to be compensated is established according to the relationship between the real scene captured colors of the reference shooting device and the adjusted real scene captured colors of the shooting device to be compensated.

4. The method according to claim 1, wherein establishing the real-life color disparity fitting LUT of the shooting device to be compensated comprises: Acquire an initial relationship function representing the real scene captured color of the reference shooting device and the real scene captured color of the shooting device to be compensated; The initial relationship function contains coefficients to be solved; Substituting each real scene captured color of the reference shooting device and each real scene captured color of the shooting device to be compensated into the initial relationship function respectively to obtain multiple equation groups; Inputting the multiple equation groups into a preset machine learning model, respectively, and training the machine learning model with minimizing the color difference between each real scene captured color of the camera to be compensated and each predicted color as an optimization goal, so as to obtain a target relationship function after obtaining the values of the coefficients to be solved solved by the machine learning model; the predicted colors refer to the predicted colors obtained by substituting the real scene captured colors of the reference camera into the current relationship function after obtaining a current relationship function based on the current values of the coefficients to be solved; A real-world visual disparity fitting LUT for the shooting device to be compensated is established according to the target relationship function.

5. According to the method of claim 4, during the training process of the machine learning model, the initial values of the model parameters are first calculated by a linear regression algorithm, and then the model parameters are adjusted using an iterative optimization algorithm.

6. The method according to claim 1, further comprising: The virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the display screen for the content to be displayed in the display area corresponding to the shooting device to be compensated.

7. A color calibration method, the method being applied to a virtual camera system, the virtual camera system comprising a display screen for displaying a virtual scene and a plurality of camera devices, the plurality of camera devices comprising a reference camera device and a camera device to be compensated; the method comprising: Acquiring a virtual scene to be displayed on the display screen; After determining the to-be-displayed content of the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using a virtual color disparity fitting LUT of the reference shooting device; After determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color difference fitting LUT and the real color difference fitting LUT of the to-be-compensated shooting device; The virtual image disparity fitting LUT and real image disparity fitting LUT of the shooting device to be compensated, and the virtual image disparity fitting LUT of the reference shooting device are obtained through the steps of the method according to any one of claims 1 to 6.

8. The method according to claim 7, wherein the color calibration is performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated, comprising: Perform color calibration using a color difference fitting LUT of the shooting device to be compensated; The color difference fitting LUT of the shooting device to be compensated is obtained by connecting the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated in series.

9. A virtual shooting system, comprising a main control terminal, a rendering engine, a display screen for displaying a virtual scene, and multiple shooting devices, wherein the multiple shooting devices include a reference shooting device and a shooting device to be compensated; The main control end is used to implement the steps of the method described in any one of claims 1 to 6; the rendering engine is used to implement the steps of the method described in any one of claims 7 or 8.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images