Color lookup table establishment method, system and device, product and storage medium

Abstract

The invention provides a color lookup table establishment method, system and device, a product and a storage medium, a virtual shooting system comprises a display screen and a plurality of shooting devices, including a reference shooting device and a to-be-compensated shooting device; acquiring virtual scene acquisition colors respectively and correspondingly acquired by the shooting equipment for the original colors of the virtual scenes displayed on the display screen, and establishing a virtual scene color difference fitting LUT of the shooting equipment; acquiring each real scene acquisition color acquired by the shooting device after shooting each real scene color card, and for the to-be-compensated shooting device, establishing a real scene chromatic aberration fitting LUT of the to-be-compensated shooting device according to a relationship between the reference shooting device and each real scene acquisition color of the to-be-compensated shooting device; the virtual scene color difference fitting LUT of the reference shooting equipment is used for performing color calibration on the to-be-displayed content corresponding to the reference shooting equipment; and the virtual scene color difference fitting LUT and the real scene color difference fitting LUT of the to-be-compensated shooting equipment are used for performing color calibration on the to-be-displayed content corresponding to the to-be-compensated shooting equipment.

Description

Technical Field

[0001] The present invention relates to the field of virtual photography technology, and in particular to a method, system, device, product and storage medium for establishing a color lookup table. Background Art

[0002] The main principle of virtual filming is to digitize the background and render it in real time onto a set of large display screens. These display screens are arranged around the actual performance area, where physical props can be arranged, actors can perform in the filming area, and the display screens can adjust the display content in real time as needed to present a realistic virtual background that blends with the actual performance area.

[0003] In virtual shooting tasks that require shooting the screen and the real scene at the same time, there will always be color differences when different shooting devices shoot the same scene, and it takes a lot of time to manually align the virtual scene and the real scene. Summary of the invention

[0004] In order to overcome the problems existing in the related art, this specification provides a method, system, device, product and storage medium for establishing a color lookup table.

[0005] According to a first aspect of an embodiment of the present specification, a method for establishing a color lookup table LUT is provided, the method being applied to a virtual shooting system, the virtual shooting system comprising a display screen for displaying a virtual scene and a plurality of shooting devices, the plurality of shooting devices comprising a reference shooting device and a shooting device to be compensated; the method comprising:

[0006] For each shooting device, after obtaining the virtual scene acquisition colors respectively corresponding to the original colors of the virtual scenes displayed by the shooting device on the display screen, a virtual scene difference fitting LUT of the shooting device is established, wherein the virtual scene difference fitting LUT represents a mapping relationship from the virtual scene acquisition colors to the virtual scene original colors;

[0007] Acquire each real scene acquisition color acquired after each shooting device shoots each real scene color card, and for each shooting device to be compensated, establish a real scene color difference fitting LUT of the shooting device to be compensated according to the relationship between each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated;

[0008] The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen;

[0009] The virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated are used to perform color calibration on the to-be-displayed content in the display area corresponding to the shooting device to be compensated on the display screen.

[0010] According to a second aspect of an embodiment of the present specification, a color calibration method is provided, the method being applied to a virtual shooting system, the virtual shooting system comprising a display screen for displaying a virtual scene and a plurality of shooting devices, the plurality of shooting devices comprising a reference shooting device and a shooting device to be compensated; the method comprising:

[0011] Acquire a virtual scene to be displayed on the display screen;

[0012] After determining the to-be-displayed content of the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using a virtual color difference fitting LUT of the reference shooting device;

[0013] After determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the to-be-compensated shooting devices;

[0014] The virtual color disparity fitting LUT and real color disparity fitting LUT of the shooting device to be compensated, and the virtual color disparity fitting LUT of the reference shooting device are obtained through the steps of the method described in the first aspect.

[0015] According to a third aspect of the embodiments of this specification, a virtual shooting system is provided, the virtual shooting system comprising a main control end, a rendering engine, a display screen for displaying a virtual scene, and a plurality of shooting devices, the plurality of shooting devices comprising a reference shooting device and a shooting device to be compensated;

[0016] The main control end is used to implement the steps of the method described in the first aspect; the rendering engine is used to implement the steps of the method described in the second aspect.

[0017] According to a fourth aspect of the embodiments of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method embodiments described in the first or second aspect are implemented.

[0018] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method embodiments described in the first or second aspect are implemented.

[0019] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0020] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0021] In the embodiment of the present specification, for each shooting device, after obtaining the virtual scene acquisition colors corresponding to each virtual scene original color displayed by the shooting device on the display screen, a virtual scene disparity fitting LUT of the shooting device is established, and the virtual scene disparity fitting LUT represents the mapping relationship from the virtual scene acquisition color to the virtual scene original color; therefore, the virtual scene disparity fitting LUT of each shooting device can fit the color difference between the shooting color of the shooting device itself and the display color of the display screen; this embodiment also performs real scene card sampling, and designs a reference shooting device as the standard, and establishes a real scene disparity fitting LUT for each shooting device to be compensated according to the relationship between each real scene acquisition color of the reference shooting device and each real scene acquisition color of each shooting device to be compensated; thus, the reference shooting device uses the virtual scene disparity fitting LUT to perform color calibration on the content to be displayed corresponding to the reference shooting device; and each shooting device to be compensated uses the virtual scene disparity fitting LUT and the real scene disparity fitting LUT of the shooting device to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated, so that virtual and real alignment of all shooting devices can be achieved. It can be seen that in this embodiment, only two rounds of color card sampling are required to achieve virtual-real alignment of all shooting devices, which greatly reduces the labor cost and uncertainty caused by manual subjective visual modification.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1A is a schematic diagram of a virtual shooting scene according to an exemplary embodiment of the present specification.

[0024] Figure 1B This is a schematic diagram of a virtual shooting link shown in this specification according to an exemplary embodiment.

[0025] Figure 1C This is another schematic diagram of a virtual shooting link shown in this specification according to an exemplary embodiment.

[0026] Figure 2A This is a flowchart of a method for establishing a color lookup table LUT according to an exemplary embodiment of this specification.

[0027] Figure 2B is a schematic diagram of another virtual shooting scene according to an exemplary embodiment of the present specification.

[0028] Figure 2CThis is another schematic diagram of a virtual shooting link shown in this specification according to an exemplary embodiment.

[0029] Figure 3 This is a flowchart of a method for establishing a color lookup table LUT according to an exemplary embodiment of this specification.

[0030] Figure 4 This is a hardware structure diagram of a computer device where a color lookup table LUT creation device / color calibration device is located according to an exemplary embodiment of this specification.

[0031] Figure 5 It is a block diagram of a device for establishing a color lookup table LUT according to an exemplary embodiment of this specification.

[0032] Figure 6 It is a block diagram of a device for establishing a color lookup table LUT according to an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0034] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0036] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0037] Virtual filming is a virtual filming technology that uses display screens to build virtual backgrounds. It combines real-time rendering and LED (Light Emitting Diode) display technology to present a realistic virtual environment in real time on site, replacing traditional green screen or blue screen filming. The main principle of virtual filming is to digitize the background and render it in real time onto a set of large display screens. These display screens are arranged around the actual performance area, where physical props can be arranged, actors can perform in the filming area, and the display screens can adjust the display content in real time as needed to present a realistic virtual background that blends with the actual performance area.

[0038] like Figure 1A As shown, it is a schematic diagram of a virtual shooting scene shown in this specification according to an exemplary embodiment. The virtual shooting scene may include a virtual shooting system composed of one or more computer devices. As an example, the virtual shooting system may include a combination of one or more of the following devices: one or more main control machines 011, one or more rendering devices 021 (also referred to as screen-mounting machines), one or more broadcast control processing devices 031, display screen 040 (in actual scenes, display screen 040 may be composed of one or more display screens, for example, the figure shows three display screens included in display screen 040: display screen 041, display screen 042 and display screen 043), and one or more shooting devices 051; wherein, the number of various types of equipment can be flexibly configured according to actual needs, and this embodiment does not limit this. In actual applications, the virtual shooting system may also include other devices, such as mobile terminals or network devices, etc., as needed, and this embodiment does not limit this.

[0039] Optionally, each host computer 011 may be connected to one or more rendering devices 021, and the specific connection method may be selected according to actual needs and device compatibility. As an example, a wired or wireless connection may be made through a local area network or the Internet, and a network transmission protocol may be used for communication. As an example, the host computer may send a variety of control instructions to the rendering device connected thereto, such as control instructions containing specific image information, etc.

[0040] Optionally, each rendering device 021 can be connected to one or more broadcast control processing devices 031; the specific connection method can be selected according to actual needs and device compatibility. As an example, it can include DP (DisplayPort, a digital display interface standard) connection, DP can be used to transmit high-quality audio and video signals. It can also be HDMI (High-Definition Multimedia Interface, a high-definition digital audio and video interface standard), HDMI can combine audio, video and control signals on a single cable for transmission. As an example, the rendering device 021 can send a variety of control instructions to the broadcast control processing device connected to it. For example, the rendering device can be used as an image signal source to send control instructions containing rendered images, and so on.

[0041] In actual applications, the broadcast control processing device 031 is optional and may not be configured in some scenarios. Optionally, each broadcast control processing device 031 can be connected to one or more display screens 040; the specific connection method can be selected according to actual needs and device compatibility. As an example, it can also include DP or HDMI connection, USB (Universal Serial Bus) or network connection, etc. The broadcast control processing device can be used to control and manage the display screen connected to it. As an example, the broadcast control processing device 031 can be used for data transmission and decoding, such as the broadcast control processing device 031 can receive signals from external sources (such as screen machines, computers, mobile terminals or media players, etc.) and decode them into a format suitable for display on the display screen; it can also be used for display control, such as overall control and scheduling of the display screen, including brightness adjustment, color correction, grayscale control, etc.; it can also be used for partition management of the display screen, and the display screen can be divided into multiple independent areas, each area can display different content.

[0042] Optionally, each host computer 011 can be connected to one or more shooting devices 051; the specific connection method can be selected according to actual needs and device compatibility. As an example, it can include wired connections such as HDMI or SDI (Serial Digital Interface, a digital video transmission standard), and can also include wireless connections such as Wi-Fi (Wireless Fidelity) or RF (Radio Frequency). The shooting device 051 can transmit the shooting data to the host computer.

[0043] Optionally, the display screen 040 may be an LED screen, an LCD screen, or other types, and may be a curved screen or a flat screen. It should be understood that those skilled in the art may customize the type, quantity, size, resolution, etc. of the display screen in the virtual shooting system according to actual needs, and this specification embodiment does not limit this. It should be understood that this specification embodiment does not limit the communication connection method between devices.

[0044] In some virtual shooting scenes, multiple shooting devices are required to shoot simultaneously; and in virtual shooting scenes that need to shoot both the screen and the real scene at the same time, there will always be color differences when different shooting devices shoot the same scene, resulting in a lot of time spent on realigning the virtual scene and the real scene each time the shooting device is changed, which relies on subjective manual work.

[0045] In the overall chain of current virtual shooting, the crew generally only allocates a few hours to adjust the shooting equipment and color. The color alignment of the virtual scene and the real scene takes the most time. This is because if the virtual scene on the screen as the background is inconsistent with the real scene as the foreground in color, it will cause shooting "goofs", so detailed manual adjustments are required. When the same shooting equipment is used to shoot scenes with different colors and different shooting equipment is used to shoot the same scene, there will be color differences, that is, shooting equipment color difference.

[0046] In order to overcome the color difference between the virtual and real scenes of the shooting equipment itself, the inherent color difference of the virtual scenes between shooting equipment, and the inherent color difference of the real scenes between shooting equipment, the staff needs to perform detailed manual comparison to modify the color palette. However, the color mapping relationship between the virtual scene and the real scene, and the virtual scene and the virtual asset editor is not a linear relationship, and manual adjustment is difficult. It is difficult to achieve full color gamut alignment using a simple RGB three-channel palette.

[0047] like Figure 1B As shown, it is a schematic diagram of a virtual shooting link in a virtual shooting scene. The shooting equipment directly shoots the virtual scene on the display screen and the real scene in the venue. At this time, the virtual scenes and real scenes captured by different shooting equipment are different. The only way is to manually align the virtual and real scenes of the shooting equipment one by one. In addition, because the spectral response and color relationship between the shooting equipment screen and the color palette of the virtual asset editor and the venue lighting are relatively complex, manual adjustment is difficult to completely align them. It is also necessary to modify the shooting results of each shooting equipment during post-processing, which is quite time-consuming and laborious.

[0048] like Figure 1C As shown in FIG. 1 , it is another schematic diagram of a virtual shooting link in a virtual shooting scene. Figure 1BOn the basis of the previous method, a virtual-to-real alignment scheme that relies on the sampling of virtual scene cards is added. By sampling the virtual scene cards, the virtual scene images captured by the shooting devices are restored to the same. This can effectively reduce the difficulty of manual virtual-to-real alignment when the color difference between the shooting devices is not large. However, manual alignment is still difficult when the color difference between the shooting devices is large. Moreover, when the number of shooting devices is large or the shooting devices need to be replaced frequently, manual adjustment of each shooting device is still a considerable workload.

[0049] Based on this, the present embodiment provides a method for establishing a color lookup table LUT, which only requires two rounds of color card sampling to achieve virtual-real alignment of all shooting devices, greatly reducing the labor cost and uncertainty caused by artificial subjective visual modification. Next, the present embodiment is described in detail.

[0050] like Figure 2A As shown, Figure 2A The present specification is a flowchart of a method for establishing a color lookup table LUT according to an exemplary embodiment. The method can be applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and a plurality of shooting devices, wherein the plurality of shooting devices include a reference shooting device and a shooting device to be compensated. The method may include the following steps:

[0051] In step 202, for each shooting device, after obtaining the collected virtual scene colors corresponding to the original colors of each virtual scene displayed by the shooting device on the display screen, a virtual scene color difference fitting LUT of the shooting device is established.

[0052] The virtual scene disparity fitting LUT represents a mapping relationship from the virtual scene acquisition color to the virtual scene original color.

[0053] In step 204, each real scene acquisition color acquired by each shooting device after shooting each real scene color card is obtained, and for each shooting device to be compensated, a real scene color difference fitting LUT of the shooting device to be compensated is established according to the relationship between each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated.

[0054] The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen.

[0055] The virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated are used to perform color calibration on the to-be-displayed content in the display area corresponding to the shooting device to be compensated on the display screen.

[0056] As an example, multiple shooting devices may be used for virtual shooting in a virtual shooting scene. The number of shooting devices may be configured according to the actual shooting scene, and this embodiment does not limit this. The multiple shooting devices in this embodiment include a reference shooting device and a shooting device to be compensated. The reference shooting device may be one, and the shooting device to be compensated may be one or more. Any one of the multiple shooting devices may be selected as the reference shooting device. When shooting, the framing ranges of different shooting devices may not overlap, that is, different shooting devices may shoot different areas. For example, Figure 2B In the virtual shooting scene schematic diagram shown, the shooting ranges of shooting device 1 and shooting device 2 are different, and shooting device 1 and shooting device 2 can respectively shoot different display areas on the display screen. Usually, the display screen is used to render and display the overall virtual shooting scene. Since different shooting devices shoot different display areas of the display screen, the display content corresponding to each shooting device is generally different, but the color style of different areas of the same virtual scene is basically the same.

[0057] The method of this embodiment can be applied to any computer device with computing capabilities in the virtual shooting system. In some examples, the method of this embodiment can be applied to the main control end of the virtual shooting system, which can be a computer running on Figure 1A The software program on the main control machine in the illustrated embodiment, wherein the main control machine may specifically be a computer device, including but not limited to a server, a cloud server, a server cluster, a tablet computer, a personal digital assistant, a laptop computer, or a desktop computer and other devices.

[0058] As an example, the color in this embodiment can be represented by a color value, for example, the color value of the RGB (RED, Green, Blue) color space can be used to represent it, and each color value contains the color classification of the three RGB channels; for example, in the color value used by the display screen, the value of each color channel can be a 10-bit value from 0 to 1023; optionally, the color accuracy of the camera will be higher. In other examples, it is also optional to use color values ​​in other color spaces, such as Lab or YUV, etc., which is not limited in this embodiment.

[0059] As an example, the original color of each virtual scene can be predetermined; for example, each color channel can uniformly sample multiple different values ​​to constitute all the original colors of the virtual scenes. The sampled values ​​can be flexibly configured, for example, can be 64 or other values, which is not limited in this embodiment.

[0060] Each virtual scene original color can be displayed on the display screen in a certain arrangement order; each shooting device can shoot each virtual scene original color displayed on the display screen, and the virtual scene acquisition color collected by the shooting device can be obtained from the shooting data of the shooting device. Each virtual scene original color is equivalent to each virtual scene card, which can be obtained through asset editing tools such as rendering engines (such as Unreal Engine (UE)), so the number of virtual scene cards can be large, so that higher-order results can be obtained by sampling.

[0061] As an example, the virtual scene colors captured by the shooting device can be arranged according to the display order of the original colors of the virtual scenes displayed on the display screen to obtain a forward LUT from the original colors of the virtual scenes to the virtual scene capture colors; the forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain the virtual scene disparity fitting LUT of the shooting device.

[0062] Specifically, after the camera collects the colors of each virtual scene corresponding to each virtual scene card, it can arrange them in a sampling order, and the arrangement result is equivalent to the high-frequency sampling of the forward color mapping space of the camera;

[0063] For example, the original color set of each virtual scene displayed by the display is M, and the set M contains 262144 virtual scene original colors from M_1 to M_262144; the shooting device 1 shoots each color in the set M, and the obtained virtual scene acquisition color set is N, which is also N_1 to N_262144, a total of 262144 virtual scene acquisition colors; after acquisition, a forward LUT representing the mapping relationship from M to N can be obtained;

[0064] Furthermore, for each actual captured color (i.e., virtual scene acquisition color) that is equal to the sampling point input value in the virtual scene disparity fitting LUT to be established, the mapping relationship between the virtual scene acquisition color of the shooting device and the virtual scene original color can be determined; for the actual captured color that is not equal to the sampling point input value, interpolation calculation can be used to predict the virtual scene original color corresponding to the actual captured color.

[0065] For example, the forward LUT is inverted, that is, the mapping relationship from N to M is represented; the virtual scene difference fitting LUT that needs to be established can be used in the rendering engine. The rendering engine has certain format requirements for the LUT. It is assumed that the virtual scene difference fitting LUT to be established is a LUT that represents the mapping relationship from N' to M'. The color set of N' is not necessarily the same as the virtual scene acquisition color set N. For example, some colors in N' exist in the set N. These colors can directly find the corresponding virtual scene original colors from the set M. However, for the virtual scene acquisition colors in N' but not in the set N, the corresponding virtual scene original colors can be calculated by interpolation.

[0066] For example, according to the format requirements of the rendering engine for LUT, the virtual scene disparity fitting LUT to be established represents the mapping relationship from the virtual scene acquisition color set N' to the virtual scene original color M'; assuming that N' contains a total of 262144 colors from n'_1 to n'_262144; the virtual scene acquisition color set collected by the shooting device for each virtual scene card is set N, and these two sets are most likely different. It can be checked whether each color in set N' exists in set N. Assuming that n'_1 is the same as a color in set N, for example, the same as N_i, the color corresponding to N_i can be determined in set M. Assuming it is M_j, the mapping relationship between n'_1 and M_j is determined. Similarly, for n'_2, this color is different from every color in set N, so the original color of the virtual scene mapped by n'_2 needs to be calculated by interpolation. The same is true for other colors in set N'.

[0067] Since the arrangement result of the virtual scene collected colors corresponding to the original colors of each virtual scene by each shooting device is equivalent to the high-frequency sampling sequence of the forward color mapping space of the shooting device, the high-frequency sampling sequence can be reversely interpolated to obtain a reverse LUT between the shooting device and the virtual scene on the display screen. Through this reverse LUT, unified management of the virtual scene color is achieved, so that the virtual scenes in the pictures of all shooting devices are consistent.

[0068] By interpolating the high-frequency sampling sequence of the virtual scene, the high-frequency sampling sequence can be reorganized into a virtual scene disparity fitting LUT in a standard format. Specifically, the forward LUT can be inverted using a reverse interpolation algorithm. The inversion operation is equivalent to solving a set of equations that outputs an equal output to the input itself after mounting the reverse LUT once and then the forward LUT once for each given input value. The solution can be performed using an input-by-input iterative inverse interpolation algorithm. First, a local reverse LUT is calculated, and then gradually updated to the global to obtain a global reverse LUT. When there are irreversible points in the color mapping LUT, an interpolation algorithm is used to fill in the theoretical color values ​​of the irreversible points. Finally, the global reverse LUT is expanded to a standard format LUT that can be read by the device, that is, a standard format LUT that can be read by the rendering engine.

[0069] like Figure 2CAs shown, this specification shows another schematic diagram of a virtual shooting link according to an exemplary embodiment. The virtual scene disparity fitting LUT obtained in the above embodiment can realize the unified management of virtual scene colors, so that the virtual scenes in the shooting pictures of all shooting devices are consistent, thereby aligning the virtual scene disparity between the shooting devices. For example, assuming that each shooting device does not need to shoot the real scene but only needs to shoot the display screen, that is, only needs to shoot the virtual scene, a virtual asset editor such as Unreal Engine can generate a virtual scene to be displayed on the display screen, and different shooting devices shoot different display areas of the display screen, and each shooting device has corresponding content to be displayed. The virtual scene disparity fitting LUT of each shooting device can be used to perform color calibration on the content to be displayed corresponding to each shooting device. For example, each frame of the virtual scene to be displayed contains multiple pixels, and each pixel has a color (i.e., a pixel value). Taking the reference shooting device as an example, the pixel value of each pixel contained in the corresponding content to be displayed is the color of each pixel contained in the picture captured by the reference shooting device. However, there is a color difference between the color displayed on the display screen and the color captured by the reference shooting device. For the color of each pixel in the picture captured by the reference shooting device, the corresponding target color can be obtained from the virtual scene difference fitting LUT. The target color of each pixel in the picture captured by the reference shooting device is used as the color calibration result of the content to be displayed, and is used to be displayed on the display area corresponding to the reference shooting device on the display screen. For example, for the virtual scene to be displayed, assuming that the color of the content to be displayed (the original color of the virtual scene) of the reference camera is (60, 100, 150), in order to make the color of the shooting screen of the camera (the color of the virtual scene acquisition) (60, 100, 150), the color needs to be calibrated. Specifically, the color (60, 100, 150) is used as the input of the virtual scene difference fitting LUT of the reference camera, and the corresponding color is (56, 108, 142). The color (56, 108, 142) is used as the calibration result of the color (60, 100, 150) and will be displayed on the display area corresponding to the reference camera on the display screen, so that the color of the shooting screen of the reference camera is closer to (60, 100, 150). The same is true for other cameras. Therefore, for the operator of the asset editor, the color of the virtual scene generated by the asset editor is consistent with the color captured by the camera on the display screen.

[0070] Next, it is necessary to fit the real scene difference between the shooting devices, because different shooting devices have different color responses. In the actual scene, the shooting device will shoot the virtual scene displayed on the display screen and the real scene of the shooting site. For the reference shooting device, after the above processing, the reference shooting device obtains the virtual scene difference fitting LUT, and the virtual scene shot by the reference shooting device and the virtual scene displayed on the display screen can be color-aligned; then the reference shooting device can be virtual-real aligned, that is, the color alignment between the real scene and the virtual scene in the shooting picture of the reference shooting device, for example, the shooting scene and the physical lighting of the scene are arranged, the display screen displays the virtual scene, and the reference shooting device shoots the picture containing the real scene area and the display screen. The shooting picture can be manually checked to see if there is a difference in the color of the real scene and the virtual scene in the shooting picture. If so, in most cases, the virtual-real alignment can be achieved by adjusting the physical lighting of the scene. The reference shooting device completes the virtual-real alignment manually, and then the virtual-real alignment of other shooting devices to be compensated can be performed.

[0071] For the reference shooting device that has completed the virtual-real alignment, the shooting picture of the reference shooting device includes the virtual scene 1 and the real scene 1 whose colors are aligned; for a shooting device to be compensated, its shooting picture also includes the virtual scene 2 and the real scene 2. Since the virtual scene 2 will also be calibrated through the virtual scene color difference fitting LUT of the shooting device to be compensated, the virtual scene 2, the virtual scene 1 and the real scene 1 are all color-aligned, but the colors of the real scene 2 and the virtual scene 2 are not aligned; in the actual shooting scene, after the on-site layout is completed and the reference shooting device completes the virtual-real alignment, the real scene lighting is usually not changed. Therefore, the color of the virtual scene 2 is adjusted so that the color of the real scene 2 can be aligned with the color of the virtual scene 2. Therefore, it is necessary to determine the color difference between real scene 1 and real scene 2, that is, to find a LUT that fits the color difference between real scene 1 and real scene 2. Real scene 1 has been color-aligned with virtual scene 1 and virtual scene 2. The LUT that fits the color difference between real scene 1 and real scene 2 has the same meaning as the LUT that fits virtual scene 2 and real scene 2. This LUT will be mounted to virtual scene 2 to align the colors between virtual scene 2 and real scene 2.

[0072] Based on this, multiple physical real scene cards can be prepared, and the real scene cards can be set in the actual shooting area. The number of real scene cards can be configured according to actual needs. Usually, the number of real scene cards is too small and cannot reach the number of virtual scene cards.

[0073] For each real scene card, each shooting device can obtain the corresponding real scene acquisition color by shooting. When each shooting device shoots each real scene card, the light received by each real scene card remains consistent.

[0074] In this embodiment, the shooting device shoots the real scene color card to obtain the real scene acquisition colors, and the acquired real scene acquisition colors can be arranged in a sampling order.

[0075] In some examples, the brightness value can be normalized according to the brightness difference in the linear space, and only the color sampling difference of the real-life color card caused by the inherent color style difference between the shooting devices is retained. Specifically, the real-life color difference fitting LUT of the shooting device to be compensated is established according to the relationship between each real-life scene acquisition color of the reference shooting device and each real-life scene acquisition color of the shooting device to be compensated, which may include:

[0076] Acquire a first brightness average of each real scene acquisition color of the reference shooting device and a second brightness average of each real scene acquisition color of the shooting device to be compensated;

[0077] Based on the ratio of the second brightness average to the first brightness average, adjusting each real scene acquisition color of the shooting device to be compensated;

[0078] According to the relationship between the real scene acquisition colors of the reference shooting device and the adjusted real scene acquisition colors of the shooting device to be compensated, a real scene disparity fitting LUT of the shooting device to be compensated is established.

[0079] For example, the brightness value of each real scene captured color of the reference shooting device can be obtained, and a first brightness mean value can be calculated; similarly, the brightness value of each real scene captured color of the shooting device to be compensated can be obtained, and a second brightness mean value can be calculated; for example, the real scene captured colors captured by the reference shooting device for each real scene color card are T_1 to T_48 respectively, and the brightness value of each color in the 48 colors can be obtained and the mean value can be calculated to obtain a first brightness mean value t; similarly, the same operation is performed on each shooting device to be compensated; for example, the real scene captured colors captured by a certain shooting device to be compensated for each real scene color card are R_1 to R_48 respectively, and the brightness value of each color in the 48 colors can be obtained and the mean value can be calculated to obtain a second brightness mean value r. Taking the first brightness mean of the reference shooting device as a reference, the ratio of the second brightness mean to the first brightness mean is calculated, and the ratio is used as a coefficient to divide the real scene acquisition colors of the shooting device to be compensated to obtain the adjusted real scene acquisition colors of the shooting device to be compensated; that is, the quotient of r divided by t is calculated, and then each color in R_1 to R_48 is divided by the quotient respectively, and the adjusted real scene acquisition colors of the shooting device to be compensated can be obtained. The calculation of the adjusted real scene acquisition colors of other shooting devices to be compensated is similar. In practical applications, in addition to the above-mentioned method of directly dividing by the ratio, other adjustment methods can also be set based on the ratio, which is not limited in this embodiment. In this way, the brightness difference between the shooting device to be compensated and the reference shooting device can be removed, so that only the color sampling difference of the real scene card caused by the inherent color style difference between the shooting devices is retained. Optionally, in the subsequent process of establishing the real-life color difference fitting LUT, the relationship between the adjusted real-life color acquisition colors of the shooting device to be compensated can be used to establish the real-life color difference fitting LUT of the shooting device to be compensated. The real-scene color disparity fitting LUT represents a mapping relationship from the real-scene color captured by the reference shooting device to the real-scene color captured by the shooting device to be compensated.

[0080] If the real scene card can achieve high-frequency sampling in actual applications, the real scene difference fitting LUT can be established through an interpolation algorithm based on the high-frequency sampling result, as in the establishment method of the virtual scene difference fitting LUT in the above embodiment. However, compared with the virtual scene card, the real scene card needs to spend a greater time cost to achieve high-frequency sampling. Therefore, when the sampling order of the real scene card is low, numerical fitting can be performed instead of high-frequency interpolation, so as to fit the real scene style color differences between the shooting devices.

[0081] This embodiment designs to achieve numerical fitting by means of a machine learning model. As an example, the establishment of a real-life color difference fitting LUT of the shooting device to be compensated may include:

[0082] Acquire an initial relationship function between the real scene acquisition color representing the reference shooting device and the real scene acquisition color of the shooting device to be compensated; the initial relationship function includes a plurality of coefficients to be solved;

[0083] Substituting each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated into the initial relationship function respectively to obtain a plurality of equation groups;

[0084] The multiple equation groups are respectively input into a preset machine learning model, and the machine learning model is trained with the optimization goal of minimizing the color difference between each real scene acquisition color of the shooting device to be compensated and each predicted color, so as to obtain the target relationship function after obtaining the value of the coefficient to be solved solved by the machine learning model; each predicted color refers to each predicted color obtained by substituting each real scene acquisition color of the reference shooting device into the current relationship function after obtaining the current relationship function based on the current value of the coefficient to be solved;

[0085] A real-life visual disparity fitting LUT of the shooting device to be compensated is established according to the target relationship function.

[0086] For example, the initial relationship function can be y=f(x|θ); wherein x is the real scene captured color of the reference shooting device, y is the real scene captured color of the shooting device to be compensated, and θ is the coefficient to be solved; in actual applications, there may be multiple coefficients to be solved, and the multiple coefficients to be solved may include linear coefficients and / or nonlinear coefficients.

[0087] For example, if the reference camera and the camera to be compensated are of the same type, the coefficient to be solved can be only the linear coefficient; research has found that for high-quality camera of the same brand, the difference in real scene style and color between the cameras is close to the linear deviation, and the same type here can refer to the same manufacturer or the same model of the same manufacturer. For non-similar reference cameras and cameras to be compensated, additional coefficient correction terms can be introduced on the basis of the linear coefficient, and the coefficient to be solved can include linear coefficients and non-linear coefficients.

[0088] Assume that the colors of the real-life color palette are a, b, and c. For the sake of convenience in illustration, the color values ​​are represented by simple characters. In practical applications, as described in the foregoing embodiments, the color values ​​can be represented by RGB three-channel values.

[0089] Assuming that the reference shooting device shoots the above three real scene cards, the acquired colors are Da, Db and Dc respectively; and the acquired colors of the shooting device 1 to be compensated are Ka, Kb and Kc respectively; three equation groups can be obtained: Ka = f (Da), Kb = f (Da), Kc = f (Dc);

[0090] The machine learning model needs to solve the values ​​of the coefficients to be solved in the initial relationship function based on these three equations; for example, the solution process of this embodiment is to solve the value of the coefficient θ in the relationship function y=f(x|θ); different values ​​of θ will make the input x of the function obtain different y; this embodiment hopes to solve the value of θ as accurately as possible; and whether the value of θ is accurate enough is measured by the value of the loss function.

[0091] Specifically, if there are s real-life color cards, there are s pairs of real mapping relationships from the real-life color captured by the reference shooting device to the real-life color captured by the shooting device to be compensated. Different values ​​of θ can obtain different relationship functions: y = f(x|θ); the training process of the machine learning model is to solve different values ​​of θ to determine which value of θ is the best.

[0092] The real scene colors of the s reference shooting devices can be substituted into the solved value y=f(x|θ) to obtain the corresponding s y values; the color difference between these s y values ​​and the s real scene colors of the shooting device to be compensated is the value of the loss function. The greater the color difference between these s y values ​​and the s real scene colors of the shooting device to be compensated, the less accurate the current value of θ is; the smaller the color difference is, the more accurate the current value of θ is, and the color difference is negatively correlated with the current value of θ.

[0093] Therefore, the color difference in this embodiment refers to the color difference between each real scene captured color of the shooting device to be compensated and each predicted color; each predicted color refers to the predicted colors obtained by substituting each real scene captured color of the reference shooting device into the current relationship function after obtaining the current relationship function based on the current value of the coefficient to be solved.

[0094] The specific calculation method of color difference can be set as needed. For example, specific indicators such as ΔE (a standard that uses the metric distance in the ITP color space to quantify the difference between two colors) that is close to the real sense of the human eye can be used as color difference. Color difference is the loss function of the machine learning model. When the loss function value reaches the preset value or the calculation exceeds the specified number of iterations, the training can be stopped, and the numerical value of the coefficient to be solved can be obtained, and then the target relationship function can be obtained.

[0095] In practical applications, the initial values ​​of the model parameters of the machine learning model (i.e., the coefficients θ to be solved in the aforementioned relationship function) can be configured as needed. In order to improve the efficiency of solving, in this embodiment, during the training process of the machine learning model, the initial values ​​of the model parameters can be calculated by a linear regression algorithm, and then the model parameters can be adjusted using an iterative optimization algorithm. For example, the linear regression algorithm can be an algorithm such as the least squares method, and the iterative optimization algorithm can be an algorithm such as the gradient descent method. The aforementioned multiple sets of equations can be fitted first by a linear regression algorithm, and a set of initial linear coefficient values ​​can be quickly solved so that the model parameters have a better starting point, and then all model parameters can be gradually adjusted using an iterative optimization algorithm, which can achieve model convergence more efficiently.

[0096] After obtaining the target relationship function, the target relationship function can be used to construct a standard real-life color disparity fitting LUT; for example, the real-life color disparity fitting LUT to be constructed needs to include a standard input-output color pair, for example, a standard input color value is d, and d is used as the x value in the target relationship function, then the y value can be calculated. For example, similar to the format requirements of the rendering engine for LUT mentioned in the above embodiment, assuming that the requirements of the real-life color disparity fitting LUT are a mapping relationship from a color set P to a color set W, if a color p1 in the color set P is the same as the real-life scene acquisition color of the real-life scene card v by the reference shooting device, then the real-life scene acquisition color wi of the real-life scene card v by the camera to be compensated can be known, and the mapping relationship between p1 and wi is obtained; if a color p2 in the color set P is not acquired by the reference shooting device, then p2 is substituted into x in the target relationship function, and the y value is calculated, and the mapping relationship between p2 and the calculated y value can be obtained.

[0097] In practical applications, when there are multiple shooting devices to be compensated, the real color difference fitting LUT of each shooting device to be compensated can be constructed by the above embodiment. In this way, the real color difference fitting LUT between the shooting devices can be obtained, so that the real color difference between the shooting devices is fitted.

[0098] Based on the above embodiment, it is possible to construct: a virtual color disparity fitting LUT of a reference shooting device; and each shooting device to be compensated has a virtual color disparity fitting LUT and a real color disparity fitting LUT. Based on this, these LUTs can be used to perform color calibration on the content to be displayed corresponding to each shooting device during actual shooting.

[0099] In some examples, the virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

[0100] For example, for each shooting device to be compensated, the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated are connected in series in the same color space, and the color values ​​corresponding to the input and output of the series LUT are recorded, which can be saved as the color difference fitting LUT of the shooting device to be compensated.

[0101] For example, the real color difference fitting LUT includes multiple pairs of input-output color values, and the virtual color difference fitting LUT also includes multiple pairs of input-output color values. The color difference fitting LUT represents a color value, which is used as the input of the virtual color difference fitting LUT to obtain the corresponding output value, and then used as the input of the real color difference fitting LUT to obtain the output value of the real color difference fitting LUT. The establishment of the color difference fitting LUT can be: for the input color value sequence C1-in of the virtual color difference fitting LUT, the corresponding output color value sequence C2-out in the virtual color difference fitting LUT is obtained; then the output color value sequence C2-out is used as each input color value of the real color difference fitting LUT to obtain the corresponding output value color value sequence C3-out in the real color difference fitting LUT. The mapping relationship between C1-in and C3-out is the color difference fitting LUT.

[0102] Since there is no color difference between the reference shooting device and itself, you can directly mount your own virtual scene color difference fitting LUT to achieve color consistency between the virtual scene and the virtual asset editor. For other shooting devices to be compensated, connect their virtual scene color difference fitting LUT and real scene color difference fitting LUT in the same color space, record the input and output corresponding color values ​​of the series LUT, and save it as the color difference fitting LUT of the shooting device to be compensated for color difference. By mounting the color difference fitting LUT between shooting devices to the corresponding shooting device, you can achieve color difference fitting between the reference shooting device and the shooting device to be compensated, and achieve virtual and real alignment of all shooting devices.

[0103] like Figure 3 FIG. 1 is a flowchart of a color calibration method according to an exemplary embodiment of the present specification. The method can be applied to a virtual shooting system, wherein the virtual shooting system further includes a display screen for displaying a virtual scene and a plurality of shooting devices, wherein the plurality of shooting devices include a reference shooting device and a shooting device to be compensated. The method can include the following steps:

[0104] In step 302, a virtual scene to be displayed on the display screen is obtained;

[0105] In step 304, after determining the content to be displayed in the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using the virtual color difference fitting LUT of the reference shooting device;

[0106] In step 306, after determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the to-be-compensated shooting devices;

[0107] The virtual color disparity fitting LUT and real color disparity fitting LUT of the photographing device to be compensated, and the virtual color disparity fitting LUT of the reference photographing device are obtained through the steps of the aforementioned color lookup table LUT establishment embodiment.

[0108] The method of this embodiment can be applied to any computer device with computing power in a virtual shooting system. As an example, considering that a rendering engine is used to generate a virtual scene of a display screen, this embodiment can be applied to a computer device equipped with a rendering engine, for example, the program corresponding to the method of this embodiment can be configured in the rendering engine.

[0109] For example, in an actual shooting scene, the rendering engine can obtain a virtual scene to be displayed on a display screen; and in actual shooting, each shooting device will correspond to different display areas on the shooting display screen, such as Figure 2B As shown, the two shooting devices correspond to different display areas. The correspondence between the shooting devices and the display areas can be pre-configured in actual shooting, so that the rendering engine can determine the content to be displayed corresponding to each shooting device according to the configured correspondence and the virtual scene to be displayed. It can be understood that the virtual scene to be displayed contains the content to be displayed corresponding to each shooting device.

[0110] For the content to be displayed of the reference shooting device, the virtual color disparity fitting LUT of the reference shooting device can be used for color calibration, that is, the color of the content to be displayed is mapped to the color of the content to be displayed after color calibration through the virtual color disparity fitting LUT, and the color-calibrated content to be displayed is displayed on the display area corresponding to the reference shooting device. For the content to be displayed of each shooting device to be compensated, the virtual color disparity fitting LUT to be compensated and the real color disparity fitting LUT can be used for color calibration, that is, the color of the content to be displayed can be first calibrated through the virtual color disparity fitting LUT, and then calibrated through the real color disparity fitting LUT, so as to obtain the color-calibrated content to be displayed, which can then be displayed on the display area corresponding to the shooting device to be compensated.

[0111] In some examples, performing color calibration using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the photographing device to be compensated includes:

[0112] Perform color calibration using the color difference fitting LUT of the shooting device to be compensated;

[0113] The color difference fitting LUT of the shooting device to be compensated is obtained by connecting in series the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated.

[0114] In this embodiment, a color difference fitting LUT can be obtained in advance by connecting in series the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated. During color calibration, the color difference fitting LUT can be directly used to perform color calibration on the content to be displayed of the shooting device to be compensated, that is, the color of the content to be displayed is mapped to the color of the calibrated content to be displayed through the color difference fitting LUT, thereby improving the color calibration efficiency.

[0115] As can be seen from the above embodiments, this embodiment only needs to use the color response data of the shooting device to the virtual scene card and the real scene card, and can generate a color lookup table LUT that can compensate for the inherent color difference between the shooting device and the real scene according to the algorithm. The use of the color lookup table can conveniently, quickly and stably achieve the color consistency guarantee of virtual scenes and real scenes between multiple shooting devices, and solve the problems of time-consuming virtual and real alignment of multiple shooting devices in virtual shooting and reliance on subjective manual adjustment. It can greatly shorten the time spent in the color management link of virtual shooting and effectively improve the efficiency of the crew's work.

[0116] The color difference fitting solution between shooting devices in virtual shooting provided in this embodiment can fit the real scene color differences between shooting devices through a machine learning model based on a linear measurement strategy and a parameter iteration algorithm after using the shooting device to shoot the virtual scene card and the real scene card on the specified display screen, and align the virtual scene color differences through the reverse interpolation mapping of the color lookup table, thereby realizing color difference compensation between multiple shooting devices.

[0117] This embodiment adopts a combination of a virtual scene card sampling LUT reverse interpolation algorithm and a real scene card chromatic aberration compensation LUT fitting algorithm in parallel. Compared with the current solution in which experienced visual effects personnel of the crew manually adjust the color palette and shooting equipment parameters for all shooting equipment to compensate for the chromatic aberration of the shooting equipment when shooting each scene with each shooting equipment, this embodiment first eliminates the virtual scene difference between multiple shooting equipment by using a high-precision reverse interpolation virtual scene card acquisition LUT, and then fits the chromatic aberration compensation LUT through a machine learning algorithm based on iterative optimization theory after the virtual and real alignment of the reference shooting equipment, thereby aligning the inherent chromatic aberration between the compensation shooting equipment and the reference shooting equipment, and realizing the compensation of chromatic aberration between multiple shooting equipment. Multiple shooting equipment only needs to perform two rounds of color card sampling to realize the virtual and real alignment of all shooting equipment, which greatly reduces the labor cost and the uncertainty caused by the subjective visual modification of the visual effects director.

[0118] This embodiment performs machine learning modeling on the real scene cards collected by the shooting device, establishes a multivariate equation group in the linear color space, uses the least squares method and other regression algorithms to initialize the linear parameters of the model, and then uses ΔE and other related indicators as loss functions to iteratively optimize the model, so as to obtain the optimal parameters in the nonlinear color space that is more in line with human eye perception, thereby achieving the fitting of the inherent color difference between multiple shooting devices, so that the virtual shooting scene only needs to shoot two rounds of color cards and do a virtual-real alignment to achieve the virtual-real alignment of all shooting devices, which greatly reduces the manual workload and ensures the consistency of the virtual scene and real scene colors of multiple shooting devices. That is, one shooting device completing the virtual-real matching is equivalent to all shooting devices being virtual-real aligned, and generating a full shooting cycle for use once, which greatly reduces the time spent on color management and virtual-real alignment and the reliance on the subjective feelings of professionals, and provides strong support for the commercialization and promotion of virtual shooting.

[0119] Corresponding to the above-mentioned embodiments of the method for establishing a color lookup table LUT / the method for color calibration, this specification also provides embodiments of an apparatus and a computer device to which the apparatus is applied.

[0120] The embodiments of the color lookup table LUT establishment device / color calibration device in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 3 As shown in the figure, it is a hardware structure diagram of the computer device where the color lookup table LUT establishment device / color calibration device of this manual is located. Figure 4In addition to the processor 410, network interface 420, memory 430, and non-volatile memory 440 shown, the computer device where the color lookup table LUT establishment device / color calibration device is located in the embodiment can also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0121] like Figure 5 As shown, Figure 5 This is a block diagram of a device for establishing a color lookup table LUT according to an exemplary embodiment of the present specification, wherein the device is applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and a plurality of shooting devices, wherein the plurality of shooting devices include a reference shooting device and a shooting device to be compensated; the device includes:

[0122] The first establishing module 51 is used to: for each shooting device, after obtaining the virtual scene acquisition colors corresponding to each virtual scene original color displayed by the shooting device on the display screen, establish a virtual scene difference fitting LUT of the shooting device, wherein the virtual scene difference fitting LUT represents a mapping relationship from the virtual scene acquisition color to the virtual scene original color;

[0123] The second establishing module 51 is used to: obtain each real scene acquisition color acquired after each shooting device shoots each real scene color card, and for each shooting device to be compensated, establish the real scene color difference fitting LUT of the shooting device to be compensated according to the relationship between each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated;

[0124] The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen;

[0125] The virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated are used to perform color calibration on the to-be-displayed content in the display area corresponding to the shooting device to be compensated on the display screen.

[0126] In some examples, the first establishing module 51 is further used to:

[0127] Arranging the collected colors of the virtual scenes collected by the shooting device according to the display order of the original colors of the virtual scenes displayed on the display screen to obtain a forward LUT from the original colors of the virtual scenes to the collected colors of the virtual scenes;

[0128] The forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain a virtual color difference fitting LUT of the shooting device.

[0129] In some examples, the second establishing module 52 is further configured to:

[0130] Acquire a first brightness average of each real scene acquisition color of the reference shooting device and a second brightness average of each real scene acquisition color of the shooting device to be compensated;

[0131] Based on the ratio of the second brightness average value to the first brightness average value, adjusting each real scene acquisition color of the shooting device to be compensated;

[0132] According to the relationship between the real scene acquisition colors of the reference shooting device and the adjusted real scene acquisition colors of the shooting device to be compensated, a real scene disparity fitting LUT of the shooting device to be compensated is established.

[0133] In some examples, the second establishing module 52 is further configured to:

[0134] Acquire an initial relationship function between the real scene acquisition color representing the reference shooting device and the real scene acquisition color of the shooting device to be compensated; the initial relationship function includes coefficients to be solved;

[0135] Substituting each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated into the initial relationship function respectively to obtain a plurality of equation groups;

[0136] The multiple equation groups are respectively input into a preset machine learning model, and the machine learning model is trained with the optimization goal of minimizing the color difference between each real scene acquisition color of the shooting device to be compensated and each predicted color, so as to obtain the target relationship function after obtaining the value of the coefficient to be solved solved by the machine learning model; each predicted color refers to each predicted color obtained by substituting each real scene acquisition color of the reference shooting device into the current relationship function after obtaining the current relationship function based on the current value of the coefficient to be solved;

[0137] A real-life visual disparity fitting LUT of the shooting device to be compensated is established according to the target relationship function.

[0138] In some examples, during the training process of the machine learning model, the initial values ​​of the model parameters are first calculated using a linear regression algorithm, and then the model parameters are adjusted using an iterative optimization algorithm.

[0139] In some examples, the second establishing module 52 is further configured to:

[0140] The virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

[0141] like Figure 6 As shown, Figure 6 is a block diagram of another color calibration device according to an exemplary embodiment of the present specification, wherein the device is applied to a virtual shooting system, wherein the virtual shooting system includes a display screen for displaying a virtual scene and a plurality of shooting devices, wherein the plurality of shooting devices include a reference shooting device and a shooting device to be compensated; the device includes:

[0142] An acquisition module 61 is used to: acquire a virtual scene to be displayed on the display screen;

[0143] A first calibration module 61 is used to: after determining the to-be-displayed content of the display area corresponding to the reference shooting device on the display screen based on the virtual scene, perform color calibration using the virtual color difference fitting LUT of the reference shooting device;

[0144] A second calibration module 61 is used to: after determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, perform color calibration using the virtual color difference fitting LUT and the real color difference fitting LUT of the to-be-compensated shooting device;

[0145] The virtual color disparity fitting LUT and real color disparity fitting LUT of the shooting device to be compensated, and the virtual color disparity fitting LUT of the reference shooting device are obtained through the steps of the aforementioned color lookup table LUT establishment method embodiment.

[0146] In some examples, the second calibration module 61 is further used to:

[0147] Perform color calibration using the color difference fitting LUT of the shooting device to be compensated;

[0148] The color difference fitting LUT of the shooting device to be compensated is obtained by connecting in series the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated.

[0149] The implementation process of the functions and effects of each module in the above-mentioned color lookup table LUT establishment device / color calibration device is specifically described in the implementation process of the corresponding steps in the above-mentioned color lookup table LUT establishment method / color calibration method, and will not be repeated here.

[0150] Accordingly, an embodiment of the present specification further provides a computer program product, including a computer program, which implements the steps of the aforementioned color lookup table LUT establishment method / color calibration method embodiment when executed by a processor.

[0151] Correspondingly, an embodiment of the present specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of an embodiment of a method for establishing a color lookup table LUT / a method for color calibration are implemented.

[0152] Accordingly, an embodiment of the present specification further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of an embodiment of a method for establishing a color lookup table LUT / a method for color calibration are implemented.

[0153] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0154] The above embodiments can be applied to one or more computer devices, where the computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0155] The computer device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0156] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0157] The network where the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0158] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this application.

[0160] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0161] The description of "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0162] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0163] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0164] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for establishing a color lookup table LUT, the method being applied to a virtual shooting system, the virtual shooting system comprising a display screen for displaying a virtual scene and a plurality of shooting devices, the plurality of shooting devices comprising a reference shooting device and a shooting device to be compensated; the method comprising: For each shooting device, after obtaining the virtual scene acquisition colors respectively corresponding to the original colors of the virtual scenes displayed by the shooting device on the display screen, a virtual scene difference fitting LUT of the shooting device is established, wherein the virtual scene difference fitting LUT represents a mapping relationship from the virtual scene acquisition colors to the virtual scene original colors; Acquire each real scene acquisition color acquired after each shooting device shoots each real scene color card, and for each shooting device to be compensated, establish a real scene color difference fitting LUT of the shooting device to be compensated according to the relationship between each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated; The virtual color difference fitting LUT of the reference shooting device is used to perform color calibration on the content to be displayed in the display area corresponding to the reference shooting device on the display screen; The virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated are used to perform color calibration on the to-be-displayed content in the display area corresponding to the shooting device to be compensated on the display screen.

2. The method according to claim 1, wherein establishing a virtual color disparity fitting LUT of the shooting device comprises: Arranging the collected colors of the virtual scenes collected by the shooting device according to the display order of the original colors of the virtual scenes displayed on the display screen to obtain a forward LUT from the original colors of the virtual scenes to the collected colors of the virtual scenes; The forward LUT is inverted and interpolated using a reverse interpolation algorithm to obtain a virtual color difference fitting LUT of the shooting device.

3. The method according to claim 1, wherein the establishing of the real scene color disparity fitting LUT of the to-be-compensated shooting device according to the relationship between the real scene colors captured by the reference shooting device and the real scene colors captured by the to-be-compensated shooting device comprises: Acquire a first brightness average of each real scene acquisition color of the reference shooting device and a second brightness average of each real scene acquisition color of the shooting device to be compensated; Based on the ratio of the second brightness average value to the first brightness average value, adjusting each real scene acquisition color of the shooting device to be compensated; According to the relationship between the real scene acquisition colors of the reference shooting device and the adjusted real scene acquisition colors of the shooting device to be compensated, a real scene disparity fitting LUT of the shooting device to be compensated is established.

4. The method according to claim 1, wherein establishing a real-life color disparity fitting LUT of the shooting device to be compensated comprises: Acquire an initial relationship function representing the real scene acquisition color of the reference shooting device and the real scene acquisition color of the shooting device to be compensated; The initial relationship function contains coefficients to be solved; Substituting each real scene acquisition color of the reference shooting device and each real scene acquisition color of the shooting device to be compensated into the initial relationship function respectively to obtain a plurality of equation groups; The multiple equation groups are respectively input into a preset machine learning model, and the machine learning model is trained with the optimization goal of minimizing the color difference between each real scene acquisition color of the shooting device to be compensated and each predicted color, so as to obtain the target relationship function after obtaining the value of the coefficient to be solved solved by the machine learning model; each predicted color refers to each predicted color obtained by substituting each real scene acquisition color of the reference shooting device into the current relationship function after obtaining the current relationship function based on the current value of the coefficient to be solved; A real-life visual disparity fitting LUT of the shooting device to be compensated is established according to the target relationship function.

5. According to the method described in claim 4, during the training process of the machine learning model, the initial values ​​of the model parameters are first calculated by a linear regression algorithm, and then the model parameters are adjusted using an iterative optimization algorithm.

6. The method according to claim 1, further comprising: The virtual color difference fitting LUT and the real color difference fitting LUT of each of the shooting devices to be compensated are connected in series to obtain the color difference fitting LUT of the shooting device to be compensated; the color difference fitting LUT is used to perform color calibration on the content to be displayed in the display area corresponding to the shooting device to be compensated on the display screen.

7. A color calibration method, the method being applied to a virtual shooting system, the virtual shooting system comprising a display screen for displaying a virtual scene and a plurality of shooting devices, the plurality of shooting devices comprising a reference shooting device and a shooting device to be compensated; the method comprising: Acquire a virtual scene to be displayed on the display screen; After determining the to-be-displayed content of the display area corresponding to the reference shooting device on the display screen based on the virtual scene, color calibration is performed using a virtual color difference fitting LUT of the reference shooting device; After determining the to-be-displayed content of the display area corresponding to each of the to-be-compensated shooting devices on the display screen based on the virtual scene, color calibration is performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the to-be-compensated shooting devices; The virtual image disparity fitting LUT and real image disparity fitting LUT of the shooting device to be compensated, and the virtual image disparity fitting LUT of the reference shooting device are obtained through the steps of the method according to any one of claims 1 to 6.

8. The method according to claim 7, wherein the color calibration is performed using the virtual color disparity fitting LUT and the real color disparity fitting LUT of the shooting device to be compensated, comprising: Perform color calibration using the color difference fitting LUT of the shooting device to be compensated; The color difference fitting LUT of the shooting device to be compensated is obtained by connecting in series the virtual color difference fitting LUT and the real color difference fitting LUT of the shooting device to be compensated.

9. A virtual shooting system, comprising a main control terminal, a rendering engine, a display screen for displaying a virtual scene, and a plurality of shooting devices, wherein the plurality of shooting devices comprises a reference shooting device and a shooting device to be compensated; The main control end is used to implement the steps of the method described in any one of claims 1 to 6; the rendering engine is used to implement the steps of the method described in any one of claims 7 or 8.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images