Light alignment method and device in virtual shooting and storage medium

Abstract

The invention relates to a light alignment method and device in virtual shooting and a storage medium. The method comprises the steps that a camera picture comprising a first illumination reference sphere and a second illumination reference sphere is acquired, the first illumination reference sphere is arranged at a first position in a physical scene, and the second illumination reference sphere is arranged at a second position associated with the first position in a display screen used for virtual shooting; based on the difference value of the color values of the first illumination reference sphere and the second illumination reference sphere in the camera picture, one or more of the brightness, the color temperature and the color cast correction channel parameters of the illumination equipment in the physical scene setting are adjusted, so that the color features of the first illumination reference sphere and the second illumination reference sphere in the camera picture are consistent. According to the embodiment of the invention, automatic alignment of virtual and real illumination can be completed in the virtual shooting scene without depending on a specific RGB lamp, so that the light atmosphere of the virtual and real scene is kept consistent, and the production efficiency and the image fusion degree of virtual shooting are improved.

Description

Technical Field

[0001] This disclosure relates to the field of virtual shooting technology, and in particular to a method, apparatus and storage medium for aligning lights in virtual shooting. Background Technology

[0002] Virtual filming technology refers to actors performing in an immersive environment constructed on a display screen (usually a light-emitting diode (LED) screen), where the screen plays virtual scenes generated by a rendering engine in real time. Through camera tracking technology, the director's monitor can present a picture that perfectly blends reality and virtuality in real time, achieving a "what you see is what you get" effect.

[0003] In such virtual shooting projects, the matching of virtual and real lighting is a key factor in determining the realism and immersion of the image. Specifically, the lighting characteristics of the physical lights on the shooting location (foreground) need to be highly consistent and synchronized with the ambient lighting in the virtual assets on the display screen (background). However, in current technology, the alignment of virtual and real lights mainly relies on lighting technicians manually adjusting them based on visual observation. This method is overly dependent on human experience, resulting in low adjustment efficiency, long processing times, and the final lighting effect often has a strong subjective and unstable nature, making it difficult to meet the high-precision shooting requirements. Summary of the Invention

[0004] In view of this, this disclosure proposes a method, apparatus and storage medium for light alignment in virtual shooting.

[0005] According to one aspect of this disclosure, a method for aligning lights during virtual shooting is provided. The method includes:

[0006] Acquire a camera view including a first lighting reference sphere and a second lighting reference sphere. The first lighting reference sphere is set at a first position in the physical scene, and the second lighting reference sphere is set at a second position associated with the first position on a display screen used for virtual shooting.

[0007] Based on the difference in color values ​​between the first and second lighting reference spheres in the camera image, adjust one or more of the parameters of the brightness, color temperature, and color shift correction channel of the lighting equipment in the physical set to make the color characteristics of the first and second lighting reference spheres in the camera image consistent.

[0008] In one possible implementation, based on the difference in color values ​​between a first lighting reference sphere and a second lighting reference sphere in the camera image, one or more of the following parameters of the lighting equipment in the physical set are adjusted: brightness, color temperature, and color cast correction channel parameters:

[0009] Based on the difference in RGB color values ​​of the first and second lighting reference spheres in the camera image, the difference in each component of the first and second lighting reference spheres in the LAB color space is determined.

[0010] Based on the differences between the components of the first and second lighting reference spheres in the LAB color space, adjust one or more of the parameters of the brightness, color temperature, and color deviation correction channel of the lighting equipment in the physical set.

[0011] In one possible implementation, based on the differences between the components of the first and second illumination reference spheres in the LAB color space, one or more of the parameters of the brightness, color temperature, and color shift correction channels of the lighting equipment in the physical set are adjusted, including:

[0012] When the difference between any one or more components of the first and second illumination reference spheres in the LAB color space exceeds the corresponding preset range, a compensation value for the corresponding component is determined. The compensation value is used to control the amount of parameter adjustment.

[0013] Adjust one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value;

[0014] The adjusted brightness, color temperature, and color deviation correction channel parameter values ​​are sent to the lighting equipment in the physical setting to control the lighting equipment to adjust the light output.

[0015] In one possible implementation, the method further includes:

[0016] After the lighting equipment adjusts the light output, the steps of acquiring camera images including the first and second lighting reference spheres and subsequent steps are repeated until the difference between all components of the first and second lighting reference spheres in the LAB color space does not exceed the corresponding preset range.

[0017] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0018] The brightness parameter value is adjusted based on the compensation values ​​corresponding to the brightness components of the first and second illumination reference spheres in the LAB color space.

[0019] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0020] The color temperature parameter value is adjusted based on the compensation values ​​corresponding to the blue and yellow components of the first and second illumination reference spheres in the LAB color space.

[0021] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0022] Based on the red and green components of the first and second illumination reference spheres in the LAB color space, adjust the color shift correction channel parameter values.

[0023] In one possible implementation, the color value of the first lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the first lighting reference sphere in the camera image, and the color value of the second lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the second lighting reference sphere in the camera image.

[0024] According to another aspect of this disclosure, a light alignment device for virtual shooting is provided. The device includes:

[0025] The acquisition module is used to acquire camera images including a first lighting reference sphere and a second lighting reference sphere. The first lighting reference sphere is set at a first position in the physical scene, and the second lighting reference sphere is set at a second position associated with the first position on a display screen used for virtual shooting.

[0026] The adjustment module is used to adjust one or more of the following parameters of the lighting equipment in the physical setting: brightness, color temperature, and color shift correction channel, based on the difference in color values ​​of the first and second lighting reference spheres in the camera image, so as to make the color characteristics of the first and second lighting reference spheres in the camera image consistent.

[0027] In one possible implementation, the module is adjusted for:

[0028] Based on the difference in RGB color values ​​of the first and second lighting reference spheres in the camera image, the difference in each component of the first and second lighting reference spheres in the LAB color space is determined.

[0029] Based on the differences between the components of the first and second lighting reference spheres in the LAB color space, adjust one or more of the parameters of the brightness, color temperature, and color deviation correction channel of the lighting equipment in the physical set.

[0030] In one possible implementation, based on the differences between the components of the first and second illumination reference spheres in the LAB color space, one or more of the parameters of the brightness, color temperature, and color shift correction channels of the lighting equipment in the physical set are adjusted, including:

[0031] When the difference between any one or more components of the first and second illumination reference spheres in the LAB color space exceeds the corresponding preset range, a compensation value for the corresponding component is determined. The compensation value is used to control the amount of parameter adjustment.

[0032] Adjust one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value;

[0033] The adjusted brightness, color temperature, and color deviation correction channel parameter values ​​are sent to the lighting equipment in the physical setting to control the lighting equipment to adjust the light output.

[0034] In one possible implementation, the device further includes:

[0035] The iterative adjustment module is used to re-execute the steps of acquiring camera images including the first and second lighting reference spheres after the lighting equipment adjusts the light output, until the difference between all components of the first and second lighting reference spheres in the LAB color space does not exceed the corresponding preset range.

[0036] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0037] The brightness parameter value is adjusted based on the compensation values ​​corresponding to the brightness components of the first and second illumination reference spheres in the LAB color space.

[0038] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0039] The color temperature parameter value is adjusted based on the compensation values ​​corresponding to the blue and yellow components of the first and second illumination reference spheres in the LAB color space.

[0040] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0041] Based on the red and green components of the first and second illumination reference spheres in the LAB color space, adjust the color shift correction channel parameter values.

[0042] In one possible implementation, the color value of the first lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the first lighting reference sphere in the camera image, and the color value of the second lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the second lighting reference sphere in the camera image.

[0043] According to another aspect of this disclosure, a light alignment device for virtual shooting is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0044] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0045] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0046] According to embodiments of this disclosure, by acquiring a camera image including a first illumination reference sphere and a second illumination reference sphere, with the first illumination reference sphere positioned at a first position in the physical scene and the second illumination reference sphere positioned at a second position associated with the first position on a display screen used for virtual shooting, and adjusting one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene based on the difference in color values ​​of the first illumination reference sphere and the second illumination reference sphere in the camera image, the color characteristics of the first illumination reference sphere and the second illumination reference sphere in the camera image can be made consistent. This allows for efficient, objective, and accurate automatic alignment of virtual and real lighting in a virtual shooting scene without relying on specific expensive RGB lighting fixtures. This ensures that the lighting atmosphere (especially color temperature and color shift correction channel parameters) of the actual physical scene and the virtual scene are consistent, significantly improving the production efficiency and image fusion of virtual shooting.

[0047] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0048] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0049] Figure 1 A schematic diagram illustrating an application scenario according to an embodiment of this disclosure is shown.

[0050] Figure 2 A flowchart illustrating a method for aligning lights during virtual shooting according to an embodiment of the present disclosure is shown.

[0051] Figure 3 A schematic diagram showing the placement of the sphere according to an embodiment of the present disclosure is provided.

[0052] Figure 4 A structural diagram of a light alignment device in virtual shooting according to an embodiment of the present disclosure is shown.

[0053] Figure 5 This is a block diagram illustrating an apparatus 1900 for aligning lights during virtual shooting, according to an exemplary embodiment. Detailed Implementation

[0054] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0055] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0056] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0057] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0058] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0059] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0060] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0061] Virtual filming technology refers to actors performing in an immersive environment constructed on a display screen (usually a light-emitting diode (LED) screen), where the screen plays virtual scenes generated by a rendering engine in real time. Through camera tracking technology, the director's monitor can present a picture that perfectly blends reality and virtuality in real time, achieving a "what you see is what you get" effect.

[0062] In such virtual shooting projects, the matching of virtual and real lighting is a key factor in determining the realism and immersion of the image. Specifically, the lighting characteristics of the physical lights on the shooting location (foreground) need to be highly consistent and synchronized with the ambient lighting in the virtual assets on the display screen (background). However, in current technology, the alignment of virtual and real lights mainly relies on lighting technicians manually adjusting them based on visual observation. This method is overly dependent on human experience, resulting in low adjustment efficiency, long processing times, and the final lighting effect often has a strong subjective and unstable nature, making it difficult to meet the high-precision shooting requirements.

[0063] Existing technology proposes a method for synchronizing virtual and real lighting in virtual shooting. This method achieves synchronization by keeping the virtual lights in Unreal Engine (UE) and real-world physical lights in the same position and outputting identical Digital Multiplex (DMX) parameters. However, the lighting model in the virtual rendering engine differs fundamentally from the light-emitting mechanism of real-world physical lights; their input-output response curves are not linearly correlated. Even with identical DMX inputs, the lighting effects presented by the virtual and physical lights will deviate. Therefore, this control method, which relies solely on identical input signals, cannot truly achieve precise alignment of lighting effects.

[0064] Existing technologies also include a method for aligning the colors of virtual and real lights. This method matches colors by adjusting the RGB values ​​of the lights. However, this method suffers from poor hardware compatibility (limited versatility): it requires the use of full-color physical lights with adjustable RGB components on set. In actual film and television shooting environments, dual-color temperature lights or traditional lights that only support color temperature and brightness adjustment are widely used. If the on-site equipment does not support RGB adjustment, this existing technology cannot be implemented. This significantly limits the application scope of this technology across different shooting levels and equipment conditions.

[0065] In view of this, the present disclosure provides a method, apparatus, and storage medium for lighting alignment in virtual shooting. The method of this disclosure acquires a camera image including a first lighting reference sphere and a second lighting reference sphere. The first lighting reference sphere is positioned at a first location in the physical scene, and the second lighting reference sphere is positioned at a second location associated with the first location on a display screen used for virtual shooting. Based on the difference in color values ​​between the first and second lighting reference spheres in the camera image, one or more of the following parameters—brightness, color temperature, and color shift correction channel parameters—of the lighting equipment in the physical scene are adjusted to ensure that the color characteristics of the first and second lighting reference spheres in the camera image are consistent. This method can achieve efficient, objective, and accurate automatic alignment of virtual and real lighting in a virtual shooting scene without relying on specific expensive RGB lighting fixtures. It ensures that the lighting atmosphere (especially color temperature and color shift correction channel parameters) of the actual physical scene and the virtual scene are consistent, significantly improving the production efficiency and image blending of virtual shooting.

[0066] Figure 1 The diagram illustrates an application scenario according to an embodiment of the present disclosure. The light alignment method in virtual shooting according to this embodiment can be used in virtual shooting scenarios, such as... Figure 1 As shown, the shooting location may include a display screen (such as an LED display screen), a physical set (foreground) located in front of the display screen, cameras, and other related equipment. The lighting for the foreground is provided by physical lighting equipment, while the virtual scene rendered on the display screen can present lighting effects corresponding to the physical lighting, thereby achieving a shooting environment that blends the virtual and real worlds.

[0067] Before the actual virtual shooting, a physical video gray sphere can be placed at the center of the physical set as the first lighting reference sphere; and a virtual video gray sphere of the same size and material can be rendered at the corresponding position on the display screen (such as the area corresponding to the center position in the virtual space) as the second lighting reference sphere. By using a camera to capture images that simultaneously contain the physical and virtual video gray spheres, the brightness, color temperature, or color cast parameters of the physical lighting equipment can be adjusted using the method of this embodiment. When the virtual scene on the display screen changes lighting under different shooting requirements, the alignment process of this embodiment can be re-executed to achieve real-time or periodic correction of the physical lighting parameters on site, so as to automatically control and calibrate the actual physical lighting parameters on site, keep the virtual and real lighting consistent, thereby meeting the requirements of automatic lighting alignment in virtual shooting and improving the blending and realism of the captured image. It should be noted that whether lighting alignment is performed every time the lighting changes can be flexibly determined according to the pre-configured triggering mechanism. For example, light alignment can be proactively triggered only once at key points such as scene switching, light preset switching, and camera position adjustment. Alternatively, relevant personnel can manually trigger light alignment according to actual shooting needs, without having to calibrate every frame for every minor change, thus balancing calibration accuracy and actual shooting efficiency.

[0068] The methods described in this disclosure can be used in terminal devices or servers. The terminal devices involved in this disclosure can be any one or more of the following: mobile phones, foldable electronic devices, tablet computers, desktop computers, laptop computers, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), and in-vehicle devices. This application does not impose any special limitations on the specific type of terminal device; it can have wired or wireless communication capabilities.

[0069] The server disclosed herein can be located locally or in the cloud, and can be a physical device or a virtual device, such as a virtual machine or container. It possesses wireless communication capabilities, which can be configured within the server's chip (system) or other components. These wireless communication capabilities can be implemented through mobile communication technologies such as 2G / 3G / 4G / 5G, as well as Wi-Fi, Bluetooth, frequency modulation (FM), data radio, and satellite communication. Alternatively, communication can be achieved via a wired connection to enable interaction with other devices.

[0070] Figure 2 A flowchart illustrating a light alignment method in virtual shooting according to an embodiment of the present disclosure is shown. Figure 2As shown, the method may include:

[0071] Step S201: Acquire camera images including the first lighting reference sphere and the second lighting reference sphere.

[0072] The illumination reference sphere can be, for example, as shown below. Figure 1 The gray sphere in this context refers to a sphere with standard reflectivity (typically 18% neutral gray) and diffuse reflection characteristics. Its surface is designed to prevent highlights or color shifts, objectively reflecting the intensity, color temperature, and direction of light in its environment, thus serving as a reference for lighting alignment in the technical solutions disclosed herein.

[0073] The first lighting reference sphere can be set at a first position in the physical set to represent the real-world lighting conditions, i.e., the actual lighting effects produced by the lighting equipment. The second lighting reference sphere is set at (or rendered on) a second position associated with the first position on a display screen (such as an LED display screen) used for virtual shooting to represent the virtual scene lighting conditions, i.e., the lighting effects generated by virtual lighting rendering.

[0074] The primary position can be the center of the physical set, such as the area where the main actors stand and perform or the location of key props, where the lighting is the visual focal point of the image.

[0075] The second position can be a spatial coordinate point rendered on the display screen in the three-dimensional coordinate system of a virtual scene (e.g., UE or Unity engine), which is associated with the first position in physical space.

[0076] Figure 3 A schematic diagram showing the placement of the sphere according to an embodiment of the present disclosure is provided. Figure 3 As shown, the spatial coordinates of the second position can be any spatial coordinate point near the third position (but not including the third position itself). The third position is a point rendered on the screen in the virtual scene that is symmetrical to the first position in physical space relative to the screen. That is, after spatial transformation, the third position in the three-dimensional coordinate system of the virtual scene corresponds to the coordinates of the first position in the coordinate system of physical space. To avoid the virtual sphere overlapping with the physical sphere at the first position in the camera view, the second position can be chosen to be near the third position, and a distance of not less than the diameter of the sphere can be maintained between them. A virtual sphere of the same size and material as the physical sphere at the first position can be placed around the second position (i.e., around the third position) to improve the effect of subsequent light alignment. The first, second, and third positions can be the center positions of the spheres.

[0077] Step S202: Based on the difference in color values ​​between the first and second lighting reference spheres in the camera image, adjust one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene to make the color characteristics of the first and second lighting reference spheres in the camera image consistent.

[0078] The camera view can be from Figure 1 The image shown can be any frame captured by the camera, which can be of any type as long as it has the ability to capture color information; the camera image can also be any frame from a video of any length captured by the camera.

[0079] The color values ​​of the first and second lighting reference spheres can be RGB (red, green, blue) color values. The lighting equipment can be any type of luminaire used for physical set lighting, including but not limited to LED lights, tungsten lamps, fluorescent lamps, or other dimmable luminaires with DMX control interfaces for film studios. Its function is to provide adjustable lighting effects for the physical foreground on the shooting location. It should be noted that, even if the on-site lighting equipment does not support RGB adjustment, this disclosure embodiment is also applicable to dual-color temperature luminaires or traditional luminaires that only support brightness and color temperature adjustment. Even if such luminaires do not have RGB channels, automatic alignment of real and virtual lighting can be achieved by adjusting their brightness, color temperature, or color shift correction parameters.

[0080] The brightness, color temperature, and color shift correction parameters of lighting equipment can be set through the Digital Multiplex (DMX) protocol (such as the DMX512 protocol). Among them, the brightness parameter is used to determine the intensity of the luminous flux output by the lighting equipment, which affects the overall brightness of the illumination. The color temperature parameter is used to adjust the warm or cool color temperature of the light source, such as transitioning from warm yellow light to cool blue light, so that the color tone trend of virtual and real lighting is consistent. The color shift correction channel parameter can be a green / magenta correction channel, which is used to make minor adjustments to the green color component of the light source to compensate for the color shift of the light source itself or the color shift caused by the ambient reflection, thereby improving the accuracy of matching virtual and real lighting.

[0081] The aforementioned color features can characterize the lighting atmosphere in a virtual shooting scene, such as brightness levels, overall color temperature trends, and ambient color shift. These can be characterized by the values ​​of any one or more components in the LAB color space, or by chromaticity values ​​(such as CIE) and color temperature (CCT). This disclosure does not impose any limitations on these aspects. By ensuring that the color features of the first and second lighting reference spheres in the camera image are consistent, it is possible to ensure that the real lighting received by the physical set area matches the virtual lighting in the virtual scene in terms of lighting atmosphere, thereby achieving consistency between virtual and real lighting and improving the naturalness and blending of the final image.

[0082] According to embodiments of this disclosure, by acquiring a camera image including a first illumination reference sphere and a second illumination reference sphere, with the first illumination reference sphere positioned at a first position in the physical scene and the second illumination reference sphere positioned at a second position associated with the first position on a display screen used for virtual shooting, and adjusting one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene based on the difference in color values ​​of the first illumination reference sphere and the second illumination reference sphere in the camera image, the color characteristics of the first illumination reference sphere and the second illumination reference sphere in the camera image can be made consistent. This allows for efficient, objective, and accurate automatic alignment of virtual and real lighting in a virtual shooting scene without relying on specific expensive RGB lighting fixtures. This ensures that the lighting atmosphere (especially color temperature and color shift correction channel parameters) of the actual physical scene and the virtual scene are consistent, significantly improving the production efficiency and image fusion of virtual shooting.

[0083] Since physical lighting equipment is usually placed above physical set in virtual shooting scenes, in one possible implementation, the color value of the first lighting reference sphere can be the average color value of the pixel area corresponding to the upper half of the first lighting reference sphere in the camera image, and the color value of the second lighting reference sphere can be the average color value of the pixel area corresponding to the upper half of the second lighting reference sphere in the camera image.

[0084] The average color value of the upper pixel area can be obtained by averaging the RGB (three-channel) color values ​​of all pixels in that area in the camera image.

[0085] By obtaining the average color value of the pixel region in the upper half of the sphere, the main lighting characteristics from the lighting equipment can be reflected more directly, avoiding interference from ground reflections, scene reflections, or ambient stray light on color acquisition. This results in more accurate and stable color values, improving the calculation accuracy of subsequent alignment of virtual and real lighting. In this embodiment, other methods can also be used, such as taking the average color value of the entire sphere as the color values ​​of the first and second lighting reference spheres.

[0086] In one possible implementation, in step S202, the following can be done:

[0087] Based on the difference in RGB color values ​​between the first and second lighting reference spheres in the camera image, determine the difference in each component of the first and second lighting reference spheres in the LAB color space; based on the difference in each component of the first and second lighting reference spheres in the LAB color space, adjust one or more of the parameters of the brightness, color temperature, and color shift correction channel of the lighting equipment in the physical set.

[0088] The RGB color value of the lighting reference sphere can be the average of the RGB color values ​​of the pixel region in the upper half of the corresponding sphere. By calculating the average of the three RGB channels of the first and second lighting reference spheres respectively, and then subtracting the values ​​of each corresponding channel, the difference in the RGB color values ​​can be obtained.

[0089] The RGB color value difference can be converted to the LAB color space using relevant color space conversion methods to obtain the difference in the luminance component (L), red-green component (a), and blue-yellow component (b). The LAB color space is a uniform color space defined by the International Commission on Illumination (ICI). The luminance component represents the brightness of a color, reflecting its lightness or darkness; the red-green component represents the color component along the red-green axis; and the blue-yellow component represents the color component along the yellow-blue axis.

[0090] For example, RGB color values ​​can be mapped to a linear RGB space, and then the linear RGB can be further converted to an XYZ color space based on the correlation matrix transformation method. Finally, based on the correlation conversion formula from XYZ color space to LAB color space, the XYZ values ​​can be converted to LAB color space to obtain the difference between the corresponding L, a, and b components. The above conversion process can be implemented based on the correlation methods in the existing technology.

[0091] Because the LAB color space has a brightness and color separation structure that is more in line with the visual perception characteristics of the human eye, it can more accurately represent the difference between real and virtual lighting, providing a more reliable basis for subsequent adjustments to lighting brightness, color temperature, and color deviation correction parameters.

[0092] The embodiments of this application are not limited to the above methods. For example, the brightness, color temperature and color deviation correction parameters can also be adjusted directly based on the preset correspondence between the color value difference of the R, G and B channels and the compensation value of one or more of the light brightness, color temperature and color deviation correction parameters.

[0093] In one possible implementation, based on the differences between the components of the first and second illumination reference spheres in the LAB color space, one or more of the parameters of the brightness, color temperature, and color shift correction channels of the lighting equipment in the physical set are adjusted, including:

[0094] When the difference between any one or more components of the first and second illumination reference spheres in the LAB color space exceeds the corresponding preset range, a compensation value for the corresponding component is determined; based on the compensation value, one or more parameter values ​​in the brightness, color temperature, and color deviation correction channels are adjusted; the adjusted brightness, color temperature, and color deviation correction channel parameter values ​​are sent to the lighting equipment of the physical scene to control the lighting equipment to adjust the light output.

[0095] The preset range can be pre-set, for example, based on the acceptable deviation of lighting in the actual virtual shooting scene, the response capability of different light types, and the differences in human eye sensitivity to different components. Different ranges can be set for different components in the LAB color space.

[0096] When the difference between different color components exceeds a preset range, adjustments to the corresponding adjustment object for that color component can be triggered. The adjustment object can be one or more of the parameters in the brightness, color temperature, and color cast correction channels. Different color components can correspond to different adjustment objects. For example, when the difference in brightness component L exceeds a preset brightness range, adjustments to the brightness channel parameters can be triggered; when the difference in red-green component a exceeds a preset color cast range, adjustments to the color cast correction channel (i.e., the green-adding / green-subtracting channel) can be triggered; and when the difference in blue-yellow component b exceeds a preset color temperature range, adjustments to the color temperature channel parameters can be triggered.

[0097] The compensation value can be used to control the adjustment amount of parameters. The sign of the difference between the LAB components indicates the adjustment direction of the corresponding parameter, and the absolute value of the component difference indicates the required adjustment range of the corresponding parameter. For example, the absolute value of the component difference can be directly proportional to the compensation value: the larger the absolute value of the difference for any component, the larger the compensation value can be set, thus increasing the adjustment range of the corresponding parameter; conversely, when the absolute value of the difference is small, the compensation value is small, and the adjustment range of the parameter decreases accordingly. In this way, quantitative adjustment of channels such as brightness, color temperature, and color shift can be achieved, and the adjustment process is smoother, more continuous, and more precise, allowing virtual and real lighting to gradually align without abrupt changes. This application does not limit the specific relationship between the absolute value of the difference and the compensation value.

[0098] For example, when the difference in the luminance component L is positive, the luminance parameter value can be decreased; when the difference is negative, the luminance parameter value can be increased. The magnitude of the decrease or increase in the luminance parameter value can be determined based on the compensation value. The larger the absolute value of the difference, the greater the adjustment of the luminance parameter value; and vice versa. The adjustment direction and logic for the red-green component a and the blue-yellow component b are similar. For example, if the difference in red-green component a is positive, the green component value in the color shift correction channel parameter value can be increased; if it is negative, the red component value in the color shift correction channel parameter value can be increased. If the difference in blue-yellow component b is positive, the color temperature parameter value can be decreased; if it is negative, the color temperature parameter value can be increased.

[0099] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0100] The luminance parameter value is adjusted based on the compensation values ​​corresponding to the luminance components of the first and second illumination reference spheres in the LAB color space; the color temperature parameter value is adjusted based on the compensation values ​​corresponding to the blue and yellow components of the first and second illumination reference spheres in the LAB color space; and the color shift correction channel parameter value is adjusted based on the red and green components of the first and second illumination reference spheres in the LAB color space.

[0101] For example, the difference between the luminance components can be expressed as ΔL = L1 - L2, where L1 is the luminance component of the first illumination reference sphere in the LAB color space, and L2 is the luminance component of the second illumination reference sphere in the LAB color space. When ΔL is positive, the luminance parameter value of the lighting device can be reduced according to the compensation value, and when ΔL is negative, the luminance parameter value of the lighting device can be increased according to the compensation value.

[0102] The difference between the yellow and blue components can be expressed as Δb = b1 - b2, where b1 is the yellow and blue component of the first illumination reference sphere in the LAB color space (the larger the value, the more yellow it is, and the smaller the value, the more blue it is), and b2 is the yellow and blue component of the second illumination reference sphere in the LAB color space. When Δb is positive, the color temperature parameter value of the lighting equipment can be reduced according to the compensation value, and when Δb is negative, the color temperature parameter value of the lighting equipment can be increased according to the compensation value.

[0103] The difference between the red and green components can be expressed as Δa = a1 - a2, where a1 is the red and green component of the first illumination reference sphere in the LAB color space (the larger the value, the more red it is, and the smaller the value, the more green it is), and a2 is the red and green component of the second illumination reference sphere in the LAB color space. When Δa is positive, the green component value in the color shift correction channel parameter value of the lighting device can be increased according to the compensation value. When Δa is negative, the red component value in the color shift correction channel parameter value of the lighting device can be increased according to the compensation value.

[0104] The adjusted brightness, color temperature, and color deviation correction channel parameter values ​​can be sent to the lighting equipment via a digital lighting control protocol (such as the DMX512 protocol) to control the lighting equipment to adjust the light output.

[0105] This allows for smooth and continuous correction of brightness, color temperature, and color deviation, thereby gradually aligning physical lighting with virtual lighting.

[0106] In one possible implementation, the method further includes:

[0107] After the lighting equipment adjusts the light output, repeat step S201 and subsequent steps until the difference between all components of the first and second lighting reference spheres in the LAB color space does not exceed the corresponding preset range.

[0108] This process involves adjusting the light output of the lighting equipment, acquiring a new frame of camera footage, and recalculating the color values ​​of the first and second lighting reference spheres and their corresponding LAB component differences based on this frame. Then, brightness, color temperature, and color cast parameters are adjusted according to these differences. This process is iterated until the virtual and real lighting achieves a preset alignment requirement across the brightness, red-green, and blue-yellow components, thus realizing precise matching of virtual and real lighting. If the "color characteristics" are represented by parameters other than the LAB component differences, other parameters can also be used to determine whether the "color characteristics of the first and second lighting reference spheres in the camera frame are consistent."

[0109] Therefore, by continuously adjusting the parameters of the physical lighting equipment, the lighting atmosphere at corresponding locations in the actual scene can be gradually synchronized with the lighting atmosphere at corresponding locations in the virtual scene, thus achieving precise alignment of virtual and real lighting. This method ensures that the physical lighting conditions at the center of the foreground are completely consistent with the virtual lighting conditions at the corresponding location in the virtual asset scene, achieving a high degree of matching between the virtual and real scenes in terms of brightness, color temperature, and color cast, thereby significantly improving the realism and integration of the virtual shooting image.

[0110] Figure 4 A structural diagram of a light alignment device in virtual shooting according to an embodiment of the present disclosure is shown. Figure 4 As shown, the device includes:

[0111] The acquisition module 401 is used to acquire a camera image including a first lighting reference sphere and a second lighting reference sphere. The first lighting reference sphere is set at a first position in the physical scene, and the second lighting reference sphere is set at a second position associated with the first position on a display screen used for virtual shooting.

[0112] The adjustment module 402 is used to adjust one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene based on the difference in color values ​​of the first lighting reference sphere and the second lighting reference sphere in the camera image, so as to make the color characteristics of the first lighting reference sphere and the second lighting reference sphere in the camera image consistent.

[0113] In one possible implementation, the module is adjusted for:

[0114] Based on the difference in RGB color values ​​of the first and second lighting reference spheres in the camera image, the difference in each component of the first and second lighting reference spheres in the LAB color space is determined.

[0115] Based on the differences between the components of the first and second lighting reference spheres in the LAB color space, adjust one or more of the parameters of the brightness, color temperature, and color deviation correction channel of the lighting equipment in the physical set.

[0116] In one possible implementation, based on the differences between the components of the first and second illumination reference spheres in the LAB color space, one or more of the parameters of the brightness, color temperature, and color shift correction channels of the lighting equipment in the physical set are adjusted, including:

[0117] When the difference between any one or more components of the first and second illumination reference spheres in the LAB color space exceeds the corresponding preset range, a compensation value for the corresponding component is determined. The compensation value is used to control the amount of parameter adjustment.

[0118] Adjust one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value;

[0119] The adjusted brightness, color temperature, and color deviation correction channel parameter values ​​are sent to the lighting equipment in the physical setting to control the lighting equipment to adjust the light output.

[0120] In one possible implementation, the device further includes:

[0121] The iterative adjustment module is used to re-execute the steps of acquiring camera images including the first and second lighting reference spheres after the lighting equipment adjusts the light output, until the difference between all components of the first and second lighting reference spheres in the LAB color space does not exceed the corresponding preset range.

[0122] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0123] The brightness parameter value is adjusted based on the compensation values ​​corresponding to the brightness components of the first and second illumination reference spheres in the LAB color space.

[0124] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0125] The color temperature parameter value is adjusted based on the compensation values ​​corresponding to the blue and yellow components of the first and second illumination reference spheres in the LAB color space.

[0126] In one possible implementation, one or more parameter values ​​in the brightness, color temperature, and color shift correction channels are adjusted based on the compensation value, including:

[0127] Based on the red and green components of the first and second illumination reference spheres in the LAB color space, adjust the color shift correction channel parameter values.

[0128] In one possible implementation, the color value of the first lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the first lighting reference sphere in the camera image, and the color value of the second lighting reference sphere is the average color value of the pixel region corresponding to the upper half of the second lighting reference sphere in the camera image.

[0129] According to embodiments of this disclosure, by acquiring a camera image including a first illumination reference sphere and a second illumination reference sphere, with the first illumination reference sphere positioned at a first position in the physical scene and the second illumination reference sphere positioned at a second position associated with the first position on a display screen used for virtual shooting, and adjusting one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene based on the difference in color values ​​of the first illumination reference sphere and the second illumination reference sphere in the camera image, the color characteristics of the first illumination reference sphere and the second illumination reference sphere in the camera image can be made consistent. This allows for efficient, objective, and accurate automatic alignment of virtual and real lighting in a virtual shooting scene without relying on specific expensive RGB lighting fixtures. This ensures that the lighting atmosphere (especially color temperature and color shift correction channel parameters) of the actual physical scene and the virtual scene are consistent, significantly improving the production efficiency and image fusion of virtual shooting.

[0130] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0131] This disclosure also provides a light alignment device for virtual shooting, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0132] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0133] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0134] Figure 5 This is a block diagram illustrating an apparatus 1900 for light alignment in virtual photography, according to an exemplary embodiment. For example, apparatus 1900 may be provided as a server or terminal device. (Refer to...) Figure 5 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0135] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0136] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0137] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0138] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0139] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0140] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0141] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0142] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0144] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for aligning lights in virtual shooting, characterized in that, The method includes: Acquire a camera view including a first lighting reference sphere and a second lighting reference sphere, wherein the first lighting reference sphere is set at a first position in a physical scene and the second lighting reference sphere is set at a second position in a display screen for virtual shooting that is associated with the first position; Based on the difference in color values ​​between the first and second lighting reference spheres in the camera image, one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical set are adjusted to make the color characteristics of the first and second lighting reference spheres in the camera image consistent.

2. The method according to claim 1, characterized in that, The step of adjusting one or more of the following parameters based on the difference in color values ​​between the first and second lighting reference spheres in the camera image: brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical set: Based on the difference in RGB color values ​​between the first and second lighting reference spheres in the camera image, the difference in each component of the first and second lighting reference spheres in the LAB color space is determined. Based on the differences between the components of the first and second lighting reference spheres in the LAB color space, one or more of the parameters of the brightness, color temperature, and color deviation correction channel of the lighting equipment in the physical scene are adjusted.

3. The method according to claim 2, characterized in that, The step of adjusting one or more of the following parameters—brightness, color temperature, and color shift correction channel parameters—of the lighting equipment in the physical set based on the differences between the components of the first and second lighting reference spheres in the LAB color space includes: When the difference between any one or more components of the first illumination reference sphere and the second illumination reference sphere in the LAB color space exceeds the corresponding preset range, a compensation value for the corresponding component is determined, and the compensation value is used to control the adjustment amount of the parameter. Based on the compensation value, adjust one or more parameter values ​​in the brightness, color temperature, and color deviation correction channels; The adjusted brightness, color temperature, and color deviation correction channel parameter values ​​are sent to the lighting equipment in the physical setting to control the lighting equipment to adjust the light output.

4. The method according to claim 3, characterized in that, The method further includes: After the lighting device adjusts the light output, the steps of acquiring the camera image including the first lighting reference sphere and the second lighting reference sphere and thereafter are repeated until the difference between all components of the first lighting reference sphere and the second lighting reference sphere in the LAB color space does not exceed the corresponding preset range.

5. The method according to claim 3, characterized in that, The adjustment of one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value includes: The brightness parameter value is adjusted based on the compensation values ​​corresponding to the brightness components of the first and second illumination reference spheres in the LAB color space.

6. The method according to claim 3, characterized in that, The adjustment of one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value includes: The color temperature parameter value is adjusted based on the compensation values ​​corresponding to the blue and yellow components of the first and second illumination reference spheres in the LAB color space.

7. The method according to claim 3, characterized in that, The adjustment of one or more parameter values ​​in the brightness, color temperature, and color shift correction channels based on the compensation value includes: Based on the red and green components of the first and second illumination reference spheres in the LAB color space, adjust the color shift correction channel parameter values.

8. The method according to claim 1, characterized in that, The color value of the first illumination reference sphere is the average color value of the pixel area corresponding to the upper half of the first illumination reference sphere in the camera image, and the color value of the second illumination reference sphere is the average color value of the pixel area corresponding to the upper half of the second illumination reference sphere in the camera image.

9. A light alignment device for virtual shooting, characterized in that, The device includes: The acquisition module is used to acquire a camera image including a first lighting reference sphere and a second lighting reference sphere, wherein the first lighting reference sphere is set at a first position in the physical scene and the second lighting reference sphere is set at a second position in the display screen for virtual shooting that is associated with the first position. The adjustment module is used to adjust one or more of the brightness, color temperature, and color shift correction channel parameters of the lighting equipment in the physical scene based on the difference in color values ​​between the first lighting reference sphere and the second lighting reference sphere in the camera image, so as to make the color characteristics of the first lighting reference sphere and the second lighting reference sphere in the camera image consistent.

10. A light alignment device for virtual shooting, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

11. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images