Method and device for realizing alignment of virtual and real light effects in virtual shooting, equipment, medium and program product

Abstract

The invention relates to a method and device for realizing virtual and real light effect alignment in virtual shooting, equipment, a medium and a program product, and the method comprises the steps: obtaining a target color value of a specified virtual material in a virtual scene displayed on a physical screen in a virtual shooting process, the target color value comprises a color value presented by a virtual material under current lighting of virtual light in a virtual scene; according to a target color value of a virtual material specified in the virtual scene, light control parameters for lighting an actual material in a real shooting environment to the target color value are determined, and the actual material and the virtual material are materials of the same object; and sending the light control parameter to a physical light system, so that an actual material in the real shooting environment presents a target color value consistent with a virtual material in a virtual scene displayed on a physical screen under the lighting of the physical light system. Therefore, the lighting efficiency can be improved, and a more accurate virtual-real alignment effect can be obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual photography, and in particular to a method, apparatus, device, medium, and program product for achieving alignment of virtual and real lighting effects in virtual photography. Background Art

[0002] In virtual filming projects based on LED screens, aligning the lighting in the actual foreground shooting area of ​​the shooting scene with the lighting in the virtual asset scene on the shooting scene's LED background screen is one of the important prerequisites for achieving a realistic and vivid virtual filming effect. Generally speaking, the virtual lighting system of the virtual scene assets that need to be played on the LED screen in the early stage is large in number and complex in layout. This makes it difficult to synchronize the lighting of the physical lighting system of the actual shooting scene. In most virtual filming, the virtual and real lighting alignment is achieved by the lighting technician manually adjusting the physical lighting effects. If the virtual lighting alignment of the virtual scene and the real scene is to be achieved during the shooting process for dynamic lighting effects (that is, the virtual scene does not maintain a fixed lighting state during the shooting process, but needs to realize the lighting effects that change in time such as "changing clouds" and "sunrise and sunset"), it is a huge challenge for both the lighting technician and the asset designer. In other words, it is difficult to achieve the virtual and real alignment of dynamic lighting effects through manual adjustment.

[0003] Furthermore, in virtual filming based on LED screens, it is necessary to ensure the consistency of the virtual scene and the real scene in the camera's image. This includes not only the consistency of the physical space of the scene, but also the consistency of the overall atmosphere. The overall atmosphere consistency is mainly affected by the virtual and real materials of the virtual and real scenes, as well as the virtual and real lighting effects, among which the role of lighting is particularly important. In order to make the real objects in the real scene appear the same color under physical lighting as the virtual objects in the virtual scene under virtual lighting, and to ensure the color consistency of the virtual and real objects in the camera's image, professional lighting technicians often need to gradually adjust the lighting effects of the physical lighting system in the real shooting environment. This also reduces the lighting efficiency in virtual filming, and the alignment of virtual and real lights is also relatively subjective. Summary of the Invention

[0004] In view of this, the present disclosure proposes a method and apparatus, equipment, medium and program product for realizing alignment of virtual and real lighting effects in virtual shooting, which can not only automatically realize dynamic alignment of virtual and real lighting effects during virtual shooting, but also ensure that the actual materials in the real shooting environment present the same color as the virtual materials in the virtual scene displayed on the physical screen under the lighting of the physical lighting system, thereby improving lighting efficiency and obtaining a more accurate virtual-reality alignment effect.

[0005] According to one aspect of the present disclosure, a method for achieving alignment of virtual and real lighting effects in virtual shooting is provided, the method comprising: during the virtual shooting process, obtaining a target color value of a virtual material specified in a virtual scene displayed on a physical screen, the target color value comprising the color value presented by the virtual material specified in the virtual scene under the current lighting of the virtual light in the virtual scene; determining, based on the target color value of the virtual material specified in the virtual scene, lighting a real material in a real shooting environment to the target color value; wherein the real material and the virtual material are materials of the same object; and sending the lighting control parameters to the physical lighting system so that the real material in the real shooting environment presents, under the lighting of the physical lighting system, the target color value consistent with the virtual material in the virtual scene displayed on the physical screen.

[0006] In one possible implementation, the method further includes: while keeping the physical camera and the physical lighting system perpendicular to the specified actual material, adjusting the lighting control parameters of the object lighting system according to a specified step size within a value range of the lighting control parameters of the physical lighting system, and simultaneously controlling the physical camera to perform image acquisition to obtain acquired images under different lighting control parameters; obtaining sample color values ​​corresponding to the different lighting control parameters by extracting the color value of the specified actual material in the acquired images under different lighting control parameters; and constructing a target data set including multiple lighting control parameters of the physical lighting system and the sample color values ​​corresponding to each lighting control parameter.

[0007] In one possible implementation, the method of determining the lighting control parameters for lighting the actual material in the real scene to the target color value of the virtual material specified in the virtual scene includes: determining the lighting control parameters for lighting the actual material in the real scene to the target color value based on the target color value of the virtual material specified in the virtual scene using a prediction model; wherein the prediction model is trained using a target data set, and the target data set includes multiple lighting control parameters of the physical lighting system and sample color values ​​corresponding to each lighting control parameter, and the sample color values ​​represent the color of the specified actual material captured by a physical camera under the lighting of the physical lighting system according to the corresponding lighting control parameters; wherein the training process of the prediction model includes: inputting the sample color values ​​in the target data set into the initial model to obtain the predicted lighting control parameters output by the initial model; and adjusting the model parameters of the initial model using the loss between the predicted lighting control parameters and the lighting control parameters corresponding to the input sample color values ​​to obtain the trained prediction model.

[0008] In a possible implementation, the method further includes: generating a color space that can be presented by the physical lighting system based on sample color values ​​corresponding to each of the multiple lighting control parameters of the physical lighting system in the target data set, wherein the sample color values ​​represent the color of a specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameters; wherein, determining the lighting control parameters that illuminate the actual material in the real scene to the target color value based on the target color value of the virtual material specified in the virtual scene includes: mapping the target color value to the color space to obtain a mapped color value; and determining the lighting control parameters based on the mapped color value.

[0009] In one possible implementation, determining the lighting control parameters for lighting an actual material in a real scene to achieve the target color value of the virtual material specified in the virtual scene according to the target color value of the virtual material specified in the virtual scene includes: determining the lighting control parameters corresponding to the target color value of the virtual material specified in the virtual scene based on a pre-constructed mapping relationship between color values ​​and lighting control parameters; wherein the mapping relationship is constructed based on multiple lighting control parameters of the physical lighting system in a target data set and sample color values ​​corresponding to each lighting control parameter, wherein the sample color value represents the color of the specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameters.

[0010] In one possible implementation, the lighting control parameters include brightness parameters and color parameters; the color values ​​include RGB values; wherein, when the physical lighting system performs lighting according to the lighting control parameters, the color of the actual material in the picture captured by the physical camera used in the virtual shooting process is consistent with the color of the virtual material in the picture.

[0011] According to another aspect of the present disclosure, a device for achieving alignment of virtual and real lighting effects in virtual shooting is provided, the device comprising: an acquisition module for acquiring, during the virtual shooting process, a target color value of a virtual material specified in a virtual scene displayed on a physical screen, the target color value comprising the color value presented by the virtual material specified in the virtual scene under the current lighting of the virtual light in the virtual scene; a control parameter determination module for determining, based on the target color value of the virtual material specified in the virtual scene, lighting control parameters for lighting an actual material in a real shooting environment to the target color value; wherein the actual material and the virtual material are materials of the same object; and a sending module for sending the lighting control parameters to the physical lighting system, so that the actual material in the real shooting environment presents, under the lighting of the physical lighting system, the target color value consistent with the virtual material in the virtual scene displayed on the physical screen.

[0012] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0013] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0014] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0015] According to various aspects of the present disclosure, by determining the lighting control parameters of the physical lighting system that illuminates the actual material in the real shooting environment to the target color value based on the target color value presented by the virtual material in the virtual scene under the virtual light, and sending the lighting control parameters to the physical lighting system, not only can the color presented by the actual material under the physical light be automatically aligned with the color presented by the virtual material in the virtual scene under the virtual light, but also when the lighting effect of the virtual scene displayed on the physical screen changes dynamically, it can automatically ensure that the physical lighting system is synchronously aligned with the dynamic lighting effect of the virtual light, thereby achieving dynamic alignment of virtual and real lighting effects, improving lighting efficiency and obtaining a more accurate virtual-reality alignment effect, and also helping to ensure that the color presented by the actual material in the shooting picture of the physical camera during the virtual shooting process under the lighting of the physical lighting system according to the lighting control parameters is consistent with the color presented by the virtual material under the lighting of the virtual light, thereby improving the picture quality of the virtual shooting.

[0016] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0018] Figure 1 A schematic diagram of a virtual shooting system according to an embodiment of the present disclosure is shown.

[0019] Figure 2 A flowchart of a method for achieving alignment of virtual and real lighting effects in virtual shooting according to an embodiment of the present disclosure is shown.

[0020] Figure 3A block diagram of an apparatus for achieving alignment of virtual and real lighting effects in virtual photography according to an embodiment of the present disclosure is shown.

[0021] Figure 4 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0023] As used herein, the terms "comprises," "comprising," "having," or variations thereof are open ended and include one or more stated features, integers, elements, steps, parts, or functions, but do not preclude the presence or addition of one or more other features, integers, elements, steps, parts, functions, or groups thereof.

[0024] When an element is referred to as being "connected," "coupled," "responsive" or variations thereof to another element, it can be directly connected, coupled or responsive to the other element or intervening elements may be present.

[0025] Although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, without departing from the teachings of the present invention, the first element / operation in some embodiments may be referred to as the second element / operation in other embodiments.

[0026] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0027] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0028] Figure 1 A schematic diagram of a virtual shooting system according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the virtual shooting system includes a physical camera 01 , a physical screen 02 , a physical lighting system 03 and a main control device 04 .

[0029] According to the different lenses used, the physical camera 01 can be a telephoto camera, a wide-angle camera, etc.; according to the different shutters used, the physical camera 01 can be a rolling camera, a global camera, etc., and this is not limited by the embodiments of the present disclosure. The physical screen 02 can be an LED screen, which can be a curved screen or a flat screen, etc. The type, quantity, size, resolution, etc. of the physical screen 02 in the virtual shooting system can be customized according to actual needs, and this is not limited by the embodiments of the present disclosure. The physical lighting system 03 can adopt any lighting system known in the art for photographic lighting. It should be understood that those skilled in the art can customize the type, quantity, position, etc. of the lamps in the physical lighting system 03 according to actual needs, and this is not limited by the embodiments of the present disclosure.

[0030] The master control device 04 may be an electronic device with computing and processing control capabilities, such as a desktop computer or laptop computer. The master control device 04 may establish communication connections with the physical camera 01, the physical screen 02, and the physical lighting system 03. It should be understood that the embodiments of the present disclosure do not limit the communication connection method between the devices. For example, the master control device 04 and the physical lighting system 03 may establish a communication connection using the DMX (Digital Multiplex) protocol. The DMX protocol is a digital communication protocol widely used in stage lighting, performing arts venues, and architectural lighting, and is primarily used to control the brightness, color, and other effects of lighting equipment.

[0031] Among them, the main control device 04 can be deployed with an Unreal Engine (UE), and a virtual environment consistent with the real shooting environment can be constructed in the Unreal Engine. The real shooting environment includes the above-mentioned physical camera 01, physical screen 02 and physical lighting system, and the virtual environment includes a virtual camera, a virtual screen and a virtual scene. The virtual scene contains a virtual lighting system and various virtual objects; among them, the camera internal parameters of the virtual camera in the virtual shooting system are consistent with the camera internal parameters of the physical camera 01, and the relative position relationship between the virtual camera and the virtual screen is consistent with the relative position relationship between the physical camera 01 and the physical screen 02; the virtual screen can be understood as a screen model that restores the physical screen 02 in a 1:1 manner, and the shape, size, etc. of the virtual screen are consistent with the physical screen 02.

[0032] In the process of virtual shooting using the above-mentioned virtual shooting system, the main control device 04 can drive the virtual camera in the Unreal Engine to render and generate a rendering picture of the virtual scene, and project the rendering picture onto the screen model after three-dimensional projection transformation, and then map it to the physical screen 02. That is, the physical screen 02 can play the virtual scene produced in advance, and at the same time, the lighting effect of the physical lighting system 03 in the real shooting environment can be adjusted based on the virtual lighting effect in the virtual scene displayed on the physical screen 02 to align with the virtual lighting effect. At the same time, the actor can perform in front of the physical screen 02, and the main control device 04 can control the physical camera 01 to shoot. The shooting picture uses the actor and the surrounding scenery as the foreground and the virtual scene in the physical screen 02 as the background. The shooting picture can also be transmitted synchronously to the director's monitor, and the director's monitor presents the effect of the actor after shooting in the virtual scene in real time, thereby realizing virtual shooting.

[0033] As described above, due to the complexity of the virtual lighting system, achieving virtual and real lighting alignment in a virtual filming system is somewhat difficult. This is especially true when dynamic lighting effects are present in the virtual scene. Therefore, to achieve matching of virtual and real lighting dynamic effects in virtual filming, embodiments of the present disclosure provide a method for achieving virtual and real lighting effect alignment in virtual filming. This method obtains target color values ​​of virtual materials specified in a virtual scene displayed on a physical screen in real time, and determines lighting control parameters that illuminate the actual materials in the real filming environment to the target color values, thereby controlling the lighting effects of the physical lighting system. When the virtual lighting in the virtual scene on the physical screen exhibits dynamic effects, the physical lighting system automatically changes to follow the changes in the virtual lighting effects in the virtual scene on the physical screen. Furthermore, the virtual and real lighting effects of each rendered frame of the virtual scene are aligned. Specifically, based on the target color values ​​of the virtual materials under the dynamic virtual lighting effects in the virtual scene, the lighting control parameters of the physical lighting system are predicted to align the colors of the actual materials with the target colors. Alternatively, the physical lighting system is automatically matched to the dynamic lighting effects in the virtual scene, thereby achieving better filming results. Due to its accuracy and real-time nature, this method can solve the problem of matching virtual and real lighting dynamic effects.

[0034] Furthermore, in the aforementioned virtual filming system, the physical lighting system 03 can generally be controlled using parameters for brightness and color. Brightness determines the intensity of the light, while color determines its color rendering. Color attributes can be controlled using various methods, including color temperature (one parameter), RGB (three parameters), and HSV (three parameters). HSV is a color space that describes color using hue, saturation, and value, while RGB is a color space that describes color using red, green, and blue channels. Controlling light color based on RGB parameters is the most common method. However, the physical lighting systems used in real-world filming vary, and the color spaces they present in cameras also vary. Furthermore, various commercially available physical lighting systems may not guarantee that a specific RGB input will accurately capture the target RGB after the camera captures the image. Currently, achieving a consistent target RGB in the captured image often requires professional lighting technicians to incrementally adjust the brightness and color parameters of the lighting fixtures, reducing lighting efficiency in virtual filming. Therefore, the method proposed in the embodiment of the present disclosure for achieving alignment of virtual and real lighting effects in virtual shooting can automatically generate lighting control parameters such as brightness and color for lighting the actual material in the real shooting environment to the target color value within the color rendering capability of the physical lighting system itself, based on the target color value of the specified virtual material. That is, it can automatically, quickly and accurately make the physical lighting system light up to the desired color, greatly improving the lighting efficiency in virtual shooting.

[0035] It should be noted that the method of the embodiment of the present disclosure can be deployed on various terminal devices (such as the above-mentioned main control device 04) through software or hardware modification. The terminal device involved in the embodiment of the present disclosure may refer to a device with a wireless connection function and / or a wired connection function. The wireless connection function means that it can be connected to other devices through wireless connection methods such as wifi and Bluetooth. The terminal device involved in the embodiment of the present disclosure can also communicate with other devices through a wired connection function. The terminal device involved in the embodiment of the present disclosure can be touch-screen, non-touch-screen, or without a screen. The touch-screen terminal device can be controlled by clicking, sliding, etc. on the display screen with a finger or a stylus. The non-touch-screen device can be connected to an input device such as a mouse, keyboard, touch panel, etc., and the terminal device can be controlled through the input device. For example, a device without a screen can be a Bluetooth speaker without a screen. For example, the terminal device of the present application can include but is not limited to user equipment (UE), mobile device, user terminal, terminal, handheld device, tablet computer, laptop computer, PDA, computing device, etc.

[0036] The method of the embodiment of the present disclosure can also be deployed on a server. The server can be located in the cloud or locally. It can be a physical device or a virtual device, such as a virtual machine, a container, etc., and has a wireless communication function, wherein the wireless communication function can be set in the chip (system) or other parts or components of the server. It can refer to a device with a wireless connection function. The wireless connection function means that it can be connected to other servers or terminal devices through wireless connection methods such as Wi-Fi and Bluetooth. The server involved in the embodiment of the present disclosure can also have the function of communicating through a wired connection. For example, the server of the embodiment of the present disclosure can be located in the cloud, communicate with the terminal device, receive the target color value sent by the terminal device, and use the method deployed on the server to generate corresponding lighting control parameters based on the target color value, and return them to the terminal device, so as to control the physical lighting system for lighting through the terminal device.

[0037] Figure 2 FIG. 1 is a flow chart showing a method for aligning virtual and real lighting effects in virtual shooting according to an embodiment of the present disclosure. Figure 2 As shown, the method includes: steps S11 to S13.

[0038] In step S11, during the virtual shooting process, a target color value of a virtual material specified in a virtual scene displayed on a physical screen is obtained, where the target color value includes the color value presented by the virtual material specified in the virtual scene under the current lighting of the virtual light in the virtual scene.

[0039] It is understandable that before virtual shooting, the virtual lights in the virtual scene are prepared in advance. In particular, when the director needs to shoot dynamic lighting effects, the dynamic effects of virtual lighting changes need to be prepared in the virtual scene (i.e., virtual assets) in advance, without having to set up the dynamic effects of the physical lighting system in advance. During the shooting process, it is only necessary to play the dynamic lighting effects in the virtual scene on the physical screen, and by obtaining in real time the target color values ​​of the virtual materials specified in the virtual scene displayed on the physical screen under the current lighting of the virtual lights in the virtual scene, it is possible to achieve dynamic lighting alignment in which the physical lighting system automatically follows the changes in the virtual lights.

[0040] In actual applications, the UE's own code interface can be used on a device deployed with the Unreal Engine (e.g., the master control device 04 of the virtual shooting system described above), and the scene capture device built into the UE can be used to obtain the color value of the virtual material specified in the rendering image of the virtual scene currently displayed on the physical screen under the current virtual lighting effect as the target color value. That is, the target color value can be the color value in the rendering data to be rendered to the screen. Among them, the specified virtual material generally uses the material of the main objects in the virtual scene and the real shooting scene, such as the material of the ground, etc., and this embodiment of the present disclosure is not limited to this. It should be understood that the user can customize the virtual material in the virtual scene according to actual needs, and this embodiment of the present disclosure is not limited to this. Among them, the target color value can be, for example, any one of RGB value, HSV value, and color temperature value, and this embodiment of the present disclosure is not limited to this. It should be understood that the embodiment of the present disclosure does not limit the method for obtaining the target color value. It should be noted that the embodiment of the present disclosure can ignore the color difference caused by the screen displaying the virtual scene, that is, it can be assumed that the color of the virtual material displayed on the physical screen is equal to the target color value in the above rendering data, making the method for obtaining the target color value simple and efficient. If this color difference is considered, the color of the virtual material displayed on the physical screen can be captured using a color acquisition device such as a colorimeter and used as the target color value. Furthermore, if the color difference caused by the physical camera shooting the physical screen is also considered, the color of the virtual material displayed on the physical screen captured by the physical camera can also be used as the target color value. This further ensures that the virtual material and the actual material color are aligned in the final image captured by the physical camera.

[0041] In step S12, according to the target color value of the virtual material specified in the virtual scene, the lighting control parameters for lighting the actual material in the real shooting environment to the target color value are determined.

[0042] The actual material and the virtual material are the material of the same object. For example, the virtual material and the actual material are both the material of the same ground or the material of the same floor. The lighting control parameters may include brightness parameters and color parameters. The color parameters may be any one of RGB parameters, HSV parameters, and color temperature parameters. This depends on the type of color parameter required to be input by the physical lighting system used in the virtual shooting, which is not limited in this embodiment of the present disclosure.

[0043] In one possible implementation, a neural network model, such as a deep learning model or a machine learning model, can be trained to enable the trained model to predict the lighting control parameters of a physical lighting system. The trained model can then predict the lighting control parameters that will produce the same color value for the corresponding actual material in the real scene based on the target color value of the virtual material specified in the virtual scene. Thus, determining the lighting control parameters that will produce the target color value for the actual material in the real scene based on the target color value of the virtual material specified in the virtual scene can include:

[0044] A prediction model is used to determine lighting control parameters for lighting actual materials in a real scene to achieve target color values ​​based on target color values ​​of virtual materials specified in a virtual scene. The prediction model can be trained using a target data set, which can include multiple lighting control parameters of a physical lighting system and sample color values ​​corresponding to each lighting control parameter. The sample color values ​​represent the color of the specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameters.

[0045] It should be understood that the embodiments of the present disclosure do not limit the process for constructing the target dataset. As long as the dataset contains multiple lighting control parameters of the physical lighting system and the sample color values ​​corresponding to each lighting control parameter, it will suffice. Optionally, the embodiments of the present disclosure further provide a process for constructing the target dataset. The process for constructing the target dataset may include:

[0046] While keeping the physical camera and the physical lighting system perpendicular to the specified actual material, adjust the lighting control parameters of the object lighting system according to the specified step size within the value range of the lighting control parameters of the physical lighting system, and at the same time control the physical camera to perform image acquisition to obtain captured images under different lighting control parameters;

[0047] By extracting the color value of the actual material specified in the captured image under different lighting control parameters, the sample color value corresponding to the different lighting control parameters is obtained;

[0048] The target data set is constructed to include various lighting control parameters of the physical lighting system and the sample color values ​​corresponding to each lighting control parameter.

[0049] Among them, the actual material specified in the target dataset construction process can be the material of the same object as the virtual material specified in the virtual shooting process. For example, it can be the material of the ground in the actual shooting scene, which is equivalent to using the color of the ground material as a reference to obtain the color actually presented by the physical lighting system under different lighting control parameters. Alternatively, a foam board (a white foam board used for light control, which can soften hard light and convert direct light into soft light through diffuse reflection, thereby improving the shooting effect) commonly used in virtual shooting can be used. Using a foam board is equivalent to using the white color of the white foam board material as a reference to obtain the color actually presented by the physical lighting system under different lighting control parameters. In this case, the virtual material specified in the virtual shooting process can be a white foam board material. It should be noted that the embodiment of the present disclosure can ignore the color difference caused by the physical camera shooting, that is, it is considered that the color of the actual material captured by the physical camera (that is, the color value of the actual material extracted from the captured image) is the color presented by the actual material under the lighting of the physical lighting system.

[0050] The construction process of the above-mentioned target dataset is introduced using the Mipineapple whiteboard as an example. After preparing the Mipineapple whiteboard, the physical lighting system (assuming that the physical lighting system can control its brightness parameters and RGB color parameters through the DMX protocol), and the physical camera, the physical lighting system and the physical camera can be perpendicular to the Mipineapple whiteboard (that is, the illumination direction of the physical lighting system and the shooting direction of the physical camera are perpendicular to the Mipineapple whiteboard). At the same time, the white color of the Mipineapple whiteboard can be set as a unified space to calibrate the camera parameters of the physical camera to ensure the exposure parameters and white balance settings of the physical camera, so that the physical camera can accurately capture the colors presented on the Mipineapple whiteboard in the subsequent process to avoid color distortion. The embodiment of the present disclosure does not limit the calibration process of the camera parameters. Among them, in the DMX protocol, 8 bits are generally used to transmit data. At this time, the value range of the lighting control parameters can be between [0, 255]. According to the above assumptions, the physical lighting system has four controllable parameters: brightness, R, G, and B. Then, within the interval [0,255], the lighting control parameters of the physical lighting system can be adjusted according to the specified step size N (assuming the specified step size is 32). For example, the first time the lighting control parameter [0,0,0,0] is sent to the physical lighting system through the DMX protocol, the physical lighting system illuminates the Mipineapple whiteboard with a brightness parameter of 0, an R parameter of 0, a G parameter of 0, and a B parameter of 0. At the same time, the physical camera performs image acquisition and obtains the captured image under the lighting control parameter [0,0,0,0]. At this time, the color value (such as R1, G1, B1) of the area where the Mipineapple whiteboard is located is extracted from the captured image, which is the sample color value corresponding to the lighting control parameter [0,0,0,0]; the second time the physical lighting system is sent to the physical lighting system, the color value (such as R1, G1, B1) of the area where the Mipineapple whiteboard is located is extracted from the captured image, which is the sample color value corresponding to the lighting control parameter [0,0,0,0]. Send the lighting control parameters of [32,0,0,0]. At this time, the physical lighting system illuminates the Mipineapple whiteboard with a brightness parameter of 32, an R parameter of 0, a G parameter of 0, and a B parameter of 0. At the same time, the physical camera performs image acquisition to obtain an acquired image under the lighting control parameters [32,0,0,0]. At this time, the color values ​​(such as R2, G2, B2) of the area where the Mipineapple whiteboard is located are extracted from the acquired image, which are the sample color values ​​corresponding to the lighting control parameters [32,0,0,0]. Similarly, a total of (256 / 32)^4=4096 groups of lighting control parameters are sent, and 4096 groups of sample color values ​​are obtained. Then, the sample color values ​​(sample RGB values) presented by the Mipineapple whiteboard and the lighting control parameters sent through DMX attraction can be constructed into a target data set in the "input:output" format.

[0051] The above-mentioned prediction model can be trained based on the target data set constructed as above, wherein the training process of the prediction model may include: inputting the sample color values ​​in the target data set into the initial model to obtain the predicted light control parameters output by the initial model; using the loss between the predicted light control parameters and the light control parameters corresponding to the input sample color values, adjusting the model parameters of the initial model to obtain a trained prediction model. In practical applications, the loss between the predicted light control parameters and the light control parameters corresponding to the input sample color values ​​can be calculated using error functions known in the art, such as mean square error, CIEDE2000 (a color difference calculation formula used to measure the color difference between two groups of colors), etc., which is not limited in the embodiments of the present disclosure. Based on the loss, the model parameters of the initial model can be adjusted by backpropagation and gradient descent, etc., until the model parameters converge to obtain a trained prediction model, which is not limited in the embodiments of the present disclosure.

[0052] In actual applications, target data sets corresponding to different actual materials can be constructed for different actual materials, and prediction models corresponding to different actual materials can be trained using the target data sets corresponding to different actual materials. In this way, the prediction model corresponding to the actual material of the same material as any specified virtual material can be used to predict the lighting control parameters used to illuminate the actual material to produce the target color value. This is not limited to the embodiments of the present disclosure.

[0053] In practical applications, the embodiments of the present disclosure do not impose any restrictions on the model type and model structure of the prediction model. The prediction model can adopt a deep learning model or a machine learning model, and the embodiments of the present disclosure do not impose any restrictions on this. For example, the prediction model can adopt a multilayer perceptron model (Multilayer Perceptron), which is a feedforward artificial neural network model consisting of an input layer, one or more hidden layers, and an output layer. The multilayer perceptron model structure adopted by the prediction model of the embodiments of the present disclosure can, for example, be composed of an input layer, two hidden layers, and an output layer, and the embodiments of the present disclosure do not impose any restrictions on this.

[0054] It should be understood that using a prediction model to output lighting control parameters based on a target color value is equivalent to calibrating the physical lighting system, and by training the prediction model, it is possible to achieve lighting calibration for any physical lighting system, so that it can automatically output lighting control parameters to automatically adjust the brightness and color parameters of the physical lighting system given the input of any target color value within the color rendering capability of the physical lighting system itself, so that the actual material presents the color of the specified target color value under the lighting of the physical lighting system, so that there is no error between the original input target color value and the color presented by the physical camera, and therefore it is more universal. During the use of the trained prediction model, any target color value can be input into the prediction model, and the prediction model can output a set of 4-dimensional lighting control parameters (i.e., brightness, R, G, B parameters), which are sent to the physical lighting system via the DMX protocol. At this time, the physical lighting system illuminates the specified physical material under the control of the lighting control parameters and the color presented by the physical camera is the target color value.

[0055] Alternatively, in one possible implementation, a color space capable of being rendered by the physical lighting system can be generated based on the sample color values ​​corresponding to each of the multiple lighting control parameters of the physical lighting system in the target dataset. This color space can be understood as the range of the physical lighting system's color rendering capabilities. Every color that the physical lighting system can produce when illuminating an actual material has a corresponding position in this color space, and each color has a one-to-one correspondence with a lighting control parameter. Similar to the prediction model, corresponding color spaces can be constructed for different actual materials, and the appropriate color space can be selected based on the actual material being aligned. In actual situations, the target color value of the virtual material may exceed the color rendering capability range of the physical lighting system. In this case, a mapping color value close to the target color value can be determined from the color space of the physical lighting system, and then the mapping color value can be used to determine the lighting control parameters of the physical lighting system to adapt to the color rendering capability range of the physical lighting system. Therefore, the above-mentioned determination of the lighting control parameters for lighting the actual material in the real scene to the target color value based on the target color value of the virtual material specified in the virtual scene can include: mapping the target color value to the color space to obtain the mapping color value; and determining the lighting control parameters based on the mapping color value.

[0056] It should be understood that the sample color value corresponding to each of the multiple lighting control parameters generated according to the specified step size within the value range of the lighting control parameters of the known physical lighting system can be obtained by, for example, interpolating, visualizing, and other processing all the sample color values ​​in the target data set to obtain the color space that the physical lighting system can present, which is equivalent to obtaining the color range that the physical lighting system can present, and the embodiments of the present disclosure do not limit this.

[0057] Furthermore, based on the color space of the physical lighting system, the color difference between the target color value and each color value in the color space can be calculated to obtain the color value with the smallest color difference with the target color value as the mapping color value corresponding to the target color value. It should be understood that the mapping color value and the target color value can be the same or similar, so that the target color value can be mapped to the color space of the physical lighting system, and the physical lighting system can be controlled within the color rendering capability of the physical lighting system.

[0058] The trained prediction model can be used to determine lighting control parameters based on the mapped color values. Specifically, the mapped color values ​​can be input into the prediction model to obtain lighting control parameters that illuminate the actual materials in the real-world shooting environment to the same or similar target color values. This approach is equivalent to taking any color value from the color space of the physical lighting system and inputting it into the prediction model, which then outputs a set of lighting control parameters to control the physical lighting system.

[0059] It should be understood that the smaller the specified step size of adjusting the lighting control parameters when constructing the target data set, the greater the number of samples in the target data set. Therefore, a smaller specified step size (such as 1, 2, 5, etc.) can be set to make the collected target data set include a large number of lighting control parameters and corresponding sample color values. Of course, considering the long data collection process, a larger specified step size can also be used to obtain multiple lighting control parameters and corresponding sample color values, and then the number of lighting control parameters and corresponding sample color values ​​in the target data set can be increased by interpolation. Furthermore, based on the large number of lighting control parameters and corresponding sample color values ​​in the target data set, a mapping relationship between color values ​​and lighting control parameters can be constructed to directly determine the lighting control parameters corresponding to the target color values ​​of the virtual material specified in the virtual scene based on the mapping relationship. Thus, optionally, determining the lighting control parameters for lighting the actual material in the real scene to achieve the target color value based on the target color value of the virtual material specified in the virtual scene may include: determining the lighting control parameters corresponding to the target color value of the virtual material specified in the virtual scene based on a pre-established mapping relationship between color values ​​and lighting control parameters; wherein the mapping relationship is established based on multiple lighting control parameters of the physical lighting system in the target dataset and the sample color values ​​corresponding to each lighting control parameter. Similar to the prediction model, corresponding mapping relationships can be established for different actual materials, and the corresponding mapping relationship can be selected for use based on the actual material to be aligned.

[0060] In practical applications, the mapping relationship between color values ​​and lighting control parameters can be expressed as a functional relationship. For example, a target data set can be used to fit the functional relationship between color values ​​and lighting control parameters. This functional relationship can be a function with color values ​​as the independent variable and lighting control parameters as the dependent variable. Alternatively, the mapping relationship between color values ​​and lighting control parameters can be expressed in the form of a MAP chart, which can list the corresponding relationship between sample color values ​​and lighting control parameters. In this way, the lighting control parameters corresponding to any target color value can be determined by looking up the table. This is not limited to this embodiment of the present disclosure. In this way, the lighting control parameters that will illuminate the actual material in the real shooting environment to the target color value can be determined more quickly.

[0061] As described above, considering the color rendering capability range of the physical lighting system, the target color value can also be mapped to the color space to obtain the mapped color value, and the lighting control parameters can be determined based on the mapped color value. Therefore, the above-mentioned determination of the lighting control parameters based on the mapped color value can also include: based on the pre-constructed mapping relationship between the color value and the lighting control parameter, determining the lighting control parameters corresponding to the mapped color value as the lighting control parameters for lighting the actual material in the real scene to the target color value.

[0062] In step S13, the lighting control parameters are sent to the physical lighting system so that the actual material in the real shooting environment presents a target color value consistent with the virtual material in the virtual scene displayed on the physical screen under the lighting of the physical lighting system.

[0063] In actual applications, the predicted lighting control parameters can be sent to the physical lighting system via the DMX protocol, so that the actual materials in the real shooting environment of the physical lighting system can be used to produce target color values ​​that are consistent with the corresponding virtual materials in the virtual scene on the physical screen, thereby achieving a virtual-real alignment effect. Moreover, when the physical lighting system is illuminated according to the lighting control parameters, since the actual materials in the real shooting environment, under the lighting of the physical lighting system, present target color values ​​that are consistent with the corresponding virtual materials in the virtual scene on the physical screen, the color of the actual materials in the picture captured by the physical camera used in the virtual shooting process can be made consistent with the color of the virtual materials in the picture, thereby improving the quality of the captured picture and achieving a more accurate virtual-real alignment effect.

[0064] In actual applications, experiments have verified that the total implementation time of the entire method of the disclosed embodiment is less than 0.1 milliseconds. Since typical shooting frame rates are 25fps or 50fps, this method can ensure that the on-site physical lighting system synchronizes with the dynamic effects of the virtual lighting when the virtual lighting in the virtual scene on the physical screen changes dynamically.

[0065] According to the method of the embodiment of the present disclosure, by determining the lighting control parameters of the physical lighting system that illuminates the actual material in the real shooting environment to the target color value based on the target color value presented by the virtual material in the virtual scene under the virtual light, and sending the lighting control parameters to the physical lighting system, not only can the color presented by the actual material under the physical light be automatically aligned with the color presented by the virtual material in the virtual scene under the virtual light, but also when the lighting effect of the virtual scene displayed on the physical screen changes dynamically, it can automatically ensure that the physical lighting system is synchronously aligned with the dynamic lighting effect of the virtual light, thereby realizing dynamic alignment of virtual and real lighting effects, improving lighting efficiency and obtaining a more accurate virtual-reality alignment effect, and also helping to ensure that the color presented by the actual material in the shooting picture of the physical camera during the virtual shooting process under the lighting of the physical lighting system according to the lighting control parameters is consistent with the color presented by the virtual material under the lighting of the virtual light, thereby improving the picture quality of the virtual shooting.

[0066] Although there are technologies in the existing technology that can automatically achieve virtual and real light alignment, for example, some technologies are based on the lighting information of real lamps to match real lamps with virtual lamps one-to-one. This method is more difficult and often cannot be implemented due to space limitations. In addition, this technology directly transmits the lighting parameters of physical lamps to virtual lamps in the virtual scene to achieve virtual and real alignment. It cannot achieve dynamic lighting alignment and cannot ensure that the colors presented by virtual and real lights on virtual and real materials are consistent. The above-mentioned method proposed in the embodiment of the present disclosure has no spatial limitations and does not require one-to-one alignment of real and virtual lamps. It can achieve virtual-real alignment of dynamic lighting effects and ensure that the colors presented by virtual and real lights on virtual and real materials are consistent. Alternatively, there are technologies that directly transmit the lighting parameter values ​​of virtual lamps in the virtual scene to physical lamps, that is, directly adjust the lighting control parameters of physical lights according to the lighting parameters of virtual lights in the virtual scene. This cannot ensure that the colors of virtual and real objects in the captured image are aligned after the virtual and real lights reach the virtual and real materials, and thus the alignment effect cannot be guaranteed. That is, this can only ensure a certain degree of effect, but cannot ensure that the colors presented by virtual and real lights on virtual and real materials are consistent. The above-mentioned method of the embodiment of the present disclosure uses a prediction model, etc., based on the target color value presented by the virtual material under virtual lighting, to predict the physical lighting parameters, so that the color of the actual material in the camera image after being illuminated by the physical light is consistent with the color of the virtual material after being illuminated by the virtual light, and can ensure a strict virtual-real alignment effect.

[0067] Figure 3 A block diagram of a device for aligning virtual and real lighting effects in virtual photography according to an embodiment of the present disclosure is shown. Figure 3 As shown, the device includes:

[0068] An acquisition module 301 is configured to acquire, during a virtual shooting process, a target color value of a virtual material specified in a virtual scene displayed on a physical screen, wherein the target color value includes a color value of the virtual material specified in the virtual scene under the current lighting of a virtual light in the virtual scene;

[0069] A parameter determination module 302 is configured to determine, based on a target color value of a virtual material specified in the virtual scene, lighting control parameters for illuminating an actual material in a real shooting environment to achieve the target color value; wherein the actual material and the virtual material are the same material;

[0070] The sending module 303 is used to send the lighting control parameters to the physical lighting system so that the actual material in the real shooting environment presents the target color value consistent with the virtual material in the virtual scene displayed on the physical screen under the lighting of the physical lighting system.

[0071] In one possible implementation, the device further includes: a data set construction module, configured to: adjust the lighting control parameters of the object lighting system according to a specified step size within a value range of the lighting control parameters of the physical lighting system while maintaining the physical camera and the physical lighting system facing the specified actual material perpendicularly, and simultaneously control the physical camera to perform image acquisition to obtain acquired images under different lighting control parameters; obtain sample color values ​​corresponding to the different lighting control parameters by extracting the color value of the specified actual material from the acquired images under different lighting control parameters; and construct a target data set including multiple lighting control parameters of the physical lighting system and the sample color values ​​corresponding to each lighting control parameter.

[0072] In one possible implementation, the method of determining the lighting control parameters for lighting the actual material in the real scene to the target color value of the virtual material specified in the virtual scene includes: determining the lighting control parameters for lighting the actual material in the real scene to the target color value based on the target color value of the virtual material specified in the virtual scene using a prediction model; wherein the prediction model is trained using a target data set, and the target data set includes multiple lighting control parameters of the physical lighting system and sample color values ​​corresponding to each lighting control parameter, and the sample color values ​​represent the color of the specified actual material captured by a physical camera under the lighting of the physical lighting system according to the corresponding lighting control parameters; wherein the training process of the prediction model includes: inputting the sample color values ​​in the target data set into the initial model to obtain the predicted lighting control parameters output by the initial model; and adjusting the model parameters of the initial model using the loss between the predicted lighting control parameters and the lighting control parameters corresponding to the input sample color values ​​to obtain the trained prediction model.

[0073] In a possible implementation, the device further includes: a color space generation module, configured to generate a color space that can be presented by the physical lighting system based on a sample color value corresponding to each of a plurality of lighting control parameters of the physical lighting system in a target data set, wherein the sample color value represents the color of a specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameter; wherein, determining the lighting control parameter for lighting the actual material in the real scene to the target color value based on the target color value of the virtual material specified in the virtual scene includes: mapping the target color value to the color space to obtain a mapped color value; and determining the lighting control parameter based on the mapped color value.

[0074] In one possible implementation, determining the lighting control parameters for lighting an actual material in a real scene to achieve the target color value of the virtual material specified in the virtual scene according to the target color value of the virtual material specified in the virtual scene includes: determining the lighting control parameters corresponding to the target color value of the virtual material specified in the virtual scene based on a pre-constructed mapping relationship between color values ​​and lighting control parameters; wherein the mapping relationship is constructed based on multiple lighting control parameters of the physical lighting system in a target data set and sample color values ​​corresponding to each lighting control parameter, wherein the sample color value represents the color of the specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameters.

[0075] In one possible implementation, the lighting control parameters include brightness parameters and color parameters; the color values ​​include RGB values; wherein, when the physical lighting system performs lighting according to the lighting control parameters, the color of the actual material in the picture captured by the physical camera used in the virtual shooting process is consistent with the color of the virtual material in the picture.

[0076] According to the device of the embodiment of the present disclosure, by determining the lighting control parameters of the physical lighting system that illuminates the actual material in the real shooting environment to the target color value based on the target color value presented by the virtual material in the virtual scene under the virtual light, and sending the lighting control parameters to the physical lighting system, it can not only automatically achieve the alignment of the color presented by the actual material under the physical light with the color presented by the virtual material in the virtual scene under the virtual light, but also when the lighting effect of the virtual scene displayed on the physical screen changes dynamically, it can automatically ensure that the physical lighting system is synchronously aligned with the dynamic lighting effect of the virtual light, thereby achieving dynamic alignment of virtual and real lighting effects, improving lighting efficiency and obtaining a more accurate virtual-reality alignment effect, and also helping to ensure that the color presented by the actual material in the shooting picture of the physical camera during the virtual shooting process under the lighting of the physical lighting system according to the lighting control parameters is consistent with the color presented by the virtual material under the lighting of the virtual light, thereby improving the picture quality of the virtual shooting.

[0077] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0078] An embodiment of the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0079] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0080] An embodiment of the present disclosure further provides a computer program product, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0081] Figure 4 FIG1 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0082] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0083] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0084] A computer-readable storage medium can be a tangible device that can hold and store programs / instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0085] The computer programs (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0086] The computer program (or computer program instructions) for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, by utilizing state information of computer-readable program instructions to personalize and customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0087] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0088] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0089] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0090] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0091] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for aligning virtual and real lighting effects in virtual photography, characterized in that: The method comprises: During the virtual shooting process, a target color value of a virtual material specified in a virtual scene displayed on a physical screen is obtained, wherein the target color value includes a color value presented by the virtual material specified in the virtual scene under the current lighting of a virtual light in the virtual scene; Determining, based on a target color value of a virtual material specified in the virtual scene, lighting control parameters for illuminating an actual material in a real shooting environment to the target color value; wherein the actual material and the virtual material are materials of the same object; The lighting control parameters are sent to the physical lighting system so that the actual material in the real shooting environment presents the target color value consistent with the virtual material in the virtual scene displayed on the physical screen under the lighting of the physical lighting system.

2. The method according to claim 1, characterized in that The method further comprises: While maintaining the physical camera and the physical lighting system perpendicular to a specified actual material, adjusting the lighting control parameters of the object lighting system according to a specified step size within a value range of the lighting control parameters of the physical lighting system, while controlling the physical camera to perform image acquisition, thereby obtaining acquired images under different lighting control parameters; By extracting the color value of the actual material specified in the captured image under different lighting control parameters, the sample color values ​​corresponding to the different lighting control parameters are obtained; The target data set is constructed to include multiple lighting control parameters of the physical lighting system and sample color values ​​corresponding to each lighting control parameter.

3. The method according to claim 1 or 2, characterized in that The step of determining, based on the target color value of the virtual material specified in the virtual scene, a lighting control parameter for lighting the actual material in the real scene to the target color value includes: Determining lighting control parameters for lighting an actual material in a real scene to achieve the target color value based on the target color value of the virtual material specified in the virtual scene using the prediction model; The prediction model is trained using a target dataset, wherein the target dataset includes multiple lighting control parameters of the physical lighting system and sample color values ​​corresponding to each lighting control parameter, where the sample color values ​​represent the color of a specified actual material captured by a physical camera under the lighting of the physical lighting system according to the corresponding lighting control parameters; The training process of the prediction model includes: Inputting the sample color values ​​in the target data set into the initial model to obtain the predicted lighting control parameters output by the initial model; The model parameters of the initial model are adjusted by using the loss between the predicted light control parameters and the light control parameters corresponding to the input sample color values ​​to obtain the trained prediction model.

4. The method according to claim 1 or 2, characterized in that The method further comprises: generating a color space that can be rendered by the physical lighting system based on sample color values ​​corresponding to each of the multiple lighting control parameters of the physical lighting system in the target dataset, wherein the sample color values ​​represent the color of a specified actual material captured by a physical camera under lighting by the physical lighting system according to the corresponding lighting control parameters; The step of determining the lighting control parameters for lighting the actual material in the real scene to achieve the target color value according to the target color value of the virtual material specified in the virtual scene includes: Mapping the target color value to the color space to obtain a mapped color value; The lighting control parameter is determined according to the mapped color value.

5. The method according to claim 1 or 2, characterized in that The step of determining, based on the target color value of the virtual material specified in the virtual scene, a lighting control parameter for lighting the actual material in the real scene to the target color value includes: determining, based on a pre-established mapping relationship between color values ​​and lighting control parameters, lighting control parameters corresponding to a target color value of a virtual material specified in the virtual scene; The mapping relationship is constructed based on multiple lighting control parameters of the physical lighting system in the target data set and the sample color values ​​corresponding to each lighting control parameter, wherein the sample color value represents the color of the specified actual material captured by the physical camera under the lighting of the physical lighting system according to the corresponding lighting control parameters.

6. The method according to any one of claims 1 to 5, characterized in that The lighting control parameters include brightness parameters and color parameters; the color values ​​include RGB values; Wherein, when the physical lighting system performs lighting according to the lighting control parameters, the color of the actual material in the picture captured by the physical camera used in the virtual shooting process is consistent with the color of the virtual material in the picture.

7. A device for aligning virtual and real lighting effects in virtual photography, characterized in that: The device comprises: An acquisition module is used to acquire, during a virtual shooting process, a target color value of a virtual material specified in a virtual scene displayed on a physical screen, wherein the target color value includes a color value presented by the virtual material specified in the virtual scene under the current lighting of a virtual light in the virtual scene; a parameter determination module for determining, based on a target color value of a virtual material specified in the virtual scene, lighting control parameters for illuminating an actual material in a real shooting environment to achieve the target color value; wherein the actual material and the virtual material are the same material; A sending module is used to send the lighting control parameters to the physical lighting system so that the actual material in the real shooting environment presents the target color value consistent with the virtual material in the virtual scene displayed on the physical screen under the lighting of the physical lighting system.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images