Illumination information processing method, device, electronic equipment and storage medium

Abstract

The application provides a light information processing method and device, electronic equipment, computer program product and computer readable storage medium; the method comprises: acquiring real light information corresponding to a real scene; acquiring virtual light information corresponding to a virtual scene; comparing and calculating the virtual light information and the real light information to obtain light difference information; performing light attenuation calculation based on the light difference information to obtain configuration parameters of at least one supplementary light source, wherein the supplementary light source is used for supplementary lighting; matching each supplementary light source configuration parameter with a lamp database to obtain a target lamp corresponding to each supplementary light source in the lamp database; taking each target lamp and the configuration parameter corresponding to each target lamp as configuration information, and performing scene light synchronization based on the configuration information. Through the application, the accuracy of synchronizing the light between the virtual scene and the real scene can be improved.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a method, apparatus, electronic device, and storage medium for processing light information. Background Technology

[0002] Display technologies based on graphics processing hardware have expanded the channels for perceiving the environment and acquiring information. In particular, virtual scene display technologies can realize diverse interactions between virtual objects controlled by users or artificial intelligence according to actual application needs, and have various typical application scenarios, such as creating videos and movies based on virtual scenes.

[0003] Virtual production combines virtual reality, augmented reality, computer-generated imagery (CGI), and game engine technologies. In some cases, during virtual production, it is necessary to shoot real scenes to obtain footage. Virtual scenes and real scenes need to have the same lighting atmosphere. In related technologies, the lighting synchronization between virtual and real scenes usually relies on manual adjustments by technicians, which is inefficient and the accuracy of lighting synchronization is affected by subjective human factors.

[0004] Currently, there is no good way to synchronize the lighting between virtual and real scenes in the relevant technologies. Summary of the Invention

[0005] This application provides a lighting information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the accuracy of lighting between synchronized virtual scenes and real scenes.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a method for processing illumination information, the method comprising:

[0008] Obtain the real-world lighting information corresponding to the real scene;

[0009] Obtain virtual lighting information corresponding to the virtual scene;

[0010] The virtual lighting information is compared and calculated with the real lighting information to obtain lighting difference information;

[0011] Light attenuation calculation is performed based on illumination difference information to obtain configuration parameters for at least one supplementary light source, wherein the supplementary light source is used for supplementary illumination;

[0012] Based on the configuration parameters of each supplementary light source, a matching process is performed with the lighting database to obtain the target lighting fixture in the lighting database corresponding to each supplementary light source;

[0013] Each target luminaire and its corresponding configuration parameters are used as configuration information, and scene lighting synchronization is performed based on the configuration information.

[0014] This application provides a light information processing device, including:

[0015] The lighting acquisition module is configured to acquire real lighting information corresponding to the real scene.

[0016] The lighting acquisition module is also configured to acquire virtual lighting information corresponding to the virtual scene;

[0017] The difference acquisition module is configured to compare and calculate the virtual lighting information with the real lighting information to obtain lighting difference information.

[0018] The light source configuration module is configured to perform light attenuation calculation based on illumination difference information to obtain configuration parameters for at least one supplementary light source, wherein the supplementary light source is used for supplementary illumination;

[0019] The light source configuration module is further configured to perform matching processing with the lamp database based on the configuration parameters of each supplementary light source to obtain the target lamp in the lamp database corresponding to each supplementary light source;

[0020] The illumination synchronization module is configured to use each target lamp and its corresponding configuration parameters as configuration information, and to perform scene illumination synchronization based on the configuration information.

[0021] This application provides an electronic device, the electronic device comprising:

[0022] Memory is used to store executable instructions for a computer;

[0023] The processor, when executing computer-executable instructions stored in the memory, implements the illumination information processing method provided in the embodiments of this application.

[0024] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the illumination information processing method provided in this application.

[0025] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implement the illumination information processing method provided in this application.

[0026] The embodiments of this application have the following beneficial effects:

[0027] By acquiring lighting information from virtual and real scenes and comparing and calculating this information, the accuracy of acquiring lighting differences between virtual and real scenes is improved. Based on the lighting difference information, the configuration parameters corresponding to the supplementary lighting source are obtained, and the target lighting fixtures for supplementary lighting are retrieved from the lighting database. Based on the configuration information, lighting synchronization is performed on the real or virtual scene to be supplemented, improving the accuracy and efficiency of lighting synchronization and saving the cost required for lighting synchronization. Attached Figure Description

[0028] Figure 1A This is a schematic diagram illustrating the application mode of the illumination information processing method provided in the embodiments of this application;

[0029] Figure 1B This is a schematic diagram illustrating the application mode of the illumination information processing method provided in the embodiments of this application;

[0030] Figure 2A This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application;

[0031] Figure 2B This is a schematic diagram of the structure of the terminal device 400 provided in the embodiments of this application;

[0032] Figures 3A to 3G This is a schematic flowchart of the illumination information processing method provided in the embodiments of this application;

[0033] Figure 4 This is a schematic diagram of the interactive flow of the illumination information processing method provided in the embodiments of this application;

[0034] Figure 5A This is a schematic diagram of the reference object provided in the embodiments of this application;

[0035] Figure 5B This is a schematic diagram of the handheld spectrometer provided in the embodiments of this application;

[0036] Figures 5C to 5D This is a schematic diagram of the human-computer interaction interface of the handheld spectrometer provided in the embodiments of this application;

[0037] Figure 5E This is a brightness simulation diagram of illumination information provided in the embodiments of this application;

[0038] Figures 6A to 6B This is a schematic flowchart of the illumination information processing method provided in the embodiments of this application;

[0039] Figure 7 This is a schematic diagram of a real-world scenario provided in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0042] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0043] It should be noted that in the embodiments of this application, user information, user feedback data and other related data are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Virtual scenes, which utilize devices to output scenes distinct from the real world, allow for visual perception through the naked eye or with device assistance. Examples include two-dimensional images output through a display screen, and three-dimensional images output through stereoscopic display technologies such as stereoscopic projection, virtual reality, and augmented reality. Furthermore, various possible hardware can be used to create auditory, tactile, olfactory, and motion perceptions, simulating the real world. In this embodiment, the virtual scene can be a digital scene created in a game engine based on the needs of technicians or a real-world scene, or a scene created in a virtual filming studio.

[0047] 2) Virtual Production: Virtual production is a broad term referring to various computer-aided methods of creating visual films. Weta Digital defines virtual production as "the area where the real and digital worlds merge." MPC further refines this definition with technical details: "Virtual production combines virtual reality, augmented reality, computer-generated imagery (CGI), and game engine technologies, allowing producers to see scenes unfold before them as if these scenes were actually composited and filmed on location."

[0048] 3) Light-Emitting Diode (LED) screen wall, hereinafter referred to as LED screen wall, is a large LED screen set up in a virtual film production studio to display virtual content.

[0049] 4) Virtual studio, also known as virtual production studio, is a real studio used to shoot virtual production films. Common virtual studios are studios that combine green screen studios with on-site lighting, wire work, and special props with post-production special effects. Currently, newer studios combine LED screens, motion capture technology, and camera tracking.

[0050] 5) On-site shooting camera: The on-site shooting camera in the virtual production studio is used to capture the fused image of the LED screen and the scene in front of the screen. In this embodiment, the on-site shooting camera is referred to as a real camera.

[0051] 6) Physical lighting refers to the light emitted by physical lighting fixtures (real lighting fixtures) used in a virtual shooting studio to illuminate the characters and set design.

[0052] 7) Unreal Engine (UE): Developed by Epic Games, it is one of the world's most widely licensed game engines, and is widely used in content creation outside of the gaming industry.

[0053] 8) Gray sphere: A sphere used as a reference point to indicate the direction of light, the intensity of shadows (e.g., rough or soft shadows), and to ensure that the light level is uniform, as well as the color temperature of the light source.

[0054] 9) A chrome ball, a sphere used as a reference object, has a smooth surface and can be used for alignment of reflections, lighting, and high dynamic range images.

[0055] 10) A standard color chart, also known as a colorimeter, is developed based on the three primary colors of colorimetry. It is made of colored glazed ceramic blocks sintered using a special process and includes at least five colors: red, yellow, green, blue, and white. The standard color chart has a smooth surface, uniform color, high color saturation, stable optical and physicochemical properties, and covers the entire visible spectrum wavelength, making it a standard measuring instrument for color measurement.

[0056] 11) High Dynamic Range (HDR) images offer greater dynamic range and detail compared to standard images. HDR images are created by acquiring Low Dynamic Range (LDR) images with varying exposure times and then synthesizing the LDR image with the best detail corresponding to each exposure time. HDR images better reflect the visual effects of real-world environments.

[0057] 12) High-Dynamic Range (HDRi) image format: HDRi is a digital image storage format used to store images with a very wide brightness range. HDRi images store significantly more brightness data compared to other image formats. Furthermore, its method of recording brightness differs from traditional images; instead of compressing brightness information into an 8-bit or 16-bit color space in a non-linear manner, it records brightness information directly.

[0058] 13) A digital multiplex (DMX) signal control console is a control console that uses digital signals to control the lighting of lamps. It is commonly used in stage lighting, studios, etc.

[0059] This application provides a lighting information processing method, a lighting information processing device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of lighting between synchronized virtual scenes and real scenes.

[0060] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of user terminals, such as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), in-vehicle terminals, virtual reality (VR) devices, and augmented reality (AR) devices, or as servers. Exemplary applications when the device is implemented as a terminal device or a server will be described below.

[0061] Virtual scenes can be game virtual scenes, in the context of... Figure 1A Before proceeding, let's first introduce the game modes involved in the terminal device and server collaborative implementation scheme. This scheme primarily involves two game modes: local game mode and cloud game mode. In local game mode, the terminal device and server collaboratively run the game processing logic. The player's input commands on the terminal device are partly processed by the terminal device's game logic, and partly by the server. Furthermore, the server-side game logic processing is often more complex and requires more computing power. In cloud game mode, the server handles all game logic processing, and the cloud server renders the game scene data into audio and video streams, which are then transmitted to the terminal device for display over the network. The terminal device only needs basic streaming media playback capabilities and the ability to receive player commands and send them to the server.

[0062] refer to Figure 1A , Figure 1A This is a schematic diagram illustrating the application mode of the illumination information processing method provided in the embodiments of this application; for example, Figure 1A The system involves server 200, network 300, terminal device 400, and lighting device 500. Terminal device 400 is connected to server 200 through network 300; terminal device 400 is connected to lighting device 500 through network 300, or terminal device 400 and lighting device 500 are directly connected. Network 300 can be a wide area network or a local area network, or a combination of both.

[0063] For example, the real scene is a virtual film studio, the virtual scene is the virtual scene of a virtual film, the lighting equipment 500 is the lighting equipment in the virtual film studio, the server 200 is the server running the game virtual engine, which is suitable for application modes that rely on the computing power of the server 200 to complete the virtual scene calculation and output the virtual scene to the terminal device 400, and the terminal device 400 integrates a digital dimming signal control console for controlling the lighting equipment 500. The following will be explained in conjunction with the example above.

[0064] In some embodiments, server 200 receives real lighting information corresponding to a real scene from terminal device 400 and virtual lighting information corresponding to a virtual scene; it compares the two to determine the target lighting fixture to be supplemented in the real scene or virtual scene, and the configuration parameters of the target lighting fixture. When the virtual scene needs supplemented lighting, server 200 synchronizes the lighting of the virtual scene and sends the synchronized virtual scene image to terminal device 400, which displays the corresponding image. When the real scene needs supplemented lighting, server 200 sends the target lighting fixture to be supplemented in the real scene and its configuration parameters to terminal device 400. Terminal device 400 generates a digital dimming signal based on the configuration parameters and sends it to the lighting device 500 corresponding to the target lighting fixture, thereby synchronizing the lighting in the real scene with the virtual scene.

[0065] In the introduction Figure 1B Previously, for Figure 1B The application scenarios will be explained. Figure 1B This is suitable for application modes where the computational power of the graphics processing hardware of the terminal device 400 is sufficient to complete the relevant data calculations for the virtual scene. Examples include standalone / offline games, where the output of the virtual scene is completed through various types of terminal devices 400, such as smartphones, tablets, and virtual reality / augmented reality devices. As an example, the types of graphics processing hardware include central processing units (CPUs) and graphics processing units (GPUs).

[0066] In some embodiments, reference Figure 1B , Figure 1B This is a schematic diagram illustrating the application mode of the illumination information processing method provided in the embodiments of this application; for example, Figure 1B The system involves a network 300, a terminal device 400, and a lighting device 500. The terminal device 400 is connected to the lighting device 500 through the network 300, or the terminal device 400 is directly connected to the lighting device 500. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0067] For example, the real scene is the virtual film studio, the virtual scene is the virtual scene of the virtual film, the lighting equipment 500 is the lighting equipment in the virtual film studio, the terminal equipment 400 runs the virtual engine of the virtual scene, and the terminal equipment 400 integrates a digital dimming signal control console for controlling the lighting equipment 500. The following is an explanation based on the example above.

[0068] In some embodiments, the terminal device 400 acquires real lighting information corresponding to a real scene and virtual lighting information corresponding to a virtual scene; it compares the two to determine the target lighting fixture to be supplemented in the real scene or the virtual scene, and the configuration parameters of the target lighting fixture. When the virtual scene needs supplemented lighting, the terminal device 400 synchronizes the lighting of the virtual scene and displays the synchronized virtual scene. When the real scene needs supplemented lighting, the terminal device 400 generates a digital dimming signal based on the target lighting fixture and its configuration parameters and sends it to the lighting device 500 corresponding to the target lighting fixture, so that the lighting in the real scene is synchronized with the virtual scene.

[0069] This application embodiment can be implemented using blockchain technology. Data related to the lighting fixtures used in this application embodiment can be uploaded to the blockchain for storage, and the reliability of lighting synchronization can be ensured through a consensus algorithm. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0070] This application embodiment can be implemented using database technology. A database, simply put, can be viewed as an electronic filing cabinet storing electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, having minimal redundancy, and being independent of application programs.

[0071] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.

[0072] This application embodiment can also be implemented using cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. It can form a resource pool, available on demand, offering flexibility and convenience. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, and driven by demands for search services, social networks, mobile commerce, and open collaboration, every item may eventually possess its own hash-coded identification mark, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0073] In some embodiments, Figure 1A The server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Electronic devices can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the invention.

[0074] See Figure 2A , Figure 2A Figure 2 is a schematic diagram of the structure of a server 200 provided in an embodiment of this application. The server 200 shown in Figure 2 includes: at least one processor 410, a memory 450, and at least one network interface 420. The various components in the terminal 400 are coupled together through a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2A The general labeled all buses as Bus System 440.

[0075] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0076] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0077] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0078] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0079] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0080] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0081] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2A A lighting information processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a lighting acquisition module 4551, a difference acquisition module 4552, a light source configuration module 4553, and a lighting synchronization module 4554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0082] See Figure 2B , Figure 2B This is a schematic diagram of the structure of the terminal device 400 provided in the embodiments of this application. Figure 2BThe terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 440 in Figure 2.

[0083] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0084] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0085] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0086] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0087] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0088] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0089] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0090] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0091] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0092] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 shows a light information processing apparatus 455 stored in a memory 450, which can be software in the form of programs and plug-ins, including the following software modules: a light acquisition module 4551, a difference acquisition module 4552, a light source configuration module 4553, and a light synchronization module 4554. These modules are logically related, and therefore can be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0093] The illumination information processing method provided in this application will be described in conjunction with exemplary applications and implementations of the terminals provided in the embodiments of this application.

[0094] The following describes the illumination information processing method provided in the embodiments of this application. As mentioned above, the electronic device implementing the illumination information processing method in the embodiments of this application can be a terminal device, a server, or a combination of both. See also Figure 3A , Figure 3A This is a flowchart illustrating the illumination information processing method provided in the embodiments of this application, which will be combined with... Figure 3A The steps shown are explained.

[0095] In step 301, the real lighting information corresponding to the real scene is obtained.

[0096] For example, lighting information can be obtained by acquiring high dynamic range images. Lighting information includes at least the following parameters: light intensity, color temperature, light color, and light direction.

[0097] In some embodiments, the real-world scene includes at least one real-world reference object, which may include: a gray sphere, a chrome sphere, and a standard color swatch. Figure 5A , Figure 5AThis is a schematic diagram of the reference object provided in the embodiments of this application; Figure 5A These are real reference objects, and corresponding virtual reference objects can be simulated in the virtual scene. The real reference objects include: gray sphere 503A, chrome sphere 501A, standard color swatch 502A, and support 504A. The following parameters can be obtained from the HDRi image corresponding to the gray sphere: the direction of light, the intensity of shadows (e.g., rough or soft shadows, ensuring consistent light levels), and the color temperature of the light source. The following parameters can be obtained from the HDRi image corresponding to the chrome sphere: reflection, illumination, and alignment of the high dynamic range image. The color swatch, as a color standard reference, exhibits different characteristics under different light spectrums due to illumination effects and can be used as a reference object when constructing ambient lighting. Support 504A is used to support and integrate the above three reference objects.

[0098] In some embodiments, reference Figure 3B , Figure 3B This is a flowchart illustrating the illumination information processing method provided in the embodiments of this application. Step 301 can be implemented through the following steps 3011 to 3014, which are described in detail below.

[0099] In step 3011, when real lighting fixtures are set in the real scene, the sub-lighting information of the reference object position where each real reference object is located is detected, and the sub-lighting information is combined to obtain the real lighting information of the real scene.

[0100] For example, the real-world scene could be a film studio, and the location of the real reference object could be the location of the object to be filmed in a movie. Sub-lighting information of the reference object's location can be obtained using a spectrometer. This sub-lighting information includes: reference object position, light intensity, light color, color temperature, and light direction. Each sub-lighting information can be treated as a vector, and these vectors can be combined into a lighting information matrix. This lighting information matrix represents the true lighting information of the real-world scene.

[0101] In step 3012, when no real lighting fixtures are set in the real scene, the following processing is performed on each real reference object in the real scene: the real reference object is photographed according to different exposure times to obtain multiple first low dynamic range images corresponding to the real reference object.

[0102] For example, step 3012 is implemented using a real camera. The real camera can be positioned as a camera during film shooting. The real camera takes multiple images of the same real reference object at the same position with different exposure times, resulting in multiple first low dynamic range images with different exposure times.

[0103] In step 3013, a first high dynamic range image is synthesized based on each first low dynamic range image.

[0104] For example, the first high dynamic range image carries sub-illumination information of the reference object position corresponding to the real reference object, and the pixel bit depth of the first high dynamic range image is higher than that of the first low dynamic range image.

[0105] The following example illustrates the process of synthesizing the first high dynamic range (HDR) image. For instance, consider three first low dynamic range (LD) images of a real reference object: 1. An underexposed image, used to capture the very bright parts of the image. 2. A normally exposed image: a regular image captured by a camera based on the illuminance predicted by the camera from natural light. 3. An overexposed image: used to capture the very dark parts of the image. The exposure of the three images increases sequentially. The three images are aligned to eliminate ghosting in the HDR image, and the aligned images are then merged into a single HDR image, which carries the illumination information.

[0106] In step 3014, the sub-illumination information of each first high dynamic range image is combined to obtain the real illumination information of the real scene.

[0107] For example, the real scene includes multiple real reference objects, each of which corresponds to at least one first high dynamic range image. The sub-lighting information corresponding to each first high dynamic range image is represented as a vector form, and the vectors are combined into a matrix. The matrix is ​​used as the real lighting information of the real scene.

[0108] In this embodiment, the accuracy of acquiring lighting information is improved by acquiring lighting information from multiple locations in a real scene; the efficiency and accuracy of acquiring lighting information are improved by acquiring lighting information by capturing high dynamic range images, and the high dynamic range images can be directly read by the computer, saving the computing resources required to acquire lighting information.

[0109] In some embodiments, before step 301, environmental parameters in the real scene are obtained, wherein the environmental parameters include the size of the real scene; a virtual scene is constructed based on the environmental parameters of the real scene, wherein the virtual scene corresponds one-to-one with the real scene.

[0110] For example, a virtual scene could be used for virtual production, while the real scene is a film studio. In this case, the virtual scene is a simulation based on the scale of the real scene. Therefore, the relative positions of virtual reference objects in the virtual scene are the same as the relative positions of real reference objects in the real scene. Furthermore, the positions of the virtual camera and the real camera also correspond one-to-one.

[0111] In step 302, the virtual lighting information corresponding to the virtual scene is obtained.

[0112] For example, the virtual scene includes at least one virtual reference object, which is a virtual device simulated based on the real reference object. The virtual reference object includes: a virtual gray sphere, a virtual chrome sphere, and a virtual color swatch. The position of each virtual reference object in the virtual scene corresponds one-to-one with the position of each real reference object in the real scene.

[0113] Before step 302, virtual reference objects and virtual cameras are configured in the virtual scene as follows: For each virtual reference object, the following processing is performed: The real position of the real camera used to capture the real reference object in the real scene is obtained, wherein the relative position of the real reference object in the real scene is the same as the relative position of the virtual reference object in the virtual scene. The virtual position corresponding to the real position in the virtual scene is obtained, and a virtual camera is set at the virtual position, wherein the virtual camera is used to capture the second low dynamic range image corresponding to the virtual reference object.

[0114] For example, if a real 3D coordinate system is established in a real scene, then the same virtual 3D coordinate system is established in the virtual scene. If real camera A and virtual camera a in the virtual scene are in a corresponding relationship, then the coordinate values ​​of real camera A in the real 3D coordinate system are the same as the coordinate values ​​of virtual camera a in the virtual 3D coordinate system. Based on this setting, virtual camera a and real camera A can acquire low dynamic range images of the corresponding reference objects.

[0115] refer to Figure 3C , Figure 3C This is a flowchart illustrating the illumination information processing method provided in this application embodiment. Step 302 can be implemented through the following steps 3021 to 3023, which are described in detail below.

[0116] In step 3021, the virtual reference object is photographed using a virtual camera corresponding to the virtual reference object according to different exposure times, resulting in multiple second low dynamic range images of the virtual reference object.

[0117] For example, step 3021 is implemented through a virtual camera, which can be implemented by a terminal device or server as the execution subject. The principle of step 3021 is the same as that of step 3012, and will not be repeated here.

[0118] In step 3022, a second high dynamic range image is synthesized based on each second low dynamic range image.

[0119] For example, the second high dynamic range image carries sub-virtual lighting information corresponding to the position of the virtual reference object, and the pixel bit depth of the second high dynamic range image is higher than that of the second low dynamic range image. (Reference) Figure 5E , Figure 5EThis is a brightness simulation diagram of illumination information provided in this application embodiment, showing the brightness information (illuminance) carried by the second high dynamic range image corresponding to the virtual scene. The illuminance range 502E represents the illuminance range of the high dynamic range image 501E as 0–3000 lux. The high dynamic range image 501E is a schematic diagram of a high dynamic range image, and the color depth of each pixel in the high dynamic range image 501E represents the illumination intensity corresponding to each pixel.

[0120] For example, the execution of step 3022 is the same as that of step 3013, and will not be repeated here.

[0121] In step 3023, the virtual lighting information of the virtual scene is obtained by taking the sub-virtual lighting information.

[0122] For example, the execution of step 3023 is the same as that of step 3014, and will not be repeated here.

[0123] In some embodiments, the lighting information of the reference object position in the virtual scene can also be obtained directly by running the virtual engine of the virtual scene.

[0124] In this embodiment, by acquiring lighting information from multiple locations in the virtual scene that correspond to the real scene, the accuracy of acquiring lighting information is improved, the consistency of acquiring lighting information in the two scenes is improved, and lighting information synchronization is facilitated; by acquiring lighting information by capturing high dynamic range images, the efficiency and accuracy of acquiring lighting information are improved, and high dynamic range images can be directly read by the computer, saving the computing resources required to acquire lighting information.

[0125] Continue to refer to Figure 3A In step 303, the virtual lighting information and the real lighting information are compared and calculated to obtain the lighting difference information.

[0126] For example, high dynamic range (HMR) images of real-world scenes and virtual scenes are compared and processed to obtain illumination difference information.

[0127] In some embodiments, real lighting information is carried by a first high dynamic range image of the real scene; virtual lighting information is carried by a second high dynamic range image of the virtual scene; step 303 can be implemented by performing image comparison processing on the first high dynamic range image and the second high dynamic range image to obtain a difference high dynamic range image; and using the lighting information carried by the difference high dynamic range image as lighting difference information.

[0128] For example, image comparison processing can be achieved by performing subtraction on corresponding pixels between two images. Image subtraction can detect the difference high dynamic range image carrying the difference information between the two images. Alternatively, the illumination information corresponding to the high dynamic range images of the real scene and the virtual scene can be represented in matrix form, and the illumination difference information can be obtained by subtracting the two matrices.

[0129] In step 304, light attenuation calculation is performed based on the illumination difference information to obtain the configuration parameters of at least one supplementary light source.

[0130] For example, supplementary light sources are used to provide supplementary lighting to synchronize the lighting effects between the real and virtual scenes. Supplementary light sources can be virtual lights in the virtual scene or real lights in the real scene; the following explanations address different scenarios.

[0131] In some embodiments, the lighting difference information includes at least one of the following: differential lighting direction, differential lighting intensity, differential lighting color, and differential lighting color temperature relative to a reference object; when the virtual scene needs supplemental lighting, the reference... Figure 3D , Figure 3D This is a schematic flowchart of the illumination information processing method provided in the embodiments of this application. Step 304 can be achieved through the following steps 3041D to 3046D, which are described in detail below.

[0132] In step 3041D, based on the first lamp position of each real lamp in the real scene and the differential lighting direction, the real lamp position of each supplementary light source is determined, and each real lamp position is mapped to the virtual lamp position in the virtual scene.

[0133] For example, the direction of differential illumination can be determined using gray spheres in a high dynamic range image, which indicate the direction of the light. After obtaining the real-world light fixture positions, the relative positions in the virtual scene that correspond to the real-world light fixture positions are used as the virtual light fixture positions.

[0134] In some embodiments, there are two situations in the virtual scene where supplementary lighting is needed (virtual lights have been set, and virtual lights have not been set). Step 3041D is implemented in the following way: when no virtual lights are set in the virtual scene, the first light position of each real light in the real scene is used as the real light position corresponding to each supplementary light source, and each real light position is mapped to the virtual light position in the virtual scene.

[0135] For example, the position of the first lamp of each real lamp in the real scene can be directly mapped to the virtual scene to obtain the position of the virtual lamp corresponding to each supplementary light source.

[0136] When virtual lights are set in the virtual scene, obtain the first light position of each real light in the real scene, as well as the reference position of the real reference object. Take the direction of each first light position toward the reference object position as the real illumination direction, take the real illumination direction parallel to the differential illumination direction as the target direction, take the first light position in the target direction as the real light position of the supplementary light source, and map each real light position to the virtual light position in the virtual scene.

[0137] For example, the position of the virtual reference object corresponds to the position of the real reference object. The differential lighting direction is the direction in the virtual scene that lacks lighting compared to the virtual scene itself; it is the direction in which the supplementary light source emits light. Therefore, the real lighting direction parallel to the differential lighting direction is the lighting direction corresponding to the supplementary light source, i.e., the target direction. The position of the first light fixture in the real scene that intersects with the target direction is the real position of the supplementary light source.

[0138] Perform steps 3042D to 3046D for each supplementary light source.

[0139] In step 3042D, the first distance between the actual lamp position and the reference object position is obtained.

[0140] For example, the reference object position is the position of a real reference object in the real scene, and the direction from the real light fixture position to the reference object position is the target direction. The goal is to obtain the first distance between the real light fixture position and the reference object position in the real scene.

[0141] In step 3043D, light attenuation calculation is performed based on the difference in illumination intensity, target direction, and first distance to obtain the luminous intensity of the supplementary light source.

[0142] In some embodiments, step 3043D is implemented by: determining the first illumination intensity of the supplementary light source for the reference object position based on the difference in illumination intensity and the target direction; obtaining the square of the first distance and multiplying the square by the first illumination intensity to obtain the luminous intensity of the supplementary light source.

[0143] For example, the illumination intensity carries directional data. The differential illumination intensity extracted from the illumination difference information includes illumination intensities in multiple directions. The illumination intensity parallel to the target direction is taken as the first illumination intensity of the supplementary light source for the reference object position. The illumination attenuation can be referenced by the inverse square law, which states that the intensity of the action of an object or particle decreases linearly with the square of the distance, that is, the force is inversely proportional to the square of the distance. Then the relationship between the luminous intensity of the supplementary light source and the first illumination intensity can be characterized by the following formula (1):

[0144]

[0145] Where I is the luminous intensity of the supplementary light source, L1 is the first distance, and E1 is the first illumination intensity. Based on formula (1), it can be determined that the luminous intensity is the product of the square of the first distance L1 and the first illumination intensity E1.

[0146] In step 3044D, the light source color of the supplementary light source in the target direction is determined based on the difference light color corresponding to each difference light direction and the target direction.

[0147] For example, the differential lighting colors extracted from the lighting difference information include lighting colors from multiple directions, and the lighting colors from directions parallel to the target direction are extracted as the light source colors of the supplementary light source.

[0148] In step 3045D, the color temperature of the supplementary light source in the target direction is determined based on the color temperature of the differential illumination corresponding to each differential illumination direction and the target direction.

[0149] For example, the differential light color temperature extracted from the light difference information includes the light color temperature in multiple directions, and the light color temperature in the direction parallel to the target direction is extracted as the light color of the supplementary light source.

[0150] In step 3046D, the target direction, light color temperature, light source color, luminous intensity, and virtual light fixture position are combined to obtain the configuration parameters of the supplementary light source.

[0151] For example, the target direction, light color temperature, light source color, luminous intensity, and virtual lamp position of the same supplementary light source can be treated as parameters of one dimension, and the parameters of each dimension can be combined into a vector. That is, the configuration parameters of the supplementary light source can be represented in vector form.

[0152] In some embodiments, the illumination difference information includes: the difference in illumination direction, difference in illumination intensity, difference in illumination color, and difference in illumination color temperature relative to a reference object; when supplementary illumination is needed in a real scene, the reference... Figure 3E , Figure 3E This is a schematic flowchart of the illumination information processing method provided in the embodiments of this application. Step 304 can be achieved through the following steps 3041E to 3046E, which will be described in detail below.

[0153] In step 3041E, based on the second lamp position and differential lighting direction of each virtual lamp in the virtual scene, the virtual lamp position of each supplementary light source is determined, and each virtual lamp position is mapped to the real lamp position in the real scene.

[0154] In some embodiments, step 3041E can be implemented in the following way: when no real lamps are set in the real scene, the second lamp position of each virtual lamp in the virtual scene is taken as the virtual lamp position corresponding to each supplementary light source; when real lamps are set in the real scene, the direction of each second lamp position toward the reference object position is taken as the virtual illumination direction, the virtual illumination direction parallel to the differential illumination direction is taken as the target direction, and the second lamp position in the target direction is taken as the virtual lamp position of the supplementary light source.

[0155] For example, obtaining the virtual light fixture location and obtaining the real light fixture location are interconnected. The implementation of step 3041E can be referenced in step 3041D.

[0156] Perform steps 3042E to 3046E for each supplementary light source.

[0157] In step 3042E, the second distance between the virtual lamp position and the reference object position is obtained.

[0158] For example, the reference object position is the position of a virtual reference object in the virtual scene, and the direction from the virtual light fixture position to the reference object position is the target direction. Obtain the second distance L2 between the virtual real light fixture position and the reference object position in the virtual scene.

[0159] In step 3043E, light attenuation calculation is performed based on the difference in illumination intensity, target direction, and second distance to obtain the luminous intensity of the supplementary light source.

[0160] In some embodiments, step 3043E can be implemented as follows: based on the difference in illumination intensity and the target direction, determine the second illumination intensity of the supplementary light source for the reference object position in the target direction; obtain the square of the second distance, and multiply the square by the second illumination intensity to obtain the luminous intensity of the supplementary light source.

[0161] For example, the illumination intensity carries directional data. The differential illumination intensity extracted from the illumination difference information includes illumination intensities in multiple directions. The illumination intensity parallel to the target direction is used as the second illumination intensity of the supplementary light source for the reference object position. The illumination attenuation can be referenced by the inverse square law, which states that the intensity of the action of an object or particle decreases linearly with the square of the distance, that is, the force is inversely proportional to the square of the distance. Then the relationship between the luminous intensity of the supplementary light source and the second illumination intensity can be characterized by the following formula (2):

[0162]

[0163] Where I is the luminous intensity of the supplementary light source, L2 is the second distance, and E2 is the second illumination intensity. Based on formula (1), it can be determined that the luminous intensity is the product of the square of the first distance L2 and the second illumination intensity E2.

[0164] In step 3044E, when no real lighting fixtures are set in the real scene, the color temperature and light source color of the virtual lighting fixtures at the virtual lighting fixture locations are obtained.

[0165] For example, when there is no real light fixture at the location of the real light source in the real scene, the color temperature and color of the virtual light fixture in the virtual scene can be directly obtained as the configuration parameters of the supplementary light source.

[0166] In step 3045E, when real lighting fixtures are already set in the real scene, the light source color of the supplementary light source is determined based on the difference in illumination color and the target direction, and the light color temperature of the supplementary light source is determined based on the difference in illumination color temperature.

[0167] For example, if a real light fixture already exists at the location corresponding to the supplementary lighting source in a real scene, it means that another corresponding light fixture needs to be added to that location. Therefore, the corresponding configuration parameters are determined based on the illumination difference information. The principle of step 3045E is the same as that of steps 3044D to 3045D.

[0168] In step 3046E, the target direction, light color temperature, light source color, luminous intensity, and actual lamp position are combined to obtain the configuration parameters of the supplementary light source.

[0169] In this embodiment, the light source configuration parameters are obtained from different directions for different situations, which improves the accuracy of obtaining the light source configuration parameters and enhances the effect of illumination synchronization processing.

[0170] Continue to refer to Figure 3A In step 305, the configuration parameters of each supplementary light source are matched with the luminaire database to obtain the target luminaire corresponding to each supplementary light source in the luminaire database.

[0171] For example, the lighting database stores the performance parameters of multiple lighting fixtures. Performance parameters are the range of values ​​for configuration parameters. For instance, lighting fixture A is an adjustable illuminance fixture with the following performance parameters: illuminance of 220 Lx to 340 Lx; the illuminance produced by a luminous flux of lumens uniformly distributed over an area of ​​1 square meter is one lux (Lx).

[0172] In some embodiments, reference Figure 3F , Figure 3F This is a flowchart illustrating the illumination information processing method provided in the embodiments of this application. Step 305 can be achieved through the following steps 3051 to 3053, which will be described in detail below.

[0173] In step 3051, the performance parameters of the lamps in the lamp database are obtained.

[0174] For example, the performance parameters of the luminaire include the value ranges of different types of illumination parameters, including illumination intensity, illumination color, and illumination color temperature. In some embodiments, the value range of the position where the luminaire can be set can also be considered as a performance parameter. For example, the luminaire can be set on a track on the ceiling of a photography studio, and the position range corresponding to the track is the value range of the luminaire's position; the luminaire can be set on a robotic arm in the photography studio, and the range of motion of the robotic arm is the value range of the luminaire's position.

[0175] Perform steps 3052 to 3053 for each supplementary light source's configuration parameters.

[0176] In step 3052, different types of lighting parameters are obtained from the configuration parameters.

[0177] For example, the lighting parameters of a supplementary light source include: light intensity, light color, and light color temperature.

[0178] In step 3053, the illumination parameters of each type are matched with the performance parameters of different lamps, and the lamps that meet the matching conditions are used as the target lamps corresponding to the supplementary light sources.

[0179] For example, the matching conditions include: each type of illumination parameter of the supplementary light source belongs to the value range of the illumination parameter corresponding to the luminaire.

[0180] For example, the configuration parameters of supplementary light source Y are: color temperature 2800K (Kelvin), light color white, and illuminance 500Lx; the performance parameters of luminaire X obtained from the luminaire database are: color temperature fixed at 2800K; light color can be varied, including white, yellow, and beige; illuminance can be adjusted from 350Lx to 750Lx. Since all the configuration parameters of supplementary light source Y fall within the range of the performance parameters corresponding to luminaire X, then luminaire X is the target luminaire.

[0181] In some embodiments, when multiple target luminaires meet the matching criteria, the most energy-efficient, smallest, or easiest-to-move luminaire can be selected as the final target luminaire. For example, the supplementary light source is a real luminaire, and the target luminaire whose positional performance parameters obtained from the target luminaire satisfy the parameters of the real luminaire.

[0182] In this embodiment, by matching configuration parameters in a lighting database, the required target lighting fixtures can be accurately obtained, which can improve shooting efficiency in real scenes, save computing resources, and reduce the manpower costs required for virtual production.

[0183] Continue to refer to Figure 3A In step 306, each target luminaire and its corresponding configuration parameters are used as configuration information, and scene lighting synchronization is performed based on the configuration information.

[0184] In some embodiments, reference Figure 3G , Figure 3G This is a schematic flowchart of the illumination information processing method provided in the embodiments of this application. Step 306 can be achieved through the following steps 3061 to 3062, which will be described in detail below.

[0185] In step 3061, when the virtual scene needs supplemental lighting, the corresponding virtual lights are set in the virtual scene based on the configuration information, and the parameters corresponding to each virtual light are configured.

[0186] For example, when a virtual scene needs additional lighting, the virtual engine of the virtual scene is called based on the configuration parameters to set the corresponding virtual lights, thereby achieving lighting synchronization of the virtual scene.

[0187] In step 3062, when the real scene needs supplemental lighting, a lighting control signal for the real light fixture corresponding to each target light fixture is generated based on the configuration information, and each lighting control signal is sent to each real light fixture in the real scene.

[0188] For example, the lighting control signal is used to set the configuration parameters of the actual lighting fixture; the lighting control signal is also known as the Digital Multiple X (DMX) signal.

[0189] In some embodiments, if additional lighting is needed in a real-world scenario, the corresponding real lighting can be turned on and moved to the corresponding position in the real-world scenario by controlling the target lighting with a digital dimming signal.

[0190] In this embodiment, different lighting operations are performed for different scenarios to synchronize the lighting of the virtual scene with that of the real scene, thereby improving the efficiency of lighting synchronization, saving the computing resources required for lighting synchronization, and improving the shooting efficiency of virtual production.

[0191] In some embodiments, the present application is implemented through collaboration between a server and a terminal device, as described above. Figure 4 , Figure 4 This is a flowchart illustrating the illumination information processing method provided in this application embodiment, showing the interaction process between the server 200, the terminal device 400, the lighting device 500, and the detection device 600.

[0192] The detection device 600 executes step 401, sending the real lighting information of the real scene to the server 200.

[0193] For example, the detection device 600 can be a camera, spectrometer, or other device used to acquire illumination information in a real scene. For instance, a camera acquires a high dynamic range image of the real scene carrying illumination information. Alternatively, a spectrometer acquires the light intensity and light spectrum of the real scene.

[0194] Server 200 executes step 402, determining configuration information based on virtual lighting information and real lighting information.

[0195] For example, the specific process of server 200 executing step 402 can be found in the above text. Figure 3A Steps 301 to 305.

[0196] Server 200 executes step 403, when the real scene needs supplemental lighting, it sends configuration information to terminal device 400.

[0197] Terminal device 400 executes step 404, sending a digital dimming signal to lighting device 500 based on configuration information.

[0198] For example, terminal device 400 may be a computer with an integrated digital dimming console, equipped with a display screen and a user interface. Technicians can use terminal device 400 to control real lighting fixtures in a real scene, or use terminal device 400 to obtain virtual lighting information formed by virtual lighting fixtures in a virtual scene.

[0199] Server 200 executes step 405, when the virtual scene needs supplemental lighting, it sends the virtual scene image after supplemental lighting to terminal device 400. Terminal device executes step 406, displaying the virtual scene image.

[0200] This application embodiment obtains lighting information from virtual and real scenes, compares and calculates the lighting information to obtain lighting difference information, thereby improving the accuracy of obtaining lighting differences between virtual and real scenes; based on the lighting difference information, it obtains the configuration parameters corresponding to the supplementary light source, and obtains the target light source for supplementary lighting from the lighting database; based on the configuration information, it performs lighting synchronization on the real or virtual scene to be supplemented with lighting, thereby improving the accuracy and efficiency of lighting synchronization and saving the cost required for lighting synchronization.

[0201] The following will describe an exemplary application of the illumination information processing method of this application in a practical application scenario.

[0202] The shooting process of virtual production differs from traditional shooting methods. Because the shooting studio content involves self-illuminating LED equipment (such as LED curtain walls), there is a process of mutual accommodation between the lighting of the virtual scene and the real scene. Due to the shooting requirements of the real scene in virtual production, there are many devices that emit light in addition to lighting equipment in the real scene. However, there is no need to set up corresponding devices in the virtual scene. How to quickly match and balance the virtual light and the real light and reproduce the effect in the virtual scene becomes very important. In related technologies, the lighting configuration of the virtual scene needs to be manually set by technicians, and the lighting matching in the real scene also needs to be configured by technicians, which leads to inaccurate lighting matching and increases the difficulty of lighting matching operations.

[0203] This application's embodiments can establish a connection between virtual and real lighting in virtual shooting by creating lighting information carried by high dynamic range (HDR) images (including synchronizing lighting from the virtual scene to the real scene, and vice versa), achieving a lighting matching effect and overcoming the bottleneck of repeatedly manually adjusting virtual and real lighting in virtual shooting. The lighting information carried by HDR images can be quickly reproduced in the virtual scene of the virtual engine and the shooting studio (real scene), effectively saving lighting adjustment time during virtual production and achieving uniform lighting effects.

[0204] For example, high dynamic range (HDR) image acquisition technology is often used in post-production. HDR images record the lighting information of the environment, which can be used to set scene lighting in virtual scenes to "illuminate" them. Many HDR image files are provided as panoramic images, which can be used as environmental backgrounds to create reflections and refractions. HDR images and panoramic images are fundamentally different. Panoramic images refer to ordinary images that contain a 360-degree view of the scene; they can be in JPG, BMP, TGA, etc., and belong to low dynamic range (LVR) images. Panoramic images do not carry lighting information.

[0205] In this embodiment, the technical solutions for synchronizing lighting in a virtual scene to the real scene and synchronizing lighting in a real scene to the virtual scene are interchangeable and inversely related. This embodiment uses synchronizing lighting in a virtual scene to the real scene as an example for explanation. Synchronizing lighting in a virtual scene to the real scene includes two scenarios: first, no lighting equipment is installed in the real scene; second, lighting equipment is already installed in the real scene.

[0206] The first scenario will be explained below; see [link / reference]. Figure 6A The terminal device 400 is the execution entity, and the terminal device 400 can be a computer running the virtual engine. Figure 6A This is a flowchart illustrating the illumination information processing method provided in the embodiments of this application, which will be combined with... Figure 6A The steps shown are explained.

[0207] In step 601A, the lighting information of key points in the virtual environment is obtained.

[0208] For example, key points refer to the placement of virtual reference objects within a virtual scene. Virtual scenes are constructed based on environmental information from real-world scenes.

[0209] After the virtual scene is generated and created (i.e., the lighting effects have been set up in the virtual scene), virtual reference objects are placed at the corresponding reference object positions for each virtual camera in the virtual scene. These virtual reference objects include: gray spheres, chrome spheres, and standard color swatches. The lighting information for each reference object position in the virtual scene can be obtained through the virtual engine. This lighting information includes various parameters such as: lighting direction, lighting intensity, lighting color, color temperature, and light source position.

[0210] refer to Figure 5A , Figure 5A This is a schematic diagram of the reference object provided in the embodiments of this application; Figure 5A These are real-world reference objects that can be used to simulate corresponding devices in a virtual scene. The real-world reference objects include a gray sphere 503A, a chrome sphere 501A, a standard color swatch 502A, and a support 504A. The gray sphere provides the following parameters: direction of light, intensity of shadows (e.g., rough or soft shadows, ensuring consistent light levels), and color temperature of the light source. The chrome sphere provides the following parameters: reflection, illumination, and alignment of high dynamic range images. The color swatch, as a color standard reference, exhibits different characteristics under different light spectrums due to illumination variations and can be used as a reference when constructing ambient lighting in the real-world environment.

[0211] Example, reference Figure 5E , Figure 5E This is a brightness simulation diagram of illumination information provided in the embodiments of this application, which shows the brightness information (light intensity) carried by the second high dynamic range image corresponding to the virtual scene.

[0212] In step 602A, the initial lighting information of key points in the real scene is obtained.

[0213] For example, this application embodiment uses a virtual filming studio as the real scene for illustration, referencing... Figure 7 , Figure 7This is a schematic diagram of a real-world scene provided in this application embodiment. The virtual filming studio 702 is equipped with various electronic devices, each with its own display screen that emits light. For example, an LED screen 701 is used to play background images (or videos), emitting light when displaying content. In this case, even if the same lighting fixtures and corresponding parameters are used in both the real and virtual scenes, the lighting differs between them. When no lighting fixtures are provided in the real scene, the initial lighting information generated by the natural light from the electronic devices in the real scene is obtained.

[0214] For example, let's illustrate this using a real-world scenario where only an LED curtain wall provides illumination. For instance, by placing identical gray spheres, chrome spheres, and standard color swatches at the same reference locations as in the virtual scene, and using a real camera to capture basic lighting information of the real environment under the illumination of only the LED curtain wall, we can eliminate the influence of the LED curtain wall on the lighting information. As an example, we can sample the real scene using a real camera to obtain a high dynamic range (HDR) image file carrying the lighting information; this HDR image file is then transmitted to the terminal device in real time.

[0215] For example, a spectrometer can also be used to measure data at the location of a reference object. The spectrometer can acquire information such as brightness, color temperature, spectrum, and color at that location in real time. A spectrometer is a device that uses photodetectors such as photomultiplier tubes to measure the intensity of spectral lines at different wavelengths.

[0216] refer to Figure 5B , Figure 5B This is a schematic diagram of the handheld spectrometer provided in this application embodiment; in the shooting studio, the light intensity at different reference object positions can be measured by the handheld spectrometer 501B. The screen displayed in the human-computer interaction interface 502B of the handheld spectrometer 501B is a schematic diagram of the interface for obtaining environmental chromatographic information, and the light intensity and other information are input into the terminal device for light information processing.

[0217] refer to Figure 5C And 5D, Figures 5C to 5D This is a schematic diagram of the human-computer interaction interface of the handheld spectrometer provided in the embodiments of this application; Figure 5C The 501C human-computer interaction interface represents the interface for obtaining ambient brightness information, where LUX218.6 represents the light intensity measured by the handheld spectrometer as 218.6 lux. Figure 5DThe interface for acquiring environmental chromatographic information is described. In the human-computer interaction interface 501D, CIE197 indicates that the color space in which the chromatographic information is located is the uniform color space recommended by the International Institute of Illumination (CIE) in 1976. U' and V' are the chromaticity coordinates of the reference object position measured by the handheld spectrometer. In the human-computer interaction interface 502D, X and Y are the coordinate values ​​of the color of the reference object position measured by the handheld spectrometer in the uniform color space.

[0218] For example, steps 602A and 601A can be performed simultaneously.

[0219] In step 603A, the high dynamic range images of key points corresponding to the virtual scene and the real scene are compared to obtain illumination difference information.

[0220] The terminal device calculates, simulates, and compares the lighting information carried in the high dynamic range (HDL) image file of the real scene with the HDL images of gray spheres, chrome spheres, and color swatches placed in the virtual scene to obtain lighting difference information. The lighting information of each reference object position in the virtual scene is known, including direction, brightness, color temperature, and color. Through image comparison, a HDL image carrying lighting difference information can be obtained.

[0221] In step 604A, based on the illumination difference information, a three-dimensional lighting matrix diagram corresponding to the real scene is simulated according to the performance parameters of the lamps and the light attenuation formula.

[0222] For example, the lighting library includes: information on commonly used lighting fixtures in the market, and how to place the lights to achieve the corresponding effects through calculations, which will be explained below.

[0223] The terminal device has a built-in library of commonly used lighting fixtures (lighting information), including but not limited to the brightness, color temperature, and hue range of each fixture. It automatically simulates lighting effects in a virtual scene. Because it uses reference objects such as gray spheres, chrome spheres, and color swatches, along with specific brightness, spectrum, and color temperature information, it performs graphical comparison calculations on the high dynamic range images of the reference objects to obtain differences in lighting information. It can simulate the types and positions of various lighting fixtures that should be placed in a real scene using light attenuation in the virtual scene, determining how to achieve the effects of pre-set values, thereby improving shooting efficiency and lighting matching. Simulating the types and positions of various lighting fixtures that should be placed in a real scene using light attenuation in the virtual scene includes: obtaining the direction of light from differences in lighting information to determine the position of the light source; and calculating light attenuation based on the distance between the light source and the reference object, and the light intensity corresponding to the reference object, to obtain the luminous intensity (brightness) of the light source.

[0224] For example, light intensity attenuation can be understood using the inverse square law, which states that the intensity of an object's or particle's force decreases linearly with the square of the distance; that is, the force is inversely proportional to the square of the distance. The light intensity on the reference surface = the luminous intensity of the light source / the square of the distance between the reference object and the light source. Based on this formula, the luminous intensity (brightness) of the light source can be obtained.

[0225] For example, the terminal device calculates and outputs a three-dimensional lighting matrix. The dimensions of the lighting matrix include: lighting setup information (the specific location of each light fixture in the real scene), light fixture brand type, and corresponding configuration parameters of the light fixture.

[0226] In step 605A, a digital dimming signal is generated based on the three-dimensional light matrix diagram, and the digital dimming signal is sent to the corresponding real light fixtures in the real scene.

[0227] For example, lighting equipment can be set up on a specific track and, upon receiving a digital dimming signal, move to the target location carried by the digital dimming signal. Lighting equipment can also be manually placed into a real-world scene.

[0228] For example, once the real lighting fixtures in the real scene are set up, the lighting in the two scenes can be checked to determine if the lighting is synchronized. This check can be performed as follows: After the real lighting fixtures are set up in the real scene, a random reference point (different from the reference point corresponding to step 601A or step 602A) is selected within any shooting range in the real scene. Data is collected and measured, and the consistency of the lighting information in the two scenes is compared. If they are consistent, it indicates that the lighting effect is accurately synchronized. If they are inconsistent, the terminal device calculates and adjusts the values ​​that the lighting fixtures need to output, such as brightness, color temperature, and color; this process continues until they are consistent.

[0229] In some embodiments, lighting fixtures are already set up in the real scene. The type of lighting fixtures to be used by the lighting technician is specified in advance, and the on-site lighting is set up according to the lighting technician's method. The position of the lights is known through lighting positioning. In some embodiments, 4-5 commonly used camera positions (such as any one of the 9 conventional camera positions in film studies) can be set up in the real scene, and gray spheres, chrome spheres, color swatches, and colorimeters are placed there. Illumination information is obtained through gray spheres, chrome spheres, color swatches, and spectrometers.

[0230] See Figure 6B The terminal device 400 is the execution entity, and the terminal device 400 can be a computer running the virtual engine. Figure 6B This is a flowchart illustrating the illumination information processing method provided in the embodiments of this application, which will be combined with... Figure 6B The steps shown are explained. For example, Figure 6BThis refers to the process of synchronizing lighting information in a real-world scenario where lighting fixtures are already in place. Steps 601B to 604B can be executed with reference to steps 601A to 603A above.

[0231] In step 601B, the lighting information of key points in the virtual scene is obtained.

[0232] In step 602B, the lighting information of key points in the real scene is obtained.

[0233] After steps 602B and 601B, step 603B is executed to compare the high dynamic range images of key points corresponding to the virtual scene and the real scene respectively to obtain illumination difference information.

[0234] In step 604B, based on the illumination difference information, the lighting parameters corresponding to the real scene are simulated according to the performance parameters of the lamps and the light attenuation formula.

[0235] For example, given the types of lighting fixtures in the real-world scene, the lighting parameters to be adjusted can be calculated using the light attenuation formula. Examples include: the light intensity to be reduced, and the color temperature to be adjusted.

[0236] In step 605B, a digital dimming signal is generated based on the lighting parameters and sent to the actual lighting fixtures that have been configured in the real scene.

[0237] For example, the terminal device running the virtual engine sends the parameters corresponding to the light to the digital dimming signal control console, which then sends the digital dimming signal to the real light fixture, so that the corresponding light is adjusted to the target value.

[0238] For example, a digital dimming signal controller can be a standalone terminal device, or it can be integrated into a computer if the computer has the function of sending digital dimming signals.

[0239] In step 606B, the high dynamic range images of key points in the virtual scene and the real scene are compared to determine if there is illumination difference information. If the result of step 606B is yes, step 607B is executed to obtain the type and location of the lamps to be added based on the illumination difference information. If the result of step 606B is no, illumination synchronization ends.

[0240] For example, after the real lighting fixtures in the real scene are adjusted, a random reference object position (different from the reference object position corresponding to step 601B or step 602B) is selected within any shooting range in the real scene. Data is collected and measured, and the consistency of the lighting information in the two scenes is compared. If they are consistent, it means that the lighting effect is accurately synchronized. If they are inconsistent, it means that the currently set real lighting fixtures cannot meet the requirements of synchronized lighting effect. Then, the terminal device calculates the real lighting fixtures to be added and their configuration parameters, turns on the corresponding real lighting fixtures, controls them to move to the corresponding positions, and configures the corresponding configuration parameters. Alternatively, the lighting technician sets the newly added lighting fixtures in the corresponding positions and configures the configuration parameters of the newly added real lighting fixtures through digital dimming signals.

[0241] This application embodiment improves on-site shooting efficiency by synchronizing lighting between virtual and real scenes. It effectively enables teams that have never used virtual shooting to quickly set up lighting schemes, restore the lighting effects in the virtual scene, save a lot of time on debugging and lighting placement, and improve shooting efficiency and shooting effect.

[0242] The following description continues to illustrate the exemplary structure of the illumination information processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2A or Figure 2B As shown, the software modules in the illumination information processing device 455 stored in the memory 440 may include: an illumination acquisition module 4551, configured to acquire real illumination information corresponding to a real scene; the illumination acquisition module 4551 is also configured to acquire virtual illumination information corresponding to a virtual scene; a difference acquisition module 4552, configured to compare and calculate the virtual illumination information with the real illumination information to obtain illumination difference information; a light source configuration module 4553, configured to perform light attenuation calculation based on the illumination difference information to obtain configuration parameters for at least one supplementary light source, wherein the supplementary light source is used for supplementary illumination; the light source configuration module 4553 is also configured to match the configuration parameters of each supplementary light source with a luminaire database to obtain the target luminaire in the luminaire database corresponding to each supplementary light source; and an illumination synchronization module 4554, configured to use each target luminaire and its corresponding configuration parameters as configuration information, and perform scene illumination synchronization based on the configuration information.

[0243] In some embodiments, the real scene includes at least one real reference object; the illumination acquisition module 4551 is configured to, when real lighting fixtures are set in the real scene, detect the sub-illumination information of the reference object position where each real reference object is located, and combine each sub-illumination information to obtain the real illumination information of the real scene; when no real lighting fixtures are set in the real scene, perform the following processing on each real reference object in the real scene: take pictures of the real reference object according to different exposure times to obtain multiple first low dynamic range images corresponding to the real reference object; synthesize a first high dynamic range image based on each first low dynamic range image, wherein the first high dynamic range image carries the sub-illumination information of the reference object position corresponding to the real reference object, and the pixel bit depth of the first high dynamic range image is higher than the pixel bit depth of the first low dynamic range image; combine the sub-illumination information of each first high dynamic range image to obtain the real illumination information of the real scene.

[0244] In some embodiments, the virtual scene includes at least one virtual reference object, wherein the position of each virtual reference object in the virtual scene corresponds one-to-one with the position of each real reference object in the real scene; the lighting acquisition module 4551 is configured to perform the following processing for each virtual reference object before acquiring the virtual lighting information corresponding to the virtual scene: acquiring the real position of the real camera used to capture the real reference object in the real scene, wherein the relative position of the real reference object in the real scene is the same as the relative position of the virtual reference object in the virtual scene; acquiring the virtual position corresponding to the real position in the virtual scene, and setting a virtual camera at the virtual position, wherein the virtual camera is used to capture a second low dynamic range image corresponding to the virtual reference object.

[0245] In some embodiments, the illumination acquisition module 4551 is configured to perform the following processing on each virtual reference object in the virtual scene: using a virtual camera corresponding to the virtual reference object, the virtual reference object is photographed according to different exposure times to obtain multiple second low dynamic range images corresponding to the virtual reference object; a second high dynamic range image is synthesized based on each second low dynamic range image, wherein the second high dynamic range image carries sub-virtual illumination information of the reference object position corresponding to the virtual reference object, and the pixel bit depth of the second high dynamic range image is higher than the pixel bit depth of the second low dynamic range image; and the virtual illumination information of the virtual scene is obtained by combining each sub-virtual illumination information.

[0246] In some embodiments, real lighting information is carried by a first high dynamic range image of the real scene; virtual lighting information is carried by a second high dynamic range image of the virtual scene; the difference acquisition module 4552 is configured to perform image comparison processing on the first high dynamic range image and the second high dynamic range image to obtain a difference high dynamic range image; and use the lighting information carried by the difference high dynamic range image as lighting difference information.

[0247] In some embodiments, the illumination difference information includes at least one of the following: difference illumination direction, difference illumination intensity, difference illumination color, and difference illumination color temperature for a reference object; the light source configuration module 4553 is configured to, when the virtual scene needs supplemental illumination, determine the real lamp position of each supplementary light source based on the first lamp position of each real lamp in the real scene and the difference illumination direction, and map each real lamp position to a virtual lamp position in the virtual scene; and perform the following processing for each supplementary light source: obtain a first distance between the real lamp position and the reference object position, wherein the reference object position is the real lamp position. The location of the real reference object in the scene, and the direction from the real light fixture to the reference object, are defined as the target direction. Light attenuation calculations are performed based on the differential illumination intensity, the target direction, and the first distance to obtain the luminous intensity of the supplementary light source. The color of the supplementary light source in the target direction is determined based on the differential illumination color corresponding to each differential illumination direction and the target direction. The color temperature of the supplementary light source in the target direction is determined based on the differential illumination color temperature corresponding to each differential illumination direction and the target direction. The configuration parameters of the supplementary light source are obtained by combining the target direction, color temperature, light source color, luminous intensity, and the virtual light fixture position.

[0248] In some embodiments, the light source configuration module 4553 is configured to, when no virtual lights are set in the virtual scene, use the first light position of each real light in the real scene as the real light position corresponding to each supplementary light source; when virtual lights are set in the virtual scene, obtain the first light position of each real light in the real scene and the reference position of the real reference object, use the direction of each first light position toward the reference object position as the real illumination direction, use the real illumination direction parallel to the differential illumination direction as the target direction, and use the first light position of the target direction as the real light position of the supplementary light source.

[0249] In some embodiments, the light source configuration module 4553 is configured to determine the first illumination intensity of the supplementary light source for the reference object position in the target direction based on the difference in illumination intensity and the target direction; obtain the square of the first distance and multiply the square by the first illumination intensity to obtain the luminous intensity of the supplementary light source.

[0250] In some embodiments, the illumination difference information includes: differential illumination direction, differential illumination intensity, differential illumination color, and differential illumination color temperature for the reference object; the light source configuration module 4553 is configured to, when supplementary illumination is needed in the real scene, determine the virtual lamp position of each supplementary light source based on the second lamp position of each virtual lamp in the virtual scene and the differential illumination direction, and map each virtual lamp position to the real lamp position in the real scene; and perform the following processing for each supplementary light source: obtain the second distance between the virtual lamp position and the reference object position, wherein the reference object position is the position of a virtual reference object in the virtual scene. The direction from the virtual light fixture to the reference object is taken as the target direction. Light attenuation is calculated based on the difference in illumination intensity, the target direction, and the second distance to obtain the luminous intensity of the supplementary light source. When no real light fixtures are set in the real scene, the color temperature and color of the virtual light fixture at the virtual light fixture's location are obtained. When real light fixtures are set in the real scene, the color of the supplementary light source is determined based on the difference in illumination color and the target direction, and the color temperature of the supplementary light source is determined based on the difference in illumination color temperature. The target direction, color temperature, color of the light source, luminous intensity, and the position of the real light fixture are combined to obtain the configuration parameters of the supplementary light source.

[0251] In some embodiments, the light source configuration module 4553 is configured to, when no real light fixtures are set in the real scene, use the second light fixture position of each virtual light fixture in the virtual scene as the virtual light fixture position corresponding to each supplementary light source; when real light fixtures are set in the real scene, use the direction of each second light fixture position toward the reference object position as the virtual illumination direction, use the virtual illumination direction parallel to the differential illumination direction as the target direction, and use the second light fixture position in the target direction as the virtual light fixture position of the supplementary light source.

[0252] In some embodiments, the light source configuration module 4553 is configured to determine the second illumination intensity of the supplementary light source relative to the reference object position in the target direction based on the difference in illumination intensity and the target direction; obtain the square of the second distance, and multiply the square by the second illumination intensity to obtain the luminous intensity of the supplementary light source.

[0253] In some embodiments, the light source configuration module 4553 is configured to obtain the performance parameters of lamps in the lamp database, wherein the performance parameters of the lamps include: the value range of different types of illumination parameters; and perform the following processing for the configuration parameters of each supplementary light source: obtain different types of illumination parameters in the configuration parameters, wherein the illumination parameters include: illumination intensity, illumination color, and illumination color temperature; match each type of illumination parameter with the performance parameters of different lamps, and take the lamps that meet the matching conditions as the target lamps corresponding to the supplementary light source; wherein the matching conditions include: each type of illumination parameter of the supplementary light source belongs to the value range of the illumination parameter corresponding to the lamp.

[0254] In some embodiments, the illumination synchronization module 4554 is configured to, when the virtual scene needs supplemental illumination, set corresponding virtual lamps in the virtual scene based on configuration information and configure the parameters corresponding to each virtual lamp; when the real scene needs supplemental illumination, generate lamp control signals for the real lamps corresponding to each target lamp based on configuration information and send each lamp control signal to each real lamp in the real scene, wherein the lamp control signals are used to set the configuration parameters of the real lamps.

[0255] In some embodiments, the illumination synchronization module 4554 is configured to acquire environmental parameters in the real scene before acquiring the real illumination information corresponding to the real scene, wherein the environmental parameters include the size of the real scene; and to construct a virtual scene based on the environmental parameters of the real scene, wherein the virtual scene corresponds one-to-one with the real scene.

[0256] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the illumination information processing method described in this application.

[0257] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to execute the lighting information processing method provided in this application. For example, ... Figure 3A The method for processing illumination information is shown.

[0258] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0259] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0260] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0261] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0262] In summary, the embodiments of this application improve the accuracy of obtaining lighting differences between virtual and real scenes by acquiring lighting information from virtual and real scenes and comparing and calculating the lighting information. Based on the lighting difference information, the configuration parameters corresponding to the supplementary light source are obtained, and the target light fixture for supplementary lighting is obtained from the lighting database. Based on the configuration information, lighting synchronization is performed on the real or virtual scene to be supplemented, which improves the accuracy and efficiency of lighting synchronization and saves the cost required for lighting synchronization.

[0263] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for processing illumination information, characterized in that, The method includes: Obtain the real-world lighting information corresponding to the real scene; Obtain virtual lighting information corresponding to the virtual scene; The virtual lighting information is compared and calculated with the real lighting information to obtain lighting difference information; When the virtual scene needs supplemental lighting, based on the first lamp position of each real lamp in the real scene, the real lamp position corresponding to each supplemental light source is determined, and each real lamp position is mapped to a virtual lamp position in the virtual scene; Based on the illumination difference information and the virtual lamp position, each of the supplementary light sources is processed to obtain configuration parameters for at least one supplementary light source, wherein the supplementary light source is used for supplementary illumination; Based on the configuration parameters of each supplementary light source, a matching process is performed with the lighting database to obtain the target lighting fixture in the lighting database corresponding to each supplementary light source; Each target luminaire and its corresponding configuration parameters are used as configuration information, and scene lighting synchronization is performed based on the configuration information.

2. The method according to claim 1, characterized in that, The real-world scenario includes at least one real-world reference object; The acquisition of real-world lighting information corresponding to the real scene includes: When real lighting fixtures are set in the real scene, the sub-lighting information of the reference object position where each real reference object is located is detected, and the sub-lighting information is combined to obtain the real lighting information of the real scene. When no real lighting fixtures are set in the real scene, the following processing is performed on each of the real reference objects in the real scene: The real reference object is photographed and processed according to different exposure times to obtain multiple first low dynamic range images corresponding to the real reference object; A first high dynamic range image is synthesized based on each of the first low dynamic range images, wherein the first high dynamic range image carries sub-illumination information of the reference object position corresponding to the real reference object, and the pixel bit depth of the first high dynamic range image is higher than that of the first low dynamic range image. The sub-illumination information of each of the first high dynamic range images is combined to obtain the real illumination information of the real scene.

3. The method according to claim 2, characterized in that, The virtual scene includes at least one virtual reference object, wherein the position of each virtual reference object in the virtual scene corresponds one-to-one with the position of each real reference object in the real scene. Before obtaining the virtual lighting information corresponding to the virtual scene, the method further includes: For each of the virtual reference objects, the following process is performed: the real position of the real camera used to capture the real reference object in the real scene is obtained, wherein the relative position of the real reference object in the real scene is the same as the relative position of the virtual reference object in the virtual scene; The virtual position corresponding to the real position in the virtual scene is obtained, and a virtual camera is set at the virtual position, wherein the virtual camera is used to capture a second low dynamic range image corresponding to the virtual reference object.

4. The method according to claim 3, characterized in that, The process of obtaining virtual lighting information corresponding to the virtual scene includes: Perform the following processing on each of the virtual reference objects in the virtual scene: By using a virtual camera corresponding to the virtual reference object, the virtual reference object is photographed and processed according to different exposure times to obtain multiple second low dynamic range images corresponding to the virtual reference object; A second high dynamic range image is synthesized based on each of the second low dynamic range images, wherein the second high dynamic range image carries sub-virtual illumination information of the reference object position corresponding to the virtual reference object, and the pixel bit depth of the second high dynamic range image is higher than that of the second low dynamic range image. The virtual lighting information of the virtual scene is obtained by taking each of the sub-virtual lighting information.

5. The method according to claim 1, characterized in that, The real lighting information is carried by a first high dynamic range image of the real scene; the virtual lighting information is carried by a second high dynamic range image of the virtual scene. The step of comparing and calculating the virtual lighting information with the real lighting information to obtain lighting difference information includes: Image comparison processing is performed on the first high dynamic range image and the second high dynamic range image to obtain the difference high dynamic range image; The illumination information carried by the high dynamic range image with the difference is used as illumination difference information.

6. The method according to claim 1, characterized in that, The illumination difference information includes at least one of the following: illumination difference direction, illumination difference intensity, illumination difference color, and illumination difference color temperature relative to the reference object. The step of determining the position of each supplementary light source corresponding to the real light fixture based on the first position of each real light fixture in the real scene includes: Based on the first lamp position of each real lamp in the real scene and the differential illumination direction, determine the real lamp position of each supplementary light source; The process of processing each supplementary light source based on the illumination difference information and the virtual light fixture position to obtain configuration parameters for at least one supplementary light source includes: Obtain a first distance between the actual lamp position and the reference object position, wherein the reference object position is the position of a real reference object in the real scene, and the direction from the actual lamp position to the reference object position is the target direction; The luminous intensity of the supplementary light source is obtained by performing light attenuation calculation based on the difference in light intensity, the target direction, and the first distance. Based on the difference in illumination color corresponding to each difference in illumination direction and the target direction, determine the light source color of the supplementary light source in the target direction; Based on the color temperature of the differential illumination corresponding to each differential illumination direction and the target direction, determine the light color temperature of the supplementary light source in the target direction; The configuration parameters of the supplementary light source are obtained by combining the target direction, the light color temperature, the light source color, the luminous intensity, and the virtual light fixture position.

7. The method according to claim 6, characterized in that, The step of determining the actual lamp position of each supplementary light source based on the first lamp position of each actual lamp in the real scene and the differential illumination direction includes: When no virtual lights are set in the virtual scene, the first light position of each real light in the real scene is taken as the real light position corresponding to each supplementary light source. When virtual lights are set in the virtual scene, the first light fixture position of each real light fixture in the real scene and the reference position of the real reference object are obtained. The direction of each first light fixture position toward the reference object position is taken as the real illumination direction. The real illumination direction parallel to the differential illumination direction is taken as the target direction. The first light fixture position in the target direction is taken as the real light fixture position of the supplementary light source.

8. The method according to claim 6, characterized in that, The step of calculating light attenuation based on the differential illumination intensity, the target direction, and the first distance to obtain the luminous intensity of the supplementary light source includes: Based on the difference in illumination intensity and the target direction, determine the first illumination intensity of the supplementary light source in the target direction for the reference object position; The square of the first distance is obtained, and the square is multiplied by the first illumination intensity to obtain the luminous intensity of the supplementary light source.

9. The method according to claim 1, characterized in that, The illumination difference information includes: differential illumination direction, differential illumination intensity, differential illumination color, and differential illumination color temperature for the reference object. The method further includes: When the real scene needs supplemental lighting, the virtual lamp position of each supplemental light source is determined based on the second lamp position of each virtual lamp in the virtual scene and the differential lighting direction, and each virtual lamp position is mapped to the real lamp position in the real scene; For each of the supplementary light sources, the following processing is performed: Obtain a second distance between the virtual lamp position and the reference object position, wherein the reference object position is the position of a virtual reference object in the virtual scene, and the direction from the virtual lamp position to the reference object position is the target direction; The luminous intensity of the supplementary light source is obtained by performing light attenuation calculation based on the difference in illumination intensity, the target direction, and the second distance. When no real lighting fixtures are set in the real scene, obtain the color temperature and light source color of the virtual lighting fixtures at the location of the virtual lighting fixtures; When real lighting fixtures are already set in the real scene, the light source color of the supplementary light source is determined based on the difference in illumination color and the target direction, and the light color temperature of the supplementary light source is determined based on the difference in illumination color temperature. The configuration parameters of the supplementary light source are obtained by combining the target direction, the light color temperature, the light source color, the luminous intensity, and the actual lamp position.

10. The method according to claim 9, characterized in that, The step of determining the virtual lamp position of each supplementary light source based on the second lamp position of each virtual lamp in the virtual scene and the differential illumination direction includes: When no real lighting fixtures are set in the real scene, the second lighting fixture position of each virtual lighting fixture in the virtual scene is taken as the virtual lighting fixture position corresponding to each supplementary light source. When real lighting fixtures are already set in the real scene, the direction of each second lighting fixture position toward the reference object position is taken as the virtual lighting direction, the virtual lighting direction parallel to the differential lighting direction is taken as the target direction, and the second lighting fixture position in the target direction is taken as the virtual lighting fixture position of the supplementary light source.

11. The method according to claim 9, characterized in that, The step of calculating light attenuation based on the differential illumination intensity, the target direction, and the second distance to obtain the luminous intensity of the supplementary light source includes: Based on the difference in illumination intensity and the target direction, determine the second illumination intensity of the supplementary light source in the target direction for the reference object position; Obtain the square of the second distance and multiply the square by the second illumination intensity to obtain the luminous intensity of the supplementary light source.

12. The method according to claim 1, characterized in that, The process of matching the configuration parameters of each supplementary light source with the luminaire database to obtain the target luminaire corresponding to each supplementary light source in the luminaire database includes: Obtain the performance parameters of the lamps in the lamp database, wherein the performance parameters of the lamps include: the value range of different types of illumination parameters; The following processing is performed for the configuration parameters of each of the supplementary light sources: Obtain different types of lighting parameters from the configuration parameters, wherein the lighting parameters include: lighting intensity, lighting color, and lighting color temperature; The illumination parameters of each type are matched with the performance parameters of different lamps, and the lamps that meet the matching conditions are used as the target lamps corresponding to the supplementary light source. The matching conditions include: the illumination parameters of each type of the supplementary light source belong to the value range of the illumination parameters corresponding to the lamp.

13. The method according to claim 1, characterized in that, The scene lighting synchronization based on the configuration information includes: When the virtual scene needs additional lighting, corresponding virtual lights are set in the virtual scene based on the configuration information, and parameters corresponding to each virtual light are configured. When the real scene needs supplemental lighting, a lighting control signal for the real light fixture corresponding to each target light fixture is generated based on the configuration information, and each lighting control signal is sent to each real light fixture in the real scene. The lighting control signal is used to set the configuration parameters of the real light fixture.

14. The method according to claim 1, characterized in that, Before obtaining the real lighting information corresponding to the real scene, the method further includes: Obtain environmental parameters from the real-world scene, wherein the environmental parameters include the dimensions of the real-world scene; A virtual scene is constructed based on the environmental parameters of the real scene, wherein the virtual scene corresponds one-to-one with the real scene.

15. A light illumination information processing device, characterized in that, The device includes: The lighting acquisition module is configured to acquire real lighting information corresponding to the real scene. The lighting acquisition module is also configured to acquire virtual lighting information corresponding to the virtual scene; The difference acquisition module is configured to compare and calculate the virtual lighting information with the real lighting information to obtain lighting difference information. The light source configuration module is configured to, when the virtual scene needs supplemental lighting, determine the real lamp position corresponding to each supplemental light source based on the first lamp position of each real lamp in the real scene, and map each real lamp position to a virtual lamp position in the virtual scene; process each supplemental light source based on the lighting difference information and the virtual lamp position to obtain configuration parameters for at least one supplemental light source, wherein the supplemental light source is used for supplemental lighting; The light source configuration module is further configured to perform matching processing with the lamp database based on the configuration parameters of each supplementary light source to obtain the target lamp in the lamp database corresponding to each supplementary light source; The illumination synchronization module is configured to use each target lamp and its corresponding configuration parameters as configuration information, and to perform scene illumination synchronization based on the configuration information.

16. The apparatus according to claim 15, characterized in that, The real-world scenario includes at least one real-world reference object; The illumination acquisition module is further configured to detect the sub-illumination information of the reference object position where each real reference object is located when real lighting fixtures are set in the real scene, and combine each sub-illumination information to obtain the real illumination information of the real scene. When no real lighting fixtures are set in the real scene, the following processing is performed on each real reference object in the real scene: the real reference object is photographed according to different exposure times to obtain multiple first low dynamic range images corresponding to the real reference object; a first high dynamic range image is synthesized based on each first low dynamic range image, wherein the first high dynamic range image carries sub-illumination information of the reference object position corresponding to the real reference object, and the pixel bit depth of the first high dynamic range image is higher than that of the first low dynamic range image; the sub-illumination information of each first high dynamic range image is combined to obtain the real illumination information of the real scene.

17. The apparatus according to claim 16, characterized in that, The virtual scene includes at least one virtual reference object, wherein the position of each virtual reference object in the virtual scene corresponds one-to-one with the position of each real reference object in the real scene; The illumination acquisition module is further configured to perform the following processing for each virtual reference object before acquiring the virtual illumination information corresponding to the virtual scene: acquiring the real position of the real camera used to capture the real reference object in the real scene, wherein the relative position of the real reference object in the real scene is the same as the relative position of the virtual reference object in the virtual scene; acquiring the virtual position corresponding to the real position in the virtual scene, and setting a virtual camera at the virtual position, wherein the virtual camera is used to capture the second low dynamic range image corresponding to the virtual reference object.

18. The apparatus according to claim 17, characterized in that, The illumination acquisition module is further configured to perform the following processing on each virtual reference object in the virtual scene: using a virtual camera corresponding to the virtual reference object, the virtual reference object is photographed according to different exposure times to obtain multiple second low dynamic range images corresponding to the virtual reference object; a second high dynamic range image is synthesized based on each second low dynamic range image, wherein the second high dynamic range image carries sub-virtual illumination information of the reference object position corresponding to the virtual reference object, and the pixel bit depth of the second high dynamic range image is higher than the pixel bit depth of the second low dynamic range image; and the virtual illumination information of the virtual scene is obtained by combining each sub-virtual illumination information.

19. The apparatus according to claim 15, characterized in that, Real lighting information is carried by the first high dynamic range image of the real scene; virtual lighting information is carried by the second high dynamic range image of the virtual scene. The difference acquisition module is further configured to perform image comparison processing on the first high dynamic range image and the second high dynamic range image to obtain a difference high dynamic range image; and to use the illumination information carried by the difference high dynamic range image as illumination difference information.

20. The apparatus according to claim 15, characterized in that, The illumination difference information includes at least one of the following: difference in illumination direction, difference in illumination intensity, difference in illumination color, and difference in illumination color temperature relative to the reference object; The light source configuration module is further configured to determine the actual lamp position of each supplementary light source based on the first lamp position of each actual lamp in the real scene and the differential illumination direction; Obtain a first distance between the actual lamp position and the reference object position, wherein the reference object position is the position of a real reference object in the real scene, and the direction from the actual lamp position to the reference object position is the target direction; The luminous intensity of the supplementary light source is obtained by performing light attenuation calculation based on the differential illumination intensity, the target direction, and the first distance; the light source color of the supplementary light source in the target direction is determined based on the differential illumination color corresponding to each differential illumination direction and the target direction; the light color temperature of the supplementary light source in the target direction is determined based on the differential illumination color temperature corresponding to each differential illumination direction and the target direction; and the configuration parameters of the supplementary light source are obtained by combining the target direction, the light color temperature, the light source color, the luminous intensity, and the virtual lamp position.

21. The apparatus according to claim 20, characterized in that, The light source configuration module is further configured to, when no virtual lights are set in the virtual scene, use the first light position of each real light in the real scene as the real light position corresponding to each supplementary light source. When virtual lights are set in the virtual scene, the first light fixture position of each real light fixture in the real scene and the reference position of the real reference object are obtained. The direction of each first light fixture position toward the reference object position is taken as the real illumination direction. The real illumination direction parallel to the differential illumination direction is taken as the target direction. The first light fixture position in the target direction is taken as the real light fixture position of the supplementary light source.

22. The apparatus according to claim 20, characterized in that, The light source configuration module is further configured to determine, based on the differential illumination intensity and the target direction, a first illumination intensity of a supplementary light source for the reference object position in the target direction; obtain the square of the first distance, and multiply the square by the first illumination intensity to obtain the luminous intensity of the supplementary light source.

23. The apparatus according to claim 15, characterized in that, The illumination difference information includes: the difference in illumination direction, difference in illumination intensity, difference in illumination color, and difference in illumination color temperature relative to the reference object; The light source configuration module is further configured to, when the real scene needs supplemental lighting, determine the virtual lamp position of each supplemental light source based on the second lamp position of each virtual lamp in the virtual scene and the differential lighting direction, and map each virtual lamp position to the real lamp position in the real scene; perform the following processing for each supplemental light source: obtain the second distance between the virtual lamp position and the reference object position, wherein the reference object position is the position of a virtual reference object in the virtual scene, and the direction of the virtual lamp position toward the reference object position is the target direction; based on the differential lighting intensity The light attenuation calculation is performed on the degree, the target direction, and the second distance to obtain the luminous intensity of the supplementary light source; when no real light fixtures are set in the real scene, the light color temperature and light source color of the virtual light fixture at the virtual light fixture position are obtained; when real light fixtures are set in the real scene, the light source color of the supplementary light source is determined based on the differential illumination color and the target direction, and the light color temperature of the supplementary light source is determined based on the differential illumination color temperature; the target direction, the light color temperature, the light source color, the luminous intensity, and the real light fixture position are combined to obtain the configuration parameters of the supplementary light source.

24. The apparatus according to claim 23, characterized in that, The light source configuration module is further configured to, when no real lights are set in the real scene, use the second light position of each virtual light in the virtual scene as the virtual light position corresponding to each supplementary light source. When real lighting fixtures are already set in the real scene, the direction of each second lighting fixture position toward the reference object position is taken as the virtual lighting direction, the virtual lighting direction parallel to the differential lighting direction is taken as the target direction, and the second lighting fixture position in the target direction is taken as the virtual lighting fixture position of the supplementary light source.

25. The apparatus according to claim 23, characterized in that, The light source configuration module is further configured to determine, based on the differential illumination intensity and the target direction, a second illumination intensity of the supplementary light source relative to the reference object position in the target direction; obtain the square of the second distance, and multiply the square by the second illumination intensity to obtain the luminous intensity of the supplementary light source.

26. The apparatus according to claim 15, characterized in that, The light source configuration module is further configured to obtain the performance parameters of the lamps in the lamp database, wherein the performance parameters of the lamps include: value ranges of different types of illumination parameters; and to perform the following processing for the configuration parameters of each supplementary light source: obtain different types of illumination parameters in the configuration parameters, wherein the illumination parameters include: illumination intensity, illumination color, and illumination color temperature; match each type of illumination parameter with the performance parameters of different lamps, and select the lamps that meet the matching conditions as the target lamps corresponding to the supplementary light source; wherein the matching conditions include: each type of illumination parameter of the supplementary light source belongs to the value range of the illumination parameter corresponding to the lamp.

27. The apparatus according to claim 15, characterized in that, The lighting synchronization module is further configured to, when the virtual scene needs supplemental lighting, set corresponding virtual lamps in the virtual scene based on the configuration information, and configure the parameters corresponding to each virtual lamp; When the real scene needs supplemental lighting, a lighting control signal for the real light fixture corresponding to each target light fixture is generated based on the configuration information, and each lighting control signal is sent to each real light fixture in the real scene. The lighting control signal is used to set the configuration parameters of the real light fixture.

28. The apparatus according to claim 15, characterized in that, The illumination synchronization module is further configured to acquire environmental parameters in the real scene, wherein the environmental parameters include the size of the real scene; and to construct a virtual scene based on the environmental parameters of the real scene, wherein the virtual scene corresponds one-to-one with the real scene.

29. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the method for processing illumination information as described in any one of claims 1 to 14.

30. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the method for processing illumination information according to any one of claims 1 to 14.

31. A computer program product, comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the method for processing illumination information as described in any one of claims 1 to 14 is implemented.

Images