Shooting illumination matching method and system based on intelligent light field

Abstract

The invention provides a shooting illumination matching method and system based on an intelligent light field, and the method comprises the steps: obtaining first multimedia information, determining illumination information from the first multimedia information, and adjusting the light-emitting parameters of a light-emitting device according to the illumination information; after the light-emitting equipment is controlled to emit light according to the light-emitting parameters, shooting processing is carried out to obtain second multimedia information; the first multimedia information and the second multimedia information are compared, and under the condition that the difference between the first multimedia information and the second multimedia information meets a preset condition, light-emitting parameters of the light-emitting equipment are stored. According to the method and the device, the problem caused by manual processing for restoring the illumination condition of the shooting scene in the related technology is solved, so that the illumination condition restoration efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of optics, and more specifically, to a method and system for intelligent light field lighting matching of physical models. Background Art

[0002] In today's film, television and animation industries, the use of physical models in pre-production and shooting is common and relatively mature. It is mainly reflected in two methods: direct application and indirect application. Direct application is to directly shoot the physical model and shoot the characters in front of the green screen, and then synthesize the characters and the scene in the later stage; indirect application is to scan the physical model into digital form and use the digital model to complete subsequent work.

[0003] After filming is complete, additional shots may be needed, for example, if a shot was missed or an error occurred during filming. In these cases, additional shots are required.

[0004] Supplementary shooting requires that the scene be consistent with the original shooting scene, and also requires that the lighting conditions during shooting be basically consistent with the original lighting conditions. This can ensure better post-processing and prevent the audience from seeing any traces of supplementary shooting.

[0005] The construction of the shooting scene is relatively easy, but the lighting of the shooting scene requires very experienced lighting technicians to debug the lighting according to the previously shot film and television animation. This is not only time-consuming and labor-intensive, but may also cause lighting mismatches, thereby increasing the difficulty of post-processing. Summary of the Invention

[0006] The embodiments of the present application provide a method and system for lighting a physical model for shooting a smart light field, which at least solves the problem of manually processing the lighting conditions of the shooting scene in the related art.

[0007] According to one aspect of the present application, a shooting illumination matching method based on an intelligent light field is provided, including: obtaining first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, and a video; determining illumination information from the first multimedia information, wherein the illumination information serves as a basis for controlling the illumination of a light-emitting device; adjusting the illumination parameters of the light-emitting device according to the illumination information; after controlling the illumination of the light-emitting device according to the illumination parameters, performing shooting processing to obtain second multimedia information; comparing the first multimedia information and the second multimedia information, and saving the illumination parameters of the light-emitting device if the difference between the first multimedia information and the second multimedia information meets a predetermined condition.

[0008] Furthermore, determining the illumination information from the first multimedia information includes: identifying an object model from the first multimedia information; and determining the illumination information according to ambient illumination of the object model.

[0009] Furthermore, determining the illumination information from the first multimedia information includes: identifying the overall ambient illumination of the first multimedia information from the entire first multimedia information; and determining the illumination information according to the overall ambient illumination.

[0010] Furthermore, determining the lighting information from the first multimedia information includes: reading the ambient lighting from the first multimedia information to generate a high dynamic range image; mapping the pixel coordinates in the high dynamic range image to the spherical light position coordinates, and converting the color coordinates of the pixels or pixel blocks in the high dynamic range image into the color and brightness of the light.

[0011] Furthermore, after controlling the light-emitting device to emit light according to the light-emitting parameters, performing shooting processing to obtain the second multimedia information includes: when there are multiple groups of light-emitting parameters, using each group of light-emitting parameters to control the light-emitting device and shooting to obtain the second multimedia information corresponding to the group of light-emitting parameters.

[0012] According to another aspect of the present application, a shooting lighting matching system based on an intelligent light field is also provided, including: an acquisition module, which acquires first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, and a video; a determination module, which determines lighting information from the first multimedia information, wherein the lighting information is used as a basis for controlling the lighting of a light-emitting device; an adjustment module, which is used to adjust the lighting parameters of the light-emitting device according to the lighting information; a shooting module, which is used to control the lighting of the light-emitting device according to the lighting parameters, and then perform shooting processing to obtain second multimedia information; a saving module, which is used to compare the first multimedia information and the second multimedia information, and save the lighting parameters of the light-emitting device if the difference between the first multimedia information and the second multimedia information meets predetermined conditions.

[0013] Furthermore, the determination module is configured to: identify an object model from the first multimedia information; and determine the illumination information according to ambient illumination of the object model.

[0014] Furthermore, the determining module is configured to: identify the overall ambient lighting of the first multimedia information from the entire first multimedia information; and determine the lighting information according to the overall ambient lighting.

[0015] Furthermore, the determination module is used to: read the ambient lighting from the first multimedia information to generate a high dynamic range image; map the pixel coordinates in the high dynamic range image to the spherical light position coordinates, and convert the color coordinates of the pixels or pixel blocks in the high dynamic range image into the color and brightness of the light.

[0016] Furthermore, the shooting module is used for: when there are multiple groups of lighting parameters, using each group of lighting parameters to control the lighting device and shoot to obtain the second multimedia information corresponding to the group of lighting parameters.

[0017] In an embodiment of the present application, a method is adopted to obtain first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, or a video; determine lighting information from the first multimedia information, wherein the lighting information serves as a basis for controlling the lighting of a light-emitting device; adjust the lighting parameters of the light-emitting device according to the lighting information; after controlling the lighting of the light-emitting device according to the lighting parameters, perform shooting processing to obtain second multimedia information; compare the first multimedia information with the second multimedia information, and save the lighting parameters of the light-emitting device if the difference between the first multimedia information and the second multimedia information meets a predetermined condition. This application solves the problem of manually processing the lighting conditions of the shooting scene in the related art, thereby improving the efficiency and accuracy of restoring the lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. The drawings that form part of this application are used to provide a further understanding of the application. The illustrative embodiments of the application and their descriptions are used to explain the application and do not constitute an improper limitation on the application. In the drawings:

[0019] Figure 1 A flowchart of a shooting illumination matching method based on an intelligent light field provided in an embodiment of the present application;

[0020] Figure 2 This is a flow chart of a lighting processing method for photographing a physical model based on an intelligent light field according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that, in the absence of conflicts, the embodiments and features in the embodiments of the present application may be combined with each other. It should be noted that the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in an order different from that shown.

[0022] First, the technical terms involved in the following embodiments are explained.

[0023] Lux is a unit of luminance. Definition: When the luminous flux received by a uniformly illuminated object per square meter is 1 lumen, the illuminance is 1 lux. Applications: Suitable illuminances for reading and sewing are approximately 60 to 100 lux. Also known as lux, its legal symbol is lx. Lux, as a unit of illuminance, is widely used in fields such as photometry and ecology, and is an indispensable physical quantity in scientific research and daily life.

[0024] High Dynamic Range Imaging (HDRI or HDR), in computer graphics and cinematography, is a set of techniques used to achieve a greater dynamic range of exposure (i.e., greater differences between light and dark) than conventional digital imaging techniques. The goal of high dynamic range imaging is to accurately represent the wide range of real-world brightness, from direct sunlight to the darkest shadows.

[0025] The following embodiments also involve LEDs. LED, short for light-emitting diode, is a commonly used light-emitting device that emits light through the recombination of electrons and holes. It is widely used in the lighting field. LEDs efficiently convert electrical energy into light energy and have a wide range of applications in modern society, such as lighting, flat-panel displays, and medical devices. It should be noted that while the following embodiments use LEDs as an example, other light-emitting devices or equipment can achieve the same effects or functions, and will not be discussed in detail here.

[0026] Reference Figure 1 The embodiment provides a shooting illumination matching method based on intelligent light field, Figure 1 A flow chart of a method for shooting illumination matching based on intelligent light field is provided in an embodiment of the present disclosure, as shown in FIG. Figure 1 As shown, the method includes the following steps:

[0027] In the first step, first multimedia information is obtained. The first multimedia information includes at least one of the following: an image, a virtual 3D scene, or a video. In this step, the first multimedia is already captured and includes multimedia information. For example, images, videos, etc. are captured from the original captured image. Here, the information obtained from the first multimedia includes lighting information that needs to be restored. In one example, the first multimedia information is a video that primarily depicts a scene in the dark, illuminated by candlelight. In this case, the lighting information of the candlelight scene needs to be restored to reshoot some of the scenes.

[0028] In the second step, lighting information is determined from the first multimedia information, where the lighting information serves as a basis for controlling the lighting of the light-emitting device. It should be noted that the first multimedia information may have various scenarios. For example, the first multimedia information may be a picture of a tall building. In this case, a model of the tall building may be constructed, and then the lighting information corresponding to the tall building model may be determined based on the lighting and shadow conditions of the tall building in the picture. In another example, the first multimedia information may be a video of farmers working in a field at dusk. After constructing a physical setting such as the field, the lighting is controlled to achieve the overall lighting and shadow conditions of dusk in the video. The lighting information determined in this case is the overall lighting information for the scene. In this step, to achieve both scenarios described in the above examples, determining the lighting information from the first multimedia information includes: identifying an object model from the first multimedia information and determining the lighting information based on the ambient lighting of the object model; or identifying the overall ambient lighting of the first multimedia information from the entire first multimedia information and determining the lighting information based on the overall ambient lighting. The object model here refers to the actual object in the first multimedia information. The actual object can be a fixed object such as a building, or a moving object. Because a model is used to simulate the actual object in the first multimedia information during the re-shooting, the actual object in the first multimedia information is also referred to as the object model (also referred to as the target object or target object).

[0029] As an optional embodiment, the step of determining the lighting information from the first multimedia information can be combined with the two situations of the above two examples. The step includes: identifying one or more main objects from the first multimedia information, and the process of re-shooting the main object is implemented using a model, that is, the main object is the object model (also called the target object or target object), determining the lighting information of each main object respectively, and determining one or more lighting parameters corresponding to each main object for the lighting information of each main object, deintersecting the lighting parameters corresponding to all main objects to obtain a first lighting parameter, and the first lighting parameter is one or more; identifying the overall ambient lighting of the first multimedia information from the entire first multimedia information; determining the lighting information based on the overall ambient lighting, and determining the second lighting parameter based on the lighting information, and the second lighting parameter is one or more; taking the intersection of the first lighting parameter and the second lighting parameter to obtain a third lighting parameter, and the third lighting parameter is used as the basis for adjusting the lighting parameters of the lighting device in the third step.

[0030] In the third step, the lighting parameters of the light-emitting device are adjusted based on the lighting information (if the lighting parameters are determined, the lighting parameters are adjusted based on the lighting parameters); the lighting parameters here will be used to transmit the light to emit light in the future. Therefore, when the lighting information is determined, the ambient lighting is read from the first multimedia information to generate a high dynamic range image; the pixel coordinates in the high dynamic range image are mapped to the spherical light position coordinates, and the color coordinates of the pixels or pixel blocks in the high dynamic range image are converted into the color and brightness of the light.

[0031] Obtain a first normal vector of the object model in the first multimedia information, and obtain first illumination information of the highlight in the first multimedia information; perform a preset arrangement calculation on the contrast in the first multimedia information to generate illumination information, and generate luminescence parameters based on the first illumination information and the illumination information; in this way, the ambient illumination of the object model in the intelligent light field can be determined based on the first normal vector and the luminescence parameters.

[0032] In an optional embodiment, there are generally multiple lights, which can be arranged according to a dome, or can also be arranged on the spherical surface, so that various lighting information can be achieved.

[0033] For example, a lighting system comprising multiple lights may be employed as follows: this lighting system is referred to as a panoramic photography light box and includes a box body, fill lights, a stage, and a drive mechanism. The box body is provided with a storage cavity, one side wall of which is provided with an opening connected to the storage cavity, the opening being used for placing and taking items and for filming by an external camera. The fill lights are provided on multiple inner walls of the 360-degree panoramic photography light box and are arranged facing the storage cavity. The stage is provided within the storage cavity and is rotatably connected to the box body. The output shaft of the drive mechanism is connected to the stage and drives the stage to rotate. The box body has a front wall, a rear wall, a top wall, a bottom wall, a left wall, and a right wall. The opening is provided on the front wall, and fill lights are provided on the inner sides of the rear wall, the top wall, the bottom wall, the left wall, and the right wall.

[0034] The 360-degree panoramic photography light box also includes a diffuser plate, which is located on the inner wall of the light box and corresponds to the fill light. The diffuser plate is located on the side of the fill light facing the accommodating cavity and covers the fill light. The 360-degree panoramic photography light box also includes an upper light panel, which is located at the opening of the box body and connected to the front wall and the top wall. The upper light panel is angled with the top wall and faces the loading platform. The 360-degree panoramic photography light box may also include left and right light panels. The bottom of the box body is provided with left and right sliding mechanisms. The left sliding mechanism is connected to the left light panel, and the right sliding mechanism is connected to the right light panel. The left light panel can slide relative to the bottom wall via the left sliding mechanism, and the right light panel can slide relative to the bottom wall via the right sliding mechanism.

[0035] In this step, the light field can be reconstructed in the following manner: obtain the three-dimensional coordinates of the object point on the target object in the camera coordinate system; based on the structural model of the focusing light field camera, perform ray tracing on each object point, reconstruct the original light field data image, and calculate the distance from the focused image point of the current object point in the main lens image to the main surface of the main lens; calculate the distance from the focused image point of the current object point in the main lens image to the microlens array surface based on the distance; calculate the image coordinate system coordinates of the imaging point of the current object point under each microlens in the microlens array surface based on the distance, the coordinates of the focused image point of the main lens and the center coordinates of the microlens; based on the image coordinate system coordinates of the imaging point of the current object point under each microlens in the microlens array surface, determine whether the current object point can be imaged under each microlens; and count the number of all microlenses that image the current object point. , and calculate the pixel value of the current object point under each microlens that images it; calculate the diffraction-limited spot size; compare the diffraction-limited spot size with the pixel size of the focusing light field camera; if the diffraction-limited spot size is larger than the pixel size of the focusing light field camera, the imaging point of the current object point on the CCD imaging area corresponds to more than one CCD pixel, and the image coordinate system coordinates of the imaging point of the current object point under each microlens that images it are used as the center of each CCD pixel of the current object point in the CCD imaging area, and generate the pixel block corresponding to the current object point; otherwise, the imaging point of the current object point on the CCD imaging area only corresponds to one CCD pixel on the CCD imaging area, and single pixel rendering is sufficient; perform the above operations on all object points to reconstruct the original light field data image.

[0036] In the fourth step, after controlling the light-emitting device to emit light according to the light-emitting parameters, shooting is performed to obtain second multimedia information.

[0037] In the fifth step, the first multimedia information and the second multimedia information are compared, and if the difference between the first multimedia information and the second multimedia information meets the predetermined condition, the luminous parameters of the luminous device are saved. In this step, as an optional embodiment, if the difference between the first multimedia information and the second multimedia information does not meet the predetermined condition, the brightness, contrast, and / or hue of the first multimedia information and the second multimedia information are compared, and the brightness, contrast, and / or hue of the second multimedia information are adjusted. After the adjustment, the first multimedia information and the second multimedia information are compared again, and if the difference between the first multimedia information and the second multimedia information meets the predetermined condition, the luminous parameters of the luminous device are saved.

[0038] The above steps solve the problem of manual processing of restoring the lighting conditions of the shooting scene in the related art, thereby improving the efficiency and accuracy of restoring the lighting conditions.

[0039] In the above steps, the first multimedia information is associated with the lighting parameters of the lighting device and then saved. The associated lighting parameters are identified as the lighting parameters of the first multimedia information. If there are multiple second multimedia information, each of the multiple second multimedia information is compared with the first multimedia information, and the second multimedia information that is closest to the ambient lighting in the first multimedia information is saved, and the lighting parameters corresponding to the second multimedia information that is closest are saved.

[0040] As another optional implementation, the object model can be generated as a software model, which also stores the aforementioned lighting information and can be used by other users. The software model stores the lighting information and luminous parameters, which can also be used as the luminous parameters of a virtual light in the software system, thereby simulating the second multimedia information in the software.

[0041] The above method steps are described below with reference to an example.

[0042] The main purpose of this example is to solve the real-time lighting problem during the preview and shooting of physical models. It mainly serves the film, television, animation, game and other industries, and is used in the fields of film, television, animation, game pre-production, film, television, animation shooting and production.

[0043] This example involves the service objects of intelligent light field technology, the current status of physical model preview and shooting, and the storage and digital conversion of physical models. These are explained below.

[0044] Intelligent light field technology constructs a camera array to take multi-angle, high-precision photos, which not only restores the three-dimensional structure of the photographed person, but also obtains the reflection information of the face, so that it can reconstruct the lighting effect of the face model under different ambient light. In fact, whether it is the purpose of research and development or the service object, intelligent light field technology is mainly centered around people. Through a controllable light field photography scanning system, people are scanned in all directions to construct 3D virtual characters (digital people). At the same time, because the scanning is performed in a space with a 360° variable and controllable light source, the scanned digital people take into account both accuracy and restoration while introducing the dimension of real light sources. Compared with traditional AI face-changing, the digital people scanned by intelligent light field technology have higher precision facial skin texture and texture, and the reflection of light on the face can match the changes in various ambient light.

[0045] In the film, television, and animation industries, the use of physical models in pre-production and filming is common and relatively mature, primarily in two ways: direct and indirect. Direct application involves photographing the physical model and shooting the characters in front of a green screen, then later compositing the characters with the scene. Indirect application involves scanning the physical model into a digital format and using the digital model for subsequent work. However, current pre-production and filming methods still have shortcomings, with the issue of image light sources being a key concern. Currently, both direct and indirect application of physical miniature models in pre-production and filming suffer from drawbacks regarding light sources. Direct application primarily involves the unification of light sources for characters and scenes, while indirect application primarily involves the need for computer simulation of light source information.

[0046] Because physical models occupy real-world space, preservation and disposal have long been a challenge. Many used physical models often find themselves in an awkward dilemma. Specifically, if they are preserved, the physical model occupies physical space, gathering dust and requiring regular cleaning; if they are destroyed, there's the concern that it might be used again in the future. Digital preservation and archiving of physical models offers a solution.

[0047] In this example, leveraging the technological advantages of intelligent light field technology for scanning figures, the light field dimension was incorporated into the physical model during the initial scanning phase. This results in a more faithful reproduction of the lighting and shadow effects than previous digital models produced purely in post-production, and can more realistically simulate the lighting and shadow atmosphere, including the sun, moon, and the setting sun. In this example, the physical model was digitized and a digital asset library was created. Firstly, a digital asset library facilitates the preservation and application of the physical model. Secondly, the digital model in the digital asset library retains the texture of the physical model while also providing complete light field information, opening up a wider range of possibilities for later applications.

[0048] The light field acquisition of a physical model can be divided into four steps:

[0049] Step 11: Test the light field acquisition system and debug it to a light field environment that meets the requirements for scanning and acquiring physical models.

[0050] In step 12, the physical model is placed in a light field acquisition system for scanning and acquisition, and a 360° light source system is used to simulate the state of the physical model under different light field conditions.

[0051] In step 13, the physical model with light field information scanned by the light field acquisition system is rendered, and the digital model is saved by a computer.

[0052] Step 14: The physical model has completed digital transformation, the digital model in the computer is archived, and a digital model digital resource library is established.

[0053] The collected models in the digital asset library are further used in the pre-production and shooting processes.

[0054] In the above steps, 1) any ambient lighting is simulated through the intelligent light field system: first, the ambient lighting is read from the image or virtual 3D scene to generate an HDRI image; then the pixel coordinates are mapped to the spherical light position coordinates, and the color coordinates of the pixels or pixel blocks are converted into the color and brightness of the light, thereby realizing the ambient lighting simulation; 2) the object is reconstructed by scanning the camera array on the spherical surface: first, 90 (18 cameras are installed on the horizontal circumference at a 20° center angle interval and 5 columns are distributed vertically, for a total of 90. This method is the optimal distribution scheme obtained after multiple test results) 4K resolution cameras are arranged on the sphere to shoot the center area of ​​the sphere; then images of the object under test are collected under different lighting conditions; and then 3D reconstruction is performed to form an asset library of the same object under different lighting conditions.

[0055] The following optional performance indicators are provided: 1) Controllable illumination range at the center of the sphere: 0-200,000 lux; 2) Controllable color temperature range: 3000K-8000K; 3) Controllable LED density on the spherical surface: 733 LEDs / square meter;

[0056] 4) Reconstruction accuracy: 2mm for 3D point cloud and 0.1mm for texture; 5) Lighting richness of a single asset: includes at least 20 types of scene lighting.

[0057] First, the physical miniature models scanned by the light field acquisition system retain the texture of the original model while also introducing real-time lighting control, solving many pain points in the film and television industry. For example, many films previously chose to directly construct scaled-down miniature models and use them to simulate real-world scenes for filming. The main reason was that digital models produced solely in post-production lacked realism and were difficult to translate to the big screen. However, this approach has the potential for inconsistent lighting effects between the characters and the scene. Because the characters and miniature models are of different scales and are filmed separately, lighting during filming often relies on the staff's experience, which is often artificial and empirical, without rigorous technical support. In this example, the physical model was already captured in a 360° light field. When filming the characters, a light meter was used to record the real-time lighting conditions on-site. This recorded lighting information was then imported into the digital model, ensuring that the digital model generated the same lighting environment as the actual filming. This technical intervention rigorously controlled the lighting uniformity between the characters and the model. This example provides new possibilities for the pre-production phase: based on the physical miniature model, it is digitally converted through the light field acquisition system to serve the pre-production phase. The real-time variable lighting effect is more intuitive and controllable than the previous method. Technically, the scanning method of the light field acquisition system (compared to the traditional scanning acquisition method) greatly expands the practicality and functionality. Although the scanned digital model cannot completely replace the physical model, the introduction of the early light field dimension makes its application scenarios more diverse. This proposal provides a new idea for the digital preservation of physical models and lays an important foundation for the establishment of a digital asset library for models.

[0058] In this example, the following technical points can be used: 1. High illumination - wide dynamic range: The hardware system's instantaneous maximum illumination exceeds 200,000 Lux (the outdoor sunlight illumination at noon is 60,000 to 100,000 lux). It is the brightest light field system in the world and can realize all-day and all-weather scene simulation. 2. High density - fine sampling: 73,332 independently controlled LEDs (757*97) can reproduce more details. Nearly one hundred 4K cameras can capture the details of the object without blind spots. 3. Physical lighting - more realistic: The dimension of the light field is introduced in the early stage of scanning the physical model, and the light and shadow effects are more restored than the previous purely post-production digital models.

[0059] Through this example, the light field acquisition system facilitates the digital conversion of physical models, enabling a more realistic and objective capture of their true appearance, thus further facilitating the establishment of a digital asset library for these models. The inclusion of light field factors during the scanning process makes the digital models scanned by the light field acquisition system more functional and comprehensive. In particular, in industries such as film, television, animation, and gaming, digital models scanned by the light field acquisition system provide a more realistic, immersive, and immersive lighting atmosphere, improving efficiency while achieving lighting effects previously difficult to achieve through post-production alone.

[0060] After saving the above-mentioned luminous parameters, object model, lighting information and other data, they can be called up and used from the software system at any time when needed.

[0061] In this embodiment, a method for photographing and processing lighting for a physical model based on an intelligent light field is also provided. Figure 2 This is a flow chart of a lighting processing method for photographing a physical model based on an intelligent light field according to an embodiment of the present application. Figure 2 As shown, the steps included in the method are described below.

[0062] Step S202: Acquire lighting parameters for controlling a lighting device, wherein the lighting device includes a plurality of lighting entities at different locations, each lighting entity being independently controllable by the lighting parameters; the lighting parameters are determined based on ambient lighting in pre-provided first multimedia information;

[0063] Step S204: sending the lighting parameters of the lighting device to the lighting device; wherein the degree to which the ambient lighting generated by the lighting device after emitting light according to the lighting parameters is close to the ambient lighting in the first multimedia information satisfies a predetermined condition;

[0064] Step S206 , after the light-emitting device emits light according to the light-emitting parameters, controlling the shooting device to shoot.

[0065] Through the above steps, it is no longer necessary for the lighting technician to repeatedly input different lighting parameters for testing. This solves the problem of low efficiency caused by the lighting technician's constant debugging when adjusting the lighting according to multimedia information in the related art, and improves the efficiency of lighting adjustment during shooting.

[0066] There are many ways to determine lighting information. For example, the lighting parameters are determined based on the ambient lighting in the first multimedia information provided in advance, including: determining the lighting information from the first multimedia information, wherein the lighting information is used as a basis for controlling the lighting of the lighting device; adjusting the lighting parameters of the lighting device according to the lighting information; after controlling the lighting device to emit light according to the lighting parameters, performing shooting processing to obtain second multimedia information; comparing the first multimedia information and the second multimedia information, and saving the lighting parameters of the lighting device if the difference between the first multimedia information and the second multimedia information meets predetermined conditions.

[0067] Optionally, determining the lighting information from the first multimedia information includes: identifying an object model from the first multimedia information, and determining the lighting information based on the ambient lighting of the object model; or, identifying the overall ambient lighting of the first multimedia information from the entire first multimedia information, and determining the lighting information based on the overall ambient lighting.

[0068] There are many ways to identify an object model. For example, identifying an object model from the first multimedia information includes: obtaining multiple sets of training data, wherein each set of training data includes input data and output data, the input data is media information, and the output data is the object model included in the media information; determining whether the number of the multiple sets of training data exceeds a threshold, wherein the threshold is pre-configured; if the threshold is exceeded, using the multiple sets of training data to train a neural network under a supervised machine learning training method; inputting the first media information into the neural network, and the neural network outputs the object model.

[0069] Optionally, after obtaining the neural network, training data is continuously acquired, and when the continuously acquired training data exceeds a second threshold, incremental training is performed on the neural network using the continuously acquired training data.

[0070] Through the above steps, it is no longer necessary for the lighting technician to repeatedly input different lighting parameters for testing. This solves the problem of low efficiency caused by the lighting technician's constant debugging when adjusting the lighting according to multimedia information in the related art, and improves the efficiency of lighting adjustment during shooting.

[0071] In this embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0072] The above program can be run in the processor, or it can be stored in the memory (or computer-readable medium), which includes permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0073] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0074] This embodiment provides such a device or system. The system or device is used to implement the functions of the method in the above embodiment. Each module in the system or device corresponds to each step in the method, and has been described in the method and will not be repeated here.

[0075] In this embodiment, a shooting lighting matching system based on an intelligent light field is provided, which is characterized by including: an acquisition module for acquiring first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, and a video; a determination module for determining lighting information from the first multimedia information, wherein the lighting information is used as a basis for controlling the lighting of a light-emitting device; an adjustment module for adjusting the lighting parameters of the light-emitting device according to the lighting information; a shooting module for controlling the lighting of the light-emitting device according to the lighting parameters, and then performing shooting processing to obtain second multimedia information; and a saving module for comparing the first multimedia information and the second multimedia information, and saving the lighting parameters of the light-emitting device if the difference between the first multimedia information and the second multimedia information meets predetermined conditions.

[0076] Optionally, the determination module is configured to: identify an object model from the first multimedia information; and determine the lighting information according to ambient lighting of the object model.

[0077] Optionally, the determination module is configured to: identify overall ambient lighting of the first multimedia information from the entire first multimedia information; and determine the lighting information according to the overall ambient lighting.

[0078] Optionally, the determination module is used to: read the ambient lighting from the first multimedia information to generate a high dynamic range image; map the pixel coordinates in the high dynamic range image to the spherical light position coordinates, and convert the color coordinates of the pixels or pixel blocks in the high dynamic range image into the color and brightness of the light.

[0079] Optionally, the shooting module is used to: when there are multiple groups of lighting parameters, use each group of lighting parameters to control the lighting device and shoot to obtain the second multimedia information corresponding to the group of lighting parameters.

[0080] In this embodiment, a physical model shooting lighting processing system based on an intelligent light field is also provided, including: a second acquisition module, used to obtain light-emitting parameters for controlling a light-emitting device, wherein the light-emitting device includes multiple light-emitting entities at different positions, and each light-emitting entity can be independently controlled by the light-emitting parameters; the light-emitting parameters are determined based on the ambient lighting in a pre-provided first multimedia information; a sending module, used to send the light-emitting parameters of the light-emitting device to the light-emitting device; wherein the degree to which the ambient lighting generated by the light-emitting device after emitting light according to the light-emitting parameters is close to the ambient lighting in the first multimedia information satisfies a predetermined condition; and a control module, used to control the shooting device to shoot after the light-emitting device emits light according to the light-emitting parameters.

[0081] Optionally, the second acquisition module is used to: determine lighting information from the first multimedia information, wherein the lighting information serves as a basis for controlling the lighting of the light-emitting device; adjust the lighting parameters of the light-emitting device according to the lighting information; after controlling the lighting of the light-emitting device according to the lighting parameters, perform shooting processing to obtain second multimedia information; compare the first multimedia information and the second multimedia information, and save the lighting parameters of the light-emitting device if the difference between the first multimedia information and the second multimedia information meets predetermined conditions.

[0082] Optionally, the second acquisition module is used to: identify an object model from the first multimedia information, and determine the lighting information based on the ambient lighting of the object model; or, identify the overall ambient lighting of the first multimedia information from the entire first multimedia information, and determine the lighting information based on the overall ambient lighting.

[0083] Optionally, the second acquisition module is used to: obtain multiple sets of training data, wherein each set of training data includes input data and output data, the input data is media information, and the output data is the object model included in the media information; determine whether the number of the multiple sets of training data exceeds a threshold, wherein the threshold is pre-configured; if the threshold is exceeded, use the multiple sets of training data to train a neural network under a supervised machine learning training method; input the first media information into the neural network, and the neural network outputs the object model.

[0084] Optionally, the second acquisition module is used to: after obtaining the neural network, continue to acquire training data, and when the training data continued to be acquired exceeds a second threshold, use the training data continued to be acquired to perform incremental training on the neural network.

[0085] The above embodiments solve the problem of manual processing of restoring the lighting conditions of the shooting scene in the related art, thereby improving the efficiency and accuracy of restoring the lighting conditions.

[0086] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A shooting illumination matching method based on intelligent light field, characterized in that: include: Acquire first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, and a video; Determining illumination information from the first multimedia information, wherein the illumination information is used as a basis for controlling the illumination of the light emitting device; adjusting the light-emitting parameters of the light-emitting device according to the light information; After controlling the light-emitting device to emit light according to the light-emitting parameter, performing shooting processing to obtain second multimedia information; The first multimedia information and the second multimedia information are compared, and when a difference between the first multimedia information and the second multimedia information meets a predetermined condition, the light-emitting parameters of the light-emitting device are saved.

2. The method according to claim 1, characterized in that Determining illumination information from the first multimedia information includes: identifying an object model from the first multimedia information; The illumination information is determined according to the ambient illumination of the object model.

3. The method according to claim 1, characterized in that Determining illumination information from the first multimedia information includes: identifying the overall ambient lighting of the first multimedia information from the entire first multimedia information; The illumination information is determined according to the overall ambient illumination.

4. The method according to claim 2 or 3, characterized in that: Determining illumination information from the first multimedia information includes: The ambient lighting is read from the first multimedia information to generate a high dynamic range image; the pixel coordinates in the high dynamic range image are mapped to the spherical light position coordinates, and the color coordinates of the pixels or pixel blocks in the high dynamic range image are converted into the color and brightness of the light.

5. The method according to claim 4, characterized in that After the light-emitting device is controlled to emit light according to the light-emitting parameter, the second multimedia information is obtained by performing shooting processing, including: In the case that there are multiple groups of light-emitting parameters, each group of light-emitting parameters is used to control the light-emitting device and perform photography to obtain the second multimedia information corresponding to the group of light-emitting parameters.

6. A shooting illumination matching system based on intelligent light field, characterized in that: include: An acquisition module is configured to acquire first multimedia information, wherein the first multimedia information includes at least one of the following: an image, a virtual 3D scene, and a video; A determination module, which determines illumination information from the first multimedia information, wherein the illumination information is used as a basis for controlling the illumination of the light-emitting device; An adjustment module, used for adjusting the light-emitting parameters of the light-emitting device according to the light information; A shooting module, configured to control the light-emitting device to emit light according to the light-emitting parameters, and then perform shooting processing to obtain second multimedia information; The saving module is used to compare the first multimedia information with the second multimedia information, and save the light-emitting parameters of the light-emitting device when the difference between the first multimedia information and the second multimedia information meets a predetermined condition.

7. The system according to claim 6, characterized in that The determination module is used for: identifying an object model from the first multimedia information; The illumination information is determined according to the ambient illumination of the object model.

8. The system according to claim 6, characterized in that The determination module is used for: identifying the overall ambient lighting of the first multimedia information from the entire first multimedia information; The illumination information is determined according to the overall ambient illumination.

9. The system according to claim 7 or 8, characterized in that: The determination module is used for: The ambient lighting is read from the first multimedia information to generate a high dynamic range image; the pixel coordinates in the high dynamic range image are mapped to the spherical light position coordinates, and the color coordinates of the pixels or pixel blocks in the high dynamic range image are converted into the color and brightness of the light.

10. The system according to claim 9, characterized in that The shooting module is used for: In the case that there are multiple groups of light-emitting parameters, each group of light-emitting parameters is used to control the light-emitting device and perform photography to obtain the second multimedia information corresponding to the group of light-emitting parameters.

Images