Intelligent lamp field control method and system based on monochromatic illumination reachability decomposition

Abstract

The invention discloses an intelligent lamp field control method and system based on monochromatic illumination reachability decomposition. Comprising the following steps: firstly, rendering according to object position information to obtain an illumination reachability graph set of a lamp field in a digital twin virtual scene and a basic contribution graph set of the lamp field in a real scene; thirdly, performing illumination decomposition on the target illumination distribution of the digital twin virtual scene to obtain a direction weight matrix and an ambient light residual error corresponding to each intelligent lamp in a lamp field; and finally, according to the direction weight matrix corresponding to each intelligent lamp in the lamp field and the ambient light residual error, parameters of each intelligent lamp in the lamp field are optimized, an optimal lamp field parameter space is obtained, and the lamp field in a real scene is controlled based on the optimal lamp field parameter space. According to the method, automatic, accurate and real-time mapping from virtual scene illumination to entity lamp parameters is realized, and the technical problems of low efficiency, insufficient precision and incapability of dynamic adjustment of a traditional light distribution method are solved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control technology, specifically relating to an intelligent lighting field control method and system based on monochromatic illumination reachability decomposition. Background Technology

[0002] In film and television virtual production, extended reality (XR) production, and virtual production workflows, achieving seamless integration of real actors and virtual scenes presents a core technological challenge: ensuring consistent lighting. Currently, the following fundamental technical bottlenecks exist:

[0003] (1) Lack of quantitative standards for illumination matching

[0004] Traditional lighting work relies entirely on the experience and subjective judgment of the lighting technician. The technician must observe the composite image on a monitor and manually adjust the parameters of each physical light through repeated "preview-adjust-preview" cycles. This process lacks objective, quantitative evaluation standards, and specifically suffers from the following problems:

[0005] ① Matching accuracy cannot be guaranteed: The human eye has physiological limitations in perceiving differences in illumination, and subtle but important illumination features are difficult to capture accurately.

[0006] ② Poor repeatability and consistency: The lighting results for the same scene varied significantly between different lighting technicians and at different times.

[0007] ③ Difficulties in quality control: The lack of quantifiable evaluation indicators makes it difficult to conduct objective quality control and acceptance.

[0008] (2) Technical gaps in reverse rendering

[0009] In traditional virtual production, the standard practice for obtaining scene lighting information is to place a virtual reflector sphere at the target point (usually the actor's position) and render an HDR (High Dynamic Range) panoramic image. This method results in mixed lighting with the following fundamental flaws:

[0010] ① Information unresolvability: The result obtained is a mixture of contributions from all light sources, and it is impossible to distinguish the independent contributions of individual light sources.

[0011] ② Missing structural information: It only contains pixel-level brightness and color values ​​and lacks structural information about the source of light.

[0012] ③ Weak optimization guidance: It cannot provide effective directional guidance for subsequent parameter optimization.

[0013] ④ Material coupling: Lighting information is tightly coupled with scene material properties, and changing the material requires re-analysis.

[0014] Technically speaking, this method transforms the target lighting effects of a virtual scene into optimal parameter settings for physical lights, a classic example of inverse rendering. Inverse rendering is a high-dimensional, non-linear, and non-convex constraint optimization problem. Current technologies offer no systematic solution to this problem, leaving lighting work still largely reliant on trial and error.

[0015] (3) Engineering challenges in optimizing efficiency

[0016] Even if the above optimization problem can be solved in theory, its practical implementation faces serious efficiency issues, for the following reasons:

[0017] ① Excessive rendering time per session: High-quality panoramic HDR rendering typically takes several minutes to several hours.

[0018] ② Parameter space dimension explosion: A medium-sized lighting field may have 100-1000 lights, each light has 5-10 adjustable parameters, and the total parameter dimension reaches 500-10000 dimensions.

[0019] ③ Lack of convergence guarantee: When searching in such a high-dimensional space, traditional methods cannot guarantee convergence to a satisfactory solution.

[0020] ④ Real-time requirements cannot be met: For moving shots, real-time lighting adjustment is required, which traditional methods cannot achieve at all.

[0021] (4) Non-intelligent resource utilization

[0022] Modern virtual film studios are typically equipped with dozens or even hundreds of intelligent lights, but current usage methods have the following problems:

[0023] ① Low equipment utilization: Due to the complexity of operation, only a small portion of the equipment is usually used on site.

[0024] ② Lack of energy consumption optimization: lack of intelligent energy-saving mechanisms.

[0025] ③Complex configuration management: Each light requires individual configuration and monitoring.

[0026] Therefore, it is necessary to propose a more efficient and optimized intelligent lighting control method. Summary of the Invention

[0027] To address the problems existing in the background technology, this invention provides an intelligent lighting field control method and system based on monochromatic illumination reachability decomposition. The method proposed in this invention achieves automated, precise, and real-time mapping from virtual scene illumination to physical lamp parameters, solving the technical problems of low efficiency, insufficient accuracy, and inability to dynamically adjust traditional lighting methods.

[0028] The technical solution adopted in this invention is:

[0029] I. A Smart Lighting Field Control Method Based on Monochromatic Illumination Accessibility Decomposition

[0030] Step 1: Construct a digital twin virtual scene;

[0031] Step 2: Render the light field in the digital twin virtual scene and the basic contribution map in the real scene based on the object position information;

[0032] Step 3: Based on the set of light accessibility maps and the set of basic contribution maps, perform light decomposition on the target light distribution of the digital twin virtual scene to obtain the direction weight matrix and ambient light residual corresponding to each smart light in the light field.

[0033] Step 4: Based on the directional weight matrix and ambient light residual of each smart light in the lighting field, optimize the parameters of each smart light in the lighting field to obtain the optimal lighting field parameter space, and control the lighting field in the real scene based on the optimal lighting field parameter space.

[0034] The control method further includes the following steps:

[0035] Step 5: Obtain the latest object position information, repeat steps 2-4, continuously optimize the lighting field parameter space and control the lighting field in the real scene.

[0036] In step 2, the set of illumination accessibility maps of the lighting field in the digital twin virtual scene includes the monochrome illumination accessibility map corresponding to each smart light. The monochrome illumination accessibility map is used to represent the geometric accessibility of each smart light.

[0037] In step 2, the set of basic contribution maps of the lighting field in the real scene includes the basic contribution map corresponding to each smart lamp. The basic contribution map is used to represent the optical response of the smart lamp to the real scene.

[0038] Step 3 specifically involves:

[0039] An optimization problem for illumination decomposition is constructed. Based on the set of illumination reachability maps, the set of basic contribution maps, and the target illumination distribution of the digital twin virtual scene, the optimization problem for illumination decomposition is solved to obtain the direction weight matrix and ambient light residual corresponding to each smart light in the lighting field. The optimization problem for illumination decomposition satisfies the following formula:

[0040]

[0041] Where W represents the decision variable of the optimization problem; Target lighting distribution for a digital twin virtual scene; For the i-th smart light in The weight of direction; Basic contribution graph, This is a monochromatic illumination reachability graph; λ1 represents the first regularization coefficient; λ2 represents the second regularization coefficient; Represents the spatial gradient of the lamp weights; Represents the square of the L2 norm; Represents the L1 norm; ⊙ denotes the square of the vector's magnitude, and ⊙ denotes element-wise multiplication.

[0042] Step 4 involves optimizing the parameters of each smart light in the lighting field based on the direction weight matrix and ambient light residual corresponding to each smart light, to obtain the optimal lighting field parameter space, including:

[0043] An initial light field parameter space is generated. Based on the current light field parameter space, the orientation weight matrix corresponding to each smart light in the light field, and the ambient light residual, scene image rendering is performed to obtain a predicted rendering image. The loss function between the predicted rendering image and the target image is calculated. Based on the loss function, the parameters of each smart light in the light field parameter space are adjusted until the loss function meets the preset conditions, thus obtaining the optimal light field parameter space.

[0044] The predicted rendered image satisfies the following formula:

[0045]

[0046]

[0047] in, Indicates based on the current lighting field parameter space The predicted rendered image; This indicates the direction of the i-th smart lamp obtained from the light decomposition. Contribution weight; This indicates that the i-th smart light has parameters p i Directional contributions; It is ambient light residual; Represents the space of lighting field parameters; Indicates the intensity adjustment factor; Indicates the color adjustment transformation factor; The standard parameters representing the smart light i; Basic contribution graph Accessibility map with light Element-wise product.

[0048] The loss function between the predicted rendered image and the target image satisfies the following formula:

[0049]

[0050]

[0051] in, This represents the total loss function value; This represents the data fitting loss; This represents the sparsity regularization loss. This represents the smoothness regularization loss; Indicates physical constraint loss; , , These are the third to fifth regularization coefficients, used to balance the weights of each item.

[0052] II. An intelligent lighting control system based on monochromatic illumination accessibility decomposition

[0053] The object position acquisition unit is used to acquire object position information;

[0054] The lighting accessibility map generation unit is used to render and obtain a set of lighting accessibility maps of the light field in the digital twin virtual scene based on the object position information;

[0055] The basic contribution map generation unit is used to render a set of basic contribution maps in a real scene based on object position information;

[0056] The first optimization unit is used to perform illumination decomposition on the target illumination distribution of the digital twin virtual scene based on the illumination reachability map set and the basic contribution map set, and to obtain the direction weight matrix and ambient light residual corresponding to each smart lamp in the lighting field.

[0057] The second optimization unit is used to optimize the parameters of each smart lamp in the lighting field based on the directional weight matrix and ambient light residual corresponding to each smart lamp in the lighting field, so as to obtain the optimal lighting field parameter space.

[0058] The control unit is used to control the lighting field in the real scene based on the current optimal lighting field parameter space.

[0059] III. A computer device

[0060] The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the intelligent lighting field control method based on monochromatic illumination accessibility decomposition.

[0061] The beneficial effects of this invention are:

[0062] 1. Compared with the manual experience-based debugging of existing methods, this invention pre-calculates the illumination reachability map and the basic contribution map, realizing the extraction of independent and structured contribution information of each lamp from mixed illumination. This transforms illumination matching from a black box process that relies on subjective experience into an optimizable problem based on quantitative data, realizing the transformation from being driven by manual experience to being driven by algorithm optimization.

[0063] 2. This invention proposes an illumination decomposition method, which decomposes the target illumination distribution into the directional weight matrix of each smart lamp and the ambient light residual, providing a clear and directional target for subsequent lamp parameter optimization (such as "which lamps need to contribute how much in which directions"), realizing the transformation from blind trial and error of "reasoning from results" to white-box optimization of "target-guided adjustment".

[0064] 3. Compared to traditional inverse rendering optimization that requires full ray tracing, the fully differentiable forward rendering model built based on pre-rendered data and decomposition results only requires efficient matrix operations and weighted summation, avoiding time-consuming ray intersection, shading, and global illumination calculations. This increases the speed of a single evaluation by more than a thousand times, thereby supporting the use of efficient gradient descent algorithms. It can complete the optimization of high-dimensional parameter spaces (hundreds to thousands of dimensions) within minutes, achieving a breakthrough in computational efficiency from infeasible to real-time / near real-time. Attached Figure Description

[0065] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0066] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] like Figure 1 As shown, the intelligent lighting field control method based on monochromatic illumination reachability decomposition proposed in this invention includes the following steps:

[0068] Step 1: Construct a digital twin virtual scene. A digital twin virtual scene is a precise 3D digital model of the real lighting field and the photographed object, including geometric models, material properties, texture maps, and the model, location, orientation, and adjustable parameter range of all smart lights.

[0069] Step 2: Based on the object position information, render the lighting accessibility map set of the lighting field in the digital twin virtual scene and the basic contribution map set in the real scene. The object position information is the three-dimensional spatial coordinates of the target (such as actors and props) that needs to be precisely illuminated in the real scene.

[0070] In one feasible implementation, the set of illumination accessibility maps of the light field in the digital twin virtual scene includes a monochromatic illumination accessibility map corresponding to each smart light, and the monochromatic illumination accessibility map is used to represent the geometric accessibility of each smart light.

[0071] Each smart light is located at the spherical probe position. Monochromatic light accessibility map Satisfy the following formula:

[0072]

[0073]

[0074] in, These are spherical coordinates, representing the direction originating from the center of the probe. and These are the azimuth and pitch angles, respectively, corresponding to a pixel position on the panoramic image (lighting map); The probe position To smart lights The distance; It is a distance decay function; From direction To smart lights The visibility function; It's a smart light. The beam shape function; This represents the soft shadow state coefficient, which is a value within the range (0,1).

[0075] In a monochromatic system, for coordinates The pixel at that location determines whether it is a smart light. Illuminate, simplified to:

[0076]

[0077] in, coordinates The pixel state at that location, Indicates that the smart light illuminate, Indicates not to be smart light illuminate.

[0078] That is, by using a threshold The accessibility of a lamp can be determined by comparison:

[0079]

[0080] in, It is a globally unified brightness transition threshold, meaning that all smart lights use the same threshold, eliminating the need for adjustments for different light types. It can be computed in parallel on the GPU, avoiding ambiguity and uncertainty in color matching.

[0081] In realistic lighting, shadow boundaries are typically not sharp, but rather consist of a penumbra. Monochromatic lighting accessibility maps naturally support soft shadow processing; in fully illuminated areas, In completely shaded areas, In the penumbra transition region, ,Right now The value represents the level of illumination. By setting an appropriate threshold... It allows for flexible control over the softness or hardness of shadows. For example, =0.5 indicates a medium hardness shading; =0.1 indicates a softer shadow, containing more penumbra area; =0.9 indicates a harder shadow, and only fully illuminated areas are recognized. It is usually taken as 0.1-0.2.

[0082] The generation of the monochromatic lighting accessibility map employs special rendering settings to ensure the purity and stability of the output results. Specifically, material neutralization: all surface materials are set to neutral gray with a diffuse reflectance of 0.5; lighting purification: lights are set to pure white with intensity normalization of 1.0; environment simplification: global illumination, ambient light, and indirect lighting are turned off; rendering optimization: the maximum bounce count is set to 1, and only direct lighting is calculated.

[0083] The monochromatic light reachability map generated by this invention has the following important characteristics:

[0084] 1. Geometric Information Purity: Contains only geometric information about light propagation (position, direction, occlusion), completely excluding optical influences such as material color, texture, and reflection properties. Geometric accessibility is relatively stable and does not change with material variations.

[0085] 2. Numerical stability: Pixel values ​​are strictly within the range of [0,1], where 0 indicates completely unreachable, 1 indicates completely reachable, and intermediate values ​​indicate partially reachable (such as penumbra regions).

[0086] 3. Consistency Guarantee: Under the same geometric conditions, the reachability map remains unchanged regardless of changes in material.

[0087] 4. Computational efficiency: Since the monochromatic illumination reachability map is a single-channel image, the amount of data to be calculated is small and the processing speed is fast.

[0088] 5. Determine simplicity: Accessibility can be determined by comparing simple thresholds.

[0089] In one feasible implementation, the set of basic contribution maps of the lighting field in the real scene includes the basic contribution map corresponding to each smart lamp, and the basic contribution map is used to represent the optical response of the smart lamp to the real scene.

[0090] Basic contribution diagram of each smart light Satisfy the following formula:

[0091]

[0092] in, It's a smart light. Standard parameters. S full It is a complete virtual scene model that includes real materials, textures, and all lighting attributes. It is a complete lighting rendering system, including direct lighting, indirect lighting, and global illumination.

[0093] Optionally, smart lights The standard parameters include:

[0094] Set the intensity to 1.0, the color to pure white (1.0, 1.0, 1.0), the color temperature to 5600K (daylight standard), and other parameters (such as beam angle and focus) to their default values.

[0095] The basic contribution map proposed in this invention records the independent illumination contribution of each lamp in a real material scene under standard parameter settings.

[0096] Monochromatic illumination reachability maps are used to provide geometric priors, i.e., to indicate smart lights. Which areas might be affected? The baseline contribution map provides an optical benchmark, indicating the performance of the smart light under standard parameters. The actual contribution effect. This invention separates geometric and optical effects through monochromatic illumination accessibility maps and fundamental contribution maps, providing clear guidance for parameter optimization. Table 1 illustrates the essential differences in concept and application between the monochromatic illumination accessibility map and the fundamental contribution map proposed in this invention.

[0097] Table 1. Essential differences in concept and application between monochromatic illumination accessibility maps and basic contribution maps.

[0098]

[0099] This invention separates the geometrical reachability information of illumination from the optical response information of materials. Through this separation process, a stable, reusable, and analyzable illumination representation can be established.

[0100] Step 3: Based on the set of light accessibility maps and the set of basic contribution maps, perform light decomposition on the target light distribution of the given digital twin virtual scene. The goal of light decomposition is to solve for the contribution weight of each smart light in the light field in each direction, obtaining the direction weight matrix and ambient light residual for each smart light in the light field; each element in the direction weight matrix represents the weight of different smart lights in different directions, such as... Indicates smart light In direction Contribution weight, ambient light residual This indicates residual illumination that cannot be attributed to any controllable light source. These two results provide a complete "map" for subsequent lamp parameter optimization, where W indicates the optimization objective (which lamps need to achieve what intensity in which areas); It indicates optimization limits (ambient light baseline; optimization cannot fall below this level).

[0101] Step 3 specifically involves:

[0102] The goal of illumination decomposition is to determine the contribution weight of each smart light in each direction. .

[0103] Mathematically, this can be formalized as a constrained optimization problem:

[0104]

[0105] In the above formula The term represents the smart light after considering geometric reachability. The effective contribution potential under standard parameters, i.e. the actual contribution after considering accessibility. The weight represents the actual adjustment ratio required relative to the standard parameter. At that time, the expression requires the full contribution under standard parameters; When this is the case, it means that half of the contribution is required; When the summation term is not needed, it indicates that the contribution of this lamp is not required. This represents the total contribution prediction after all lights have been adjusted. This represents the Frobenius norm.

[0106] The following constraints need to be satisfied during the solution process:

[0107] Nonnegativity constraint: The contribution of light cannot be negative.

[0108] Reachability constraint: when No contribution is made where it cannot be reached. . It is the accessibility threshold, usually set to 0.1.

[0109] Normalization constraints: That is, the total contribution of lights in a certain direction does not exceed 100%.

[0110] Sparsity promotion: Encourage Sparse, meaning that only a few lights contribute significantly in most areas.

[0111] An optimization problem for illumination decomposition is constructed. Based on the set of illumination reachability maps, the set of basic contribution maps, and the target illumination distribution of a given digital twin virtual scene, the optimization problem for illumination decomposition is solved to obtain the direction weight matrix and ambient light residual corresponding to each smart light in the light field. The optimization problem for illumination decomposition constructed in this invention satisfies the following formula:

[0112]

[0113] Where W represents the decision variable of the optimization problem, specifically the weight of each smart light; Target lighting distribution for a digital twin virtual scene; For the i-th smart light in The weight of direction; Basic contribution graph, This is a monochromatic illumination reachability graph; λ1 represents the first regularization coefficient; λ2 represents the second regularization coefficient; Represents the spatial gradient of the lamp weights; Represents the square of the L2 norm; Represents the L1 norm; ⊙ denotes the square of the vector's magnitude, and ⊙ denotes element-wise multiplication.

[0114] In one feasible implementation, during the solution process, the coordinates on the spherical image can be determined first. The set of light sources that contribute to the pixels at that location It satisfies the following formula:

[0115]

[0116] In light source set Based on this, we can optimize the solution, which can significantly reduce the search space for subsequent optimizations.

[0117] In one feasible implementation, the optimization problem of illumination decomposition can be decomposed into three easily solvable subproblems, which can be solved iteratively using the ADMM algorithm. Therefore, the computational complexity of the illumination decomposition optimization problem in this invention mainly depends on reachability determination, local optimization, and global optimization. The computational complexity of reachability determination is O(N×P), where N is the number of lights and P is the number of pixels. The computational complexity of local optimization is... , This represents the average number of reachable lights per pixel. The computational complexity of global optimization is O(K×P×N), where K is the number of iterations. Since... Typically very small (a pixel is usually only directly illuminated by a few lights), and the P×N operation is highly parallel, so the entire decomposition process can be completed in seconds.

[0118] Step 4: Based on the directional weight matrix and ambient light residual of each smart light in the lighting field, optimize the parameters of each smart light in the lighting field to obtain the optimal lighting field parameter space, and control the lighting field in the real scene based on the optimal lighting field parameter space.

[0119] In one feasible implementation, the parameters of each smart light in the lighting field are optimized based on the directional weight matrix and ambient light residual corresponding to each smart light in the lighting field to obtain the optimal lighting field parameter space, including:

[0120] An initial light field parameter space is generated. Based on the current light field parameter space, the orientation weight matrix corresponding to each smart light in the light field, and the ambient light residual, scene image rendering is performed to obtain a predicted rendering image. The loss function between the predicted rendering image and the target image is calculated. Based on the loss function, the parameters of each smart light in the light field parameter space are adjusted until the loss function meets the preset conditions (such as the loss function converges) to obtain the optimal light field parameter space.

[0121] The predicted rendered image satisfies the following formula:

[0122]

[0123]

[0124] in, Indicates based on the current lighting field parameter space The predicted rendered image; This indicates the direction of the i-th lamp obtained from the light decomposition. Contribution weight; This indicates that the i-th lamp is in the parameter p i Directional contributions; It is ambient light residual; This represents the lighting field parameter space, which is the set of all lamp parameters. , and These are the parameters for the first intelligent light and the Nth intelligent light, respectively. For medium-sized lighting fields, N is typically 500-10000; Indicates the intensity adjustment factor; Indicates the color adjustment transformation factor; Indicates the standard parameters of lamp i; This represents the pre-rendered standard effective contribution base contribution graph. Accessibility map with monochromatic lighting The element-wise product contains information about the smart light's geometric reachability and baseline optical response.

[0125] In traditional inverse rendering optimization, a set of parameters is evaluated each time. All of these require performing complete rendering calculations. Therefore, a single rendering typically takes several seconds to several minutes. If hundreds or thousands of rendering processes are required, the total time consumption is cumulative. However, the scene image rendering process of this invention completely avoids expensive operations such as ray tracing, shading calculations, and global illumination solutions, and only involves matrix multiplication, color transformation, and weighted summation. The computational complexity is reduced from O (rendering complexity) to O(N×P), achieving an acceleration of more than a thousand times.

[0126] In one feasible implementation, each smart light The controllable parameters constitute a parameter vector. It satisfies the following formula:

[0127]

[0128] in, This is the normalized intensity parameter; For color temperature parameters (Kelvin); For horizontal and pitch rotation angles; This refers to either the focal length or the beam angle parameter; the beam angle parameter defines the size of the illumination cone angle, affecting the range of illumination and edge attenuation characteristics. It may also include other device-specific parameters. T indicates transpose operation.

[0129] Each parameter has physical and technical constraints, specifically including range constraints, equipment limitations, and operational safety. Specifically, the range constraints are: , This represents a component inequality, i.e. Each component satisfies this inequality. This represents the minimum value of the component. This indicates the maximum value of the component. Equipment limitations include maximum power, minimum color temperature, and mechanical movement range. Operational safety includes avoiding excessive brightness, overheating, and mechanical impact.

[0130] In one feasible implementation, the intensity adjustment factor satisfies the following formula:

[0131]

[0132] in, It is the strength value in the standard parameters (usually 1.0). This represents the intensity value among the parameters of smart light i.

[0133] In one feasible implementation, the color adjustment transformation factor is specifically a 3×3 color transformation matrix. The formula is as follows:

[0134] .

[0135] Color transformation matrix is a 3×3 linear transformation matrix that transforms the standard parameters The base color vector for rendering Transformed into target parameters The color vector below ,Right now , These are the red, green, and blue components that contribute to the basic color vector. The target parameters are respectively The red, green, and blue components of the color vector. The color transformation matrix can be decomposed into a color temperature transformation matrix. With RGB gain matrix The product of, and may include a device color response correction matrix. ,Right now In practical applications, the color transformation matrix can be efficiently obtained through a lookup table or a fitted parametric model.

[0136] The goal of optimization is to find the parameters that make the predicted rendered image as close as possible to the target image, while satisfying various constraints and requirements. The loss function between the predicted rendered image and the target image satisfies the following formula:

[0137]

[0138]

[0139] in, This represents the total loss function value; This represents the data fitting loss; This represents the sparsity regularization loss. This represents the smoothness regularization loss; Indicates physical constraint loss; , , These are the third to fifth regularization coefficients, used to balance the weights of each item.

[0140] Specifically, the data fitting loss is used to measure the difference between the predicted image and the target image, and the formula is as follows:

[0141]

[0142] Where P is the total number of pixels. Mean squared error (MSE) is used as a measure of difference due to its favorable mathematical properties (differentiability, convexity, etc.).

[0143] The sparsity regularization loss is used to encourage simple lighting schemes and avoid using too many lights at the same time. The formula is as follows:

[0144]

[0145] The first term in the formula is L1 regularization, which promotes intensity sparsity; the second term is a smooth L0 regularization approximation, which encourages lights to be completely turned off. and It is a parameter that controls the strength of sparsity.

[0146] The smoothness regularization loss is used to ensure that parameter changes between adjacent lamps are gradual, producing a natural lighting transition. The formula is as follows:

[0147]

[0148] in, It is a set of adjacent light pairs, defined based on spatial distance and beam overlap. It is the adjacent weight, reflecting the correlation between lamp i and j; This represents the L2 norm of a vector. This represents the parameter of smart light j, which is adjacent to smart light i.

[0149] Physical constraint loss is used to ensure that parameters are within the allowable range of the equipment, and the formula is as follows:

[0150]

[0151] in, This is the rectified linear unit function. When the parameter is within the allowable range, this term is 0; when the parameter is outside the range, a secondary penalty is incurred. p represents the parameter vector of the i-th lamp. i The kth component (e.g., intensity, color temperature, etc.); and These represent the technically permissible maximum and minimum values ​​for the parameter component, respectively.

[0152] The optimization issues in the scene image rendering process are:

[0153]

[0154] in, It is the feasible set of parameters, defined by the physical constraints of the device.

[0155] Since the forward computation model proposed in this invention is completely differentiable, the gradient of the loss function with respect to the parameters can be calculated automatically through differentiation. :

[0156]

[0157] Gradient calculation is achieved through backpropagation, with the same computational complexity as forward calculation, making it equally efficient.

[0158] Once the gradient is obtained, classic optimization algorithms (such as gradient descent) can be used to optimize and solve the problem, ultimately obtaining the optimal parameters for all the lights.

[0159] Step 5: Obtain the latest object position information, repeat steps 2-4, continuously optimize the lighting field parameter space and control the lighting field in the real scene.

[0160] The intelligent lighting field control system based on monochromatic illumination reachability decomposition proposed in this invention includes:

[0161] The object position acquisition unit is used to acquire object position information;

[0162] The lighting accessibility map generation unit is used to render and obtain a set of lighting accessibility maps of the light field in the digital twin virtual scene based on the object position information;

[0163] The basic contribution map generation unit is used to render a set of basic contribution maps in a real scene based on object position information;

[0164] The first optimization unit is used to perform illumination decomposition on the target illumination distribution of the digital twin virtual scene based on the illumination reachability map set and the basic contribution map set, and to obtain the direction weight matrix and ambient light residual corresponding to each smart lamp in the lighting field.

[0165] The second optimization unit is used to optimize the parameters of each smart lamp in the lighting field based on the directional weight matrix and ambient light residual corresponding to each smart lamp in the lighting field, so as to obtain the optimal lighting field parameter space.

[0166] The control unit is used to control the lighting field in the real scene based on the current optimal lighting field parameter space.

[0167] The present invention proposes a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of an intelligent lighting field control method based on monochromatic illumination reachability decomposition.

Claims

1. A smart lighting field control method based on monochromatic illumination reachability decomposition, characterized in that, Includes the following steps: Step 1: Construct a digital twin virtual scene; Step 2: Render the light field in the digital twin virtual scene and the basic contribution map in the real scene based on the object position information; Step 3: Based on the set of light accessibility maps and the set of basic contribution maps, perform light decomposition on the target light distribution of the digital twin virtual scene to obtain the direction weight matrix and ambient light residual corresponding to each smart light in the light field. Step 4: Based on the directional weight matrix and ambient light residual of each smart light in the lighting field, optimize the parameters of each smart light in the lighting field to obtain the optimal lighting field parameter space, and control the lighting field in the real scene based on the optimal lighting field parameter space.

2. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 1, characterized in that, The control method further includes the following steps: Step 5: Obtain the latest object position information, repeat steps 2-4, continuously optimize the lighting field parameter space and control the lighting field in the real scene.

3. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 1, characterized in that, In step 2, the set of illumination accessibility maps of the lighting field in the digital twin virtual scene includes the monochrome illumination accessibility map corresponding to each smart light. The monochrome illumination accessibility map is used to represent the geometric accessibility of each smart light.

4. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 1, characterized in that, In step 2, the set of basic contribution maps of the lighting field in the real scene includes the basic contribution map corresponding to each smart lamp. The basic contribution map is used to represent the optical response of the smart lamp to the real scene.

5. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 1, characterized in that, Step 3 specifically involves: An optimization problem for illumination decomposition is constructed. Based on the set of illumination reachability maps, the set of basic contribution maps, and the target illumination distribution of the digital twin virtual scene, the optimization problem for illumination decomposition is solved to obtain the direction weight matrix and ambient light residual corresponding to each smart light in the lighting field. The optimization problem for illumination decomposition satisfies the following formula: Where W represents the decision variable of the optimization problem; Target lighting distribution for a digital twin virtual scene; For the i-th smart light in The weight of direction; Basic contribution graph, This is a monochromatic illumination reachability graph; λ1 represents the first regularization coefficient; λ2 represents the second regularization coefficient; Represents the spatial gradient of the lamp weights; Represents the square of the L2 norm; Represents the L1 norm; ⊙ denotes the square of the vector's magnitude, and ⊙ denotes element-wise multiplication.

6. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 1, characterized in that, Step 4 involves optimizing the parameters of each smart light in the lighting field based on the direction weight matrix and ambient light residual corresponding to each smart light, to obtain the optimal lighting field parameter space, including: An initial light field parameter space is generated. Based on the current light field parameter space, the orientation weight matrix corresponding to each smart light in the light field, and the ambient light residual, scene image rendering is performed to obtain a predicted rendering image. The loss function between the predicted rendering image and the target image is calculated. Based on the loss function, the parameters of each smart light in the light field parameter space are adjusted until the loss function meets the preset conditions, thus obtaining the optimal light field parameter space.

7. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 6, characterized in that, The predicted rendered image satisfies the following formula: in, Indicates based on the current lighting field parameter space The predicted rendered image; This indicates the direction of the i-th smart lamp obtained from the light decomposition. Contribution weight; This indicates that the i-th smart light has parameters p i Directional contributions; It is ambient light residual; Represents the space of lighting field parameters; Indicates the intensity adjustment factor; Indicates the color adjustment transformation factor; The standard parameters representing the smart light i; Basic contribution graph Accessibility map with light Element-wise product.

8. The intelligent lighting field control method based on monochromatic illumination reachability decomposition according to claim 6, characterized in that, The loss function between the predicted rendered image and the target image satisfies the following formula: in, This represents the total loss function value; This represents the data fitting loss; This represents the sparsity regularization loss. This represents the smoothness regularization loss; Indicates physical constraint loss; , , These are the third to fifth regularization coefficients.

9. A smart lighting field control system based on monochromatic illumination reachability decomposition, characterized in that, include: The object position acquisition unit is used to acquire object position information; The lighting accessibility map generation unit is used to render and obtain a set of lighting accessibility maps of the light field in the digital twin virtual scene based on the object position information; The basic contribution map generation unit is used to render a set of basic contribution maps in a real scene based on object position information; The first optimization unit is used to perform illumination decomposition on the target illumination distribution of the digital twin virtual scene based on the illumination reachability map set and the basic contribution map set, and to obtain the direction weight matrix and ambient light residual corresponding to each smart lamp in the lighting field. The second optimization unit is used to optimize the parameters of each smart lamp in the lighting field based on the directional weight matrix and ambient light residual corresponding to each smart lamp in the lighting field, so as to obtain the optimal lighting field parameter space. The control unit is used to control the lighting field in the real scene based on the current optimal lighting field parameter space.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the intelligent lighting field control method based on monochromatic illumination reachability decomposition as described in any one of claims 1 to 8.

Images