Illumination probe configuration optimization method and device, storage medium and electronic equipment

Abstract

The invention discloses a configuration optimization method and device of an illumination probe, a storage medium and electronic equipment. The method comprises the steps that at least one illumination probe associated with a virtual grid model and a corresponding first configuration are obtained, and the first configuration is an association relationship between a vertex in the obtained virtual grid model and an illumination probe in the at least one illumination probe by using geometric prior; acquiring illumination baking data of the virtual network model in at least one illumination scene based on at least one illumination probe of the first configuration; acquiring real illumination data of the virtual network model in at least one illumination scene; obtaining a loss function corresponding to the first configuration, the loss function being used for representing an error between the illumination baking data and the real illumination data; and optimizing the first configuration by minimizing the loss function to obtain a second configuration. The technical problem that the configuration accuracy of the illumination probe is low is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a configuration optimization method, device, storage medium, and electronic device for an illumination probe. Background Art

[0002] Light Probes are points that record spatial lighting information. By sampling lighting data, they can approximate the lighting conditions of the surrounding area. In Light Probe configuration scenarios, a common approach is to generate Light Probes within a certain range around each object, based on the distribution of objects in the scene. The position and number of these probes can be adjusted based on the object's shape, size, and lighting variations.

[0003] However, this distribution method relies heavily on empirical judgment, which makes the Light Probe configuration lacking in precision. Furthermore, the lack of effective configuration optimization methods further exacerbates the problem of low Light Probe configuration accuracy. Consequently, the Light Probe configuration accuracy is low.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, storage medium, and electronic device for optimizing the configuration of a light probe, so as to at least solve the technical problem of low configuration accuracy of the light probe.

[0006] According to one aspect of an embodiment of the present application, a method for optimizing the configuration of lighting probes is provided, including: obtaining a first configuration corresponding to at least one lighting probe associated with a virtual grid model, wherein the first configuration is an association relationship between vertices in the virtual grid model and lighting probes in the at least one lighting probe obtained by utilizing geometric priors; obtaining lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration; obtaining real lighting data of the virtual network model in the at least one lighting scene; obtaining a loss function corresponding to the first configuration, wherein the loss function is used to represent the error between the lighting baking data and the real lighting data; and optimizing the first configuration by minimizing the loss function to obtain a second configuration.

[0007] According to another aspect of an embodiment of the present application, a configuration optimization device for a lighting probe is also provided, including: a first acquisition unit, used to obtain a first configuration corresponding to at least one lighting probe associated with a virtual grid model, wherein the above-mentioned first configuration is an association relationship between vertices in the above-mentioned virtual grid model and the lighting probes in the above-mentioned at least one lighting probe obtained by using geometric prior; a second acquisition unit, used to obtain lighting baking data of the above-mentioned virtual network model in at least one lighting scene based on the at least one lighting probe of the above-mentioned first configuration; a third acquisition unit, used to obtain real lighting data of the above-mentioned virtual network model in the above-mentioned at least one lighting scene; a fourth acquisition unit, used to obtain a loss function corresponding to the above-mentioned first configuration, wherein the above-mentioned loss function is used to represent the error between the above-mentioned lighting baking data and the above-mentioned real lighting data; an optimization unit, used to optimize the above-mentioned first configuration by minimizing the above-mentioned loss function to obtain the second configuration.

[0008] As an optional solution, the fourth acquisition unit includes: a quantization module for quantifying the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error; a weighting module for using a directional weight matrix to perform differential weighting on the lighting errors in different directions or positions to obtain the loss function, wherein the directional weight matrix is used to adjust the importance of lighting in each direction in the loss function.

[0009] As an optional solution, the above-mentioned device includes: a first acquisition module, which is used to obtain the spherical harmonic function basis function obtained after convolution of the above-mentioned lighting baking data before the above-mentioned quantification of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error, wherein the above-mentioned spherical harmonic function basis function is used to encode the lighting direction information into mathematical coefficients; a second acquisition module, which is used to obtain the barycentric coordinate matrix before the above-mentioned quantification of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error, wherein the above-mentioned barycentric coordinate matrix is used to represent the lighting interpolation between the above-mentioned vertices; a third acquisition module, which is used to obtain the weight matrix before the above-mentioned quantification of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error, wherein the above-mentioned weight matrix is used to represent the association relationship corresponding to the above-mentioned first configuration, and the above-mentioned weight matrix allows Adjustments are made in the process of minimizing the above-mentioned loss function; a fourth acquisition module is used to obtain the baked spherical harmonic coefficient matrix before the above-mentioned quantization of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error, wherein the above-mentioned spherical harmonic coefficient matrix is used to store the lighting information pre-calculated by the above-mentioned at least one lighting probe; a fifth acquisition module is used to obtain the product of the above-mentioned spherical harmonic function basis function, the above-mentioned barycentric coordinate matrix, the above-mentioned weight matrix, and the above-mentioned spherical harmonic coefficient matrix to obtain predicted lighting data before the above-mentioned quantization of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error; a sixth acquisition module is used to obtain the difference between the above-mentioned predicted lighting data and the above-mentioned real lighting data to obtain the above-mentioned overall deviation before the above-mentioned quantization of the overall deviation between the above-mentioned lighting baking data and the above-mentioned real lighting data to obtain the lighting error.

[0010] As an optional solution, the above-mentioned optimization unit includes: a seventh acquisition module, used to obtain the gradient of the above-mentioned loss function with respect to the above-mentioned weight matrix, wherein the above-mentioned gradient is used to indicate adjustment of the above-mentioned weight matrix to reduce the error between the above-mentioned lighting baking data and the above-mentioned real lighting data; an update module, used to use the above-mentioned gradient to iteratively update the above-mentioned weight matrix until the error between the above-mentioned lighting baking data and the above-mentioned real lighting data meets the convergence condition; a determination module, used to determine the configuration corresponding to the weight matrix when the above-mentioned convergence condition is met as the above-mentioned second configuration.

[0011] As an optional solution, the above-mentioned seventh acquisition module includes: a first acquisition sub-module, used to obtain the product of the above-mentioned spherical harmonic function basis function, the above-mentioned center of gravity coordinate matrix, and the above-mentioned spherical harmonic coefficient matrix to obtain a first value; a second acquisition sub-module, used to obtain the product of the above-mentioned overall deviation degree and twice the above-mentioned direction weight matrix to obtain a second value; a third acquisition sub-module, used to obtain the product of the above-mentioned first value and the above-mentioned second value to obtain the above-mentioned gradient.

[0012] As an optional solution, the above-mentioned device also includes: a fifth acquisition unit, which is used to obtain the first baking data of the above-mentioned virtual network model in the first lighting scene based on the at least one lighting probe of the above-mentioned first configuration in the process of obtaining the lighting baking data of the above-mentioned virtual network model in at least one lighting scene based on the at least one lighting probe of the above-mentioned first configuration, wherein the at least one lighting scene includes the above-mentioned first lighting scene, and the above-mentioned lighting baking data includes the above-mentioned first baking data; the above-mentioned device also includes: a sixth acquisition unit, which is used to obtain the first lighting data of the above-mentioned virtual network model in the above-mentioned first lighting scene in the process of obtaining the real lighting data of the above-mentioned virtual network model in the above-mentioned at least one lighting scene, wherein the above-mentioned real lighting data includes the above-mentioned first lighting data.

[0013] As an optional solution, the at least one lighting scene includes the first lighting scene and the second lighting scene, and the fifth acquisition unit includes: an eighth acquisition module, which is used to acquire the first baking data and the second baking data of the virtual network model in the second lighting scene based on the at least one lighting probe of the first configuration, wherein the lighting baking data includes the second baking data; the sixth acquisition unit includes: a ninth acquisition module, which is used to acquire the first lighting data and the second lighting data of the virtual network model in the second lighting scene, wherein the real lighting data includes the second lighting data; the fourth acquisition unit includes: a tenth acquisition module, which is used to acquire the first error between the first baking data and the first lighting data; an eleventh acquisition module, which is used to acquire the second error between the second baking data and the second lighting data; and an integration module, which is used to integrate the first error and the second error to obtain the loss function.

[0014] As an optional solution, the at least one lighting scene includes a third lighting scene and a fourth scene, and the apparatus further includes: a construction unit for constructing a standard scene before obtaining the lighting baking data of the virtual network model in at least one lighting scene by the at least one lighting probe based on the first configuration, wherein the standard scene has a first lighting condition and a second lighting condition; a simulation unit for simulating a first performance of the virtual grid model in the first lighting condition and a second performance of the virtual grid model in the second lighting condition by rotating the virtual grid model in the standard scene before obtaining the lighting baking data of the virtual network model in at least one lighting scene by the at least one lighting probe based on the first configuration; and a performance unit for determining the first performance as the performance of the virtual grid model in the third lighting scene and the second performance as the performance of the virtual grid model in the fourth lighting scene before obtaining the lighting baking data of the virtual network model in at least one lighting scene by the at least one lighting probe based on the first configuration.

[0015] As an optional solution, the above-mentioned device also includes: a seventh acquisition unit, used to obtain at least two sampling points on the virtual grid model before obtaining the first configuration corresponding to at least one lighting probe associated with the above-mentioned virtual grid model; a configuration unit, used to configure the above-mentioned at least one lighting probe based on the above-mentioned at least two sampling points to obtain the above-mentioned first configuration before obtaining the first configuration corresponding to at least one lighting probe associated with the above-mentioned virtual grid model.

[0016] According to another aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the above-described method for optimizing the configuration of illumination probes.

[0017] According to another aspect of an embodiment of the present application, an electronic device is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned method for optimizing the configuration of the illumination probe through the computer program.

[0018] In an embodiment of the present application, at least one lighting probe associated with the virtual grid model and a corresponding first configuration are obtained, wherein the first configuration is an association relationship between vertices in the virtual grid model and lighting probes in the at least one lighting probe obtained by using geometric priors; based on the at least one lighting probe of the first configuration, lighting baking data of the virtual network model in at least one lighting scene is obtained; real lighting data of the virtual network model in the at least one lighting scene is obtained; a loss function corresponding to the first configuration is obtained, wherein the loss function is used to represent the error between the lighting baking data and the real lighting data; and the first configuration is optimized by minimizing the loss function to obtain a second configuration.

[0019] First, using geometric prior knowledge, we establish a relationship between the vertices of the virtual mesh model and at least one Light Probe, forming a first configuration. This geometry-based configuration more accurately considers the object's shape and spatial distribution, improving the accuracy of the initial configuration.

[0020] Next, based on the association between the Light Probes and the vertices of the virtual mesh model in the first configuration, lighting baking data for the model is obtained under at least one lighting scenario. Light baking is a technique that pre-calculates lighting information and stores it in textures or vertex data. Sampling the Light Probe data from the first configuration can simulate the effects of the model under different lighting scenarios. Because light baking data is generated based on the Light Probe information from the first configuration, its accuracy depends to a certain extent on the quality of the first configuration. Properly configuring Light Probes can yield more accurate light baking data, providing a foundation for subsequent lighting effect evaluation and optimization.

[0021] At the same time, we obtain real-world lighting data for the virtual mesh model under the same lighting scenario. This data, obtained through physical simulation, real-life photography, or other high-precision lighting calculations, reflects the true effect of the object under actual lighting conditions and is an important basis for evaluating the accuracy of light baking data. Real-world lighting data provides a benchmark for evaluating light baking data errors. By comparing the two, we can quantitatively analyze the accuracy of light probe configurations, identify existing problems and deficiencies, and provide guidance for subsequent configuration optimization.

[0022] Finally, a loss function is constructed for the first configuration. This loss function represents the error between the baked lighting data and the real-world lighting data. By minimizing this loss function, the first configuration is optimized to obtain the second configuration. During the optimization process, the loss function is gradually reduced, improving the accuracy of the Light Probe configuration. The loss function provides a clear goal and evaluation criteria for optimizing the Light Probe configuration. By minimizing the loss function, the Light Probe configuration can be automatically adjusted to more closely resemble the real-world lighting effect.

[0023] By using geometric prior knowledge to establish associations, obtain lighting baking data and real lighting data, construct loss functions and optimize configurations, the problem of low lighting probe configuration optimization accuracy is solved from multiple aspects. It not only improves the initial accuracy of the light probe configuration, but also provides an effective configuration optimization method. It can dynamically adjust the configuration according to the actual lighting effect to meet the high-precision requirements of lighting effects in complex scenes, thereby achieving the purpose of dynamically optimizing the initial configuration of the light probe, thereby achieving the technical effect of improving the configuration accuracy of the light probe, and thus solving the technical problem of low configuration accuracy of the light probe. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a schematic diagram of an application environment of an optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a process of an optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of an optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0029] Figure 5 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0030] Figure 6 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0031] Figure 7 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0032] Figure 8 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0033] Figure 9 is a schematic diagram of another optional method for optimizing the configuration of illumination probes according to an embodiment of the present application;

[0034] Figure 10 is a schematic diagram of an optional configuration optimization device for illumination probes according to an embodiment of the present application;

[0035] Figure 11 It is a schematic structural diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] According to one aspect of the embodiment of the present application, a configuration optimization method of an illumination probe is provided. Optionally, as an optional implementation, the configuration optimization method of the illumination probe can be applied to, but is not limited to, Figure 1 In the environment shown, the environment may include, but is not limited to, a user device 102 and a server 112 . The user device 102 may include, but is not limited to, a display 104 , a processor 106 , and a memory 108 . The server 112 includes a database 114 and a processing engine 116 .

[0039] The specific process can be as follows:

[0040] Step S102: the user device 102 obtains at least one lighting probe associated with the virtual grid model and a corresponding first configuration;

[0041] Step S104, sending the first configuration to the server 112 via the network 110;

[0042] In steps S106-S112, the server 112 obtains lighting baking data of the virtual network model in at least one lighting scene based on at least one lighting probe of the first configuration through the processing engine 116; obtains real lighting data of the virtual network model in at least one lighting scene; obtains a loss function corresponding to the first configuration; and optimizes the first configuration by minimizing the loss function to obtain a second configuration.

[0043] Step S114 : sending the second configuration to the user device 102 via the network 110 . The user device 102 displays the second configuration on the display 104 via the processor 106 and stores the second configuration in the memory 108 .

[0044] remove Figure 1 In addition to the examples shown, the above-mentioned user equipment can be a terminal device configured with a target client, which can include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, an MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, etc. The target client can be a video client, an instant messaging client, a browser client, an education client, etc. The above-mentioned network can include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The above-mentioned server can be a single server, or it can be a server cluster composed of multiple servers, or a cloud server. The above is only an example, and no limitation is made to this in this embodiment.

[0045] As an example, terminal devices may include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. This embodiment can be applied to various scenarios, including but not limited to digital humans, virtual humans, games, virtual reality, and extended reality (XR).

[0046] For example, the server can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0047] Alternatively, as an optional implementation, Figure 2 As shown, the configuration optimization method of the illumination probe can be executed by an electronic device, which can be, for example, Figure 1 The user device or server shown in the figure includes the following steps:

[0048] S202: Obtain a first configuration corresponding to at least one Light Probe associated with the virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and Light Probes in the at least one Light Probe obtained using a geometric prior.

[0049] In an optional embodiment, the virtual grid model can be a model constructed in a computer and composed of a large number of grid units for simulating three-dimensional space structures, such as terrain and building models in a game scene.

[0050] In an optional embodiment, the lighting probe may be a point that records spatial lighting information, and by sampling lighting data from the point, the lighting conditions of the surrounding area can be approximately represented.

[0051] In an optional embodiment, the first configuration may be to utilize geometric priors to obtain an association relationship between vertices in the virtual mesh model and lighting probes in at least one lighting probe, which determines how the lighting probe interacts with the virtual mesh model to obtain lighting information.

[0052] Geometric priors refer to prior knowledge about the geometric shape and topological structure of objects. In a scene configured with a light probe, this includes information about the geometric features of the object's surface, such as vertex positions, normal directions, curvature, as well as the spatial relationships, relative positions, and shapes between the object's parts.

[0053] To further illustrate, an optional geometric prior could be a prior on vertex positions: the three-dimensional coordinates of each vertex on the surface of an object, which determines the specific shape of the object in space. For example, a cube has 8 vertices, each with a clear (x, y, z) coordinate value.

[0054] A geometric prior can also be used to define normal directions: the normal vector at a vertex indicates the orientation of the surface at that point. Normal directions are crucial for calculating reflection and refraction effects of lighting, and different normal directions lead to different lighting results. For example, the normal direction of a sphere varies at different locations on its surface, causing light to reflect differently on the surface.

[0055] Geometric priors can also be used for curvature: a metric that describes the degree of curvature of an object's surface, reflecting its local geometric characteristics. Where curvature is large, the surface is highly curved, and illumination changes may be more dramatic. Where curvature is small, the surface is relatively flat, and illumination changes are relatively gradual. For example, the curvature of a wavy surface can vary significantly at different locations.

[0056] Geometric priors can also be used to determine spatial relationships and relative positions: the spatial relationships between parts of an object, such as the angle or distance between adjacent faces. This information helps determine how light probes illuminate different parts of an object. For example, in a building model, the spatial relationships between different floors and rooms affect the propagation and distribution of light.

[0057] Geometric priors can also be used for shape: the overall shape of an object, such as a circle, square, triangle, etc. Different shapes have different lighting characteristics. For example, a sphere has relatively uniform lighting effects in all directions, while a cube has a clear boundary between light and dark.

[0058] Specifically, this embodiment utilizes geometric prior knowledge to analyze the geometric features of the virtual mesh model (such as vertex positions and normal directions), establishes associations between vertices in the virtual mesh model and light probes, and obtains a first configuration. For example, based on factors such as the distance and angle between the vertex and the light probe, it determines which light probes influence each vertex and the weight of the influence.

[0059] S204: Acquire lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration;

[0060] In an optional embodiment, the lighting baking data can be based on pre-calculated lighting information, by calculating and storing the lighting effects of the virtual mesh model in a specific lighting scene, so as to quickly obtain lighting information during real-time rendering and improve rendering efficiency.

[0061] To further illustrate, a certain number of Light Probes can be placed in the virtual scene. These probes record the lighting information of the surrounding space. Then, based on the relationship between the virtual mesh model's vertices and the Light Probes, the Light Probe data is interpolated and calculated to obtain the lighting baked data for each vertex. For example, in an indoor scene, Light Probes can be placed in each corner of the room and near windows. By calculating the influence of these probes on the model's vertices, the lighting information of the model surface can be obtained.

[0062] Specifically, this embodiment samples the Light Probes based on the associations between Light Probes and vertices of the virtual mesh model determined in the first configuration, calculates and simulates the lighting effects of the virtual mesh model in at least one lighting scenario, and generates lighting baking data. For example, in an indoor scene, according to the first configuration, lighting information is obtained from Light Probes at different locations, and parameters such as brightness and color of the model under different lighting conditions are calculated.

[0063] S206, obtaining real lighting data of the virtual network model in at least one lighting scene;

[0064] In an optional embodiment, real lighting data may refer to the lighting effects of a virtual mesh model under specific lighting scenarios, obtained through high-precision, high-fidelity calculations or measurements. This data closely simulates real-world lighting conditions, reflects the actual performance of objects under real-world lighting conditions, and serves as an important reference for evaluating the accuracy of lighting simulations.

[0065] To further illustrate, physics-based lighting models and algorithms, such as ray tracing and photon mapping, can be used to accurately calculate the lighting of virtual scenes. These methods take into account physical phenomena such as light propagation, reflection, refraction, and scattering, and can generate high-quality, realistic lighting effect data. For example, ray tracing algorithms can simulate the reflection and refraction of light on an object's surface, calculating the object's true lighting color and brightness from different viewing angles.

[0066] Alternatively, similar lighting conditions can be created in a real environment, and high-precision photography equipment can be used to photograph real objects to obtain lighting data. Image processing technology can then be used to convert the captured image data into a lighting data format that can be used in the virtual scene.

[0067] Alternatively, you can use professional rendering engines such as Arnold and V-Ray to render the virtual scene. These engines utilize advanced rendering techniques and algorithms to produce realistic lighting effects. Through the rendered output, you can obtain realistic lighting data for the virtual mesh model under specific lighting scenarios.

[0068] Specifically, this embodiment uses physical simulation, real-life photography, or other high-precision lighting calculation methods to obtain real-world lighting effect data for the virtual mesh model under the same lighting scenario. For example, a ray-following algorithm is used to accurately calculate the virtual scene and obtain the reflection, refraction, and other effects of the model under real-world lighting.

[0069] S208, obtaining a loss function corresponding to the first configuration, where the loss function is used to represent the error between the lighting baking data and the actual lighting data;

[0070] In an optional embodiment, the loss function can be a mathematical function used to measure the difference between the model's predictions and the true values. In lighting simulation, the loss function is used to represent the error between the baked lighting data and the actual lighting data. By minimizing the loss function, the configuration of the Light Probes can be optimized, improving the accuracy of the lighting simulation.

[0071] The loss function provides a clear goal for optimizing the Light Probe configuration. By minimizing the loss function, parameters such as the location, number, and weight of the Light Probes can be adjusted to make the lighting baking data closer to the real lighting data, thereby improving the accuracy of the lighting simulation.

[0072] To further illustrate, an optional loss function could be the Mean Squared Error (MSE): Calculate the average of the squares of the difference between the baked lighting data and the real lighting data at each pixel or vertex. A smaller MSE value indicates a smaller error between the two.

[0073] Another loss function can be the Mean Absolute Error (MAE): This calculates the average of the absolute differences between the baked lighting data and the real lighting data at each pixel or vertex. MAE is relatively insensitive to outliers and can more stably reflect the error situation.

[0074] The loss function can also be a custom loss function: Based on the specific application requirements, you can design a custom loss function. For example, to pay more attention to the lighting effects of certain areas, you can design a weighted loss function to give higher weight to the lighting errors in these areas.

[0075] Specifically, this embodiment constructs a function that compares the baked lighting data with the real lighting data and calculates the error between the two. This function is called the loss function. For example, the mean square error (MSE) is used as the loss function to calculate the average square of the difference between the baked lighting data and the real lighting data at each pixel or vertex.

[0076] S210 , optimizing the first configuration by minimizing the loss function to obtain a second configuration.

[0077] In an optional embodiment, the second configuration may be an association relationship between the lighting probes and the vertices of the virtual mesh model obtained by optimizing the first configuration by minimizing a loss function, and may have higher accuracy than the first configuration.

[0078] Specifically, this embodiment optimizes the first configuration to obtain the second configuration by minimizing the loss function and continuously adjusting parameters such as the position, number, and weight of the Light Probes using optimization methods such as gradient descent and genetic algorithms. For example, using a gradient descent algorithm, the Light Probe parameters are gradually adjusted based on the gradient direction of the loss function, gradually reducing the value of the loss function.

[0079] Optionally, this embodiment can be applied to business scenarios that require model rendering, such as games, movies, animations, and autonomous driving.

[0080] Specifically, using film and television / animation production as an example, we can densely deploy probes in highly reflective areas based on geometric priors (such as surface curvature and material type). We compare the baked results with real-world rendering data from path tracing to calculate a loss function. We optimize the probe configuration to minimize errors, for example by adding probes in areas with strong caustics effects. This allows us to achieve realistic ambient occlusion (AO) and soft shadows in animated films.

[0081] Taking the business scenario of autonomous driving simulation as an example, probes can be deployed based on normal orientation in key areas such as lane markings and traffic signs. Baked data for scenes like midday, dusk, and nighttime can be compared with real-world physically rendered data. By minimizing a loss function, probe configuration can be dynamically adjusted to accommodate varying weather conditions. This enhances the realism of the simulation, reduces the false detection rate of perception algorithms, and simulates the reflection of light from road materials (such as asphalt and road markings) by LiDAR and cameras.

[0082] It should be noted that this embodiment aims to solve the problem of reliance on experience-based judgment and low configuration accuracy in traditional light probe configuration methods, and to improve the accuracy of light probe configuration. Specifically, using geometric prior knowledge, an association relationship between the vertices of the virtual mesh model and the light probes is established to obtain a first configuration, which provides a basis for subsequent lighting calculations. Lighting baking data is obtained based on the first configuration, and real lighting data is obtained through other methods to provide data support for evaluating and optimizing light probe configurations. A loss function is constructed to quantify the error between the lighting baking data and the real lighting data, and to clarify the optimization target. By minimizing the loss function, the first configuration is optimized to obtain the second configuration, so that the configuration of the light probe is closer to the real lighting effect.

[0083] To illustrate further, the optional Figure 3As shown, at least one lighting probe 306 (such as lighting probe 1, lighting probe 2, lighting probe 3, and lighting probe 4) associated with the virtual mesh model 302 is obtained, and the corresponding first configuration 308 is obtained, wherein the first configuration 308 is obtained by using geometric priors, and the association relationship between the vertices 304 (such as vertex 1, vertex 2, vertex 3, and vertex 4) in the virtual mesh model 302 and the lighting probes 306 in at least one lighting probe 306 (such as the association relationship between lighting probe 1 and vertex 1, the association relationship between lighting probe 2 and vertex 2, and the association relationship between lighting probe 3 and vertex 3) is obtained. , the association relationship between the lighting probe 4 and the vertex 4); based on the at least one lighting probe 306 of the first configuration 308, obtain the lighting baking data 310 of the virtual network model in at least one lighting scene; obtain the real lighting data 312 of the virtual network model in at least one lighting scene; obtain the loss function 314 corresponding to the first configuration 308, wherein the loss function 314 is used to represent the error between the lighting baking data 310 and the real lighting data 312; by minimizing the loss function 314, optimize the first configuration 308 to obtain the second configuration 316.

[0084] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0085] Through the embodiments provided herein, geometric prior knowledge is first used to establish an association between the vertices of the virtual mesh model and at least one Light Probe, forming a first configuration. This geometric feature-based configuration can more reasonably consider the object's shape and spatial distribution, improving the accuracy of the initial configuration.

[0086] Next, based on the association between the Light Probes and the vertices of the virtual mesh model in the first configuration, lighting baking data for the model is obtained under at least one lighting scenario. Light baking is a technique that pre-calculates lighting information and stores it in textures or vertex data. Sampling the Light Probe data from the first configuration can simulate the effects of the model under different lighting scenarios. Because light baking data is generated based on the Light Probe information from the first configuration, its accuracy depends to a certain extent on the quality of the first configuration. Properly configuring Light Probes can yield more accurate light baking data, providing a foundation for subsequent lighting effect evaluation and optimization.

[0087] At the same time, we obtain real-world lighting data for the virtual mesh model under the same lighting scenario. This data, obtained through physical simulation, real-life photography, or other high-precision lighting calculations, reflects the true effect of the object under actual lighting conditions and is an important basis for evaluating the accuracy of light baking data. Real-world lighting data provides a benchmark for evaluating light baking data errors. By comparing the two, we can quantitatively analyze the accuracy of light probe configurations, identify existing problems and deficiencies, and provide guidance for subsequent configuration optimization.

[0088] Finally, a loss function is constructed for the first configuration. This loss function represents the error between the baked lighting data and the real-world lighting data. By minimizing this loss function, the first configuration is optimized to obtain the second configuration. During the optimization process, the loss function is gradually reduced, improving the accuracy of the Light Probe configuration. The loss function provides a clear goal and evaluation criteria for optimizing the Light Probe configuration. By minimizing the loss function, the Light Probe configuration can be automatically adjusted to more closely resemble the real-world lighting effect.

[0089] By using geometric prior knowledge to establish associations, obtain lighting baking data and real lighting data, construct loss functions and optimize configurations, the problem of low accuracy in lighting probe configuration optimization is solved from multiple aspects. This not only improves the initial accuracy of the light probe configuration, but also provides an effective configuration optimization method. It can dynamically adjust the configuration according to the actual lighting effect, meet the high-precision requirements of lighting effects in complex scenes, and thus achieve the purpose of dynamically optimizing the initial configuration of the light probe, thereby realizing the technical effect of improving the configuration accuracy of the light probe.

[0090] As an optional solution, obtaining the loss function corresponding to the first configuration includes:

[0091] S1-1, quantify the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error;

[0092] Optionally, a comparative analysis is performed on the lighting baking data and the real lighting data, and the overall deviation between them is calculated using a suitable mathematical method (such as mean square error, etc.), and the deviation is quantified to obtain the lighting error.

[0093] S1-2, use the directional weight matrix to differentially weight the illumination errors in different directions or positions to obtain the loss function, where the directional weight matrix is used to adjust the importance of illumination in each direction in the loss function.

[0094] Optionally, the directional weight matrix is combined with the illumination error, and the illumination error is differentially weighted according to the weight values of different directions or positions in the matrix.

[0095] In an optional embodiment, directional weight matrix: a matrix used to adjust the importance of each directional lighting in the loss function.

[0096] In an optional embodiment, differential weighting is performed: different weights are assigned to illumination errors according to different directions or positions.

[0097] It should be noted that this embodiment aims to obtain the loss function corresponding to the first configuration. Specifically, the lighting error is obtained by quantifying the overall deviation between the lighting baking data and the real lighting data, and then the lighting errors in different directions or positions are differentially weighted using the directional weight matrix to obtain the loss function. The directional weight matrix can adjust the importance of lighting in each direction in the loss function.

[0098] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0099] Through the embodiments provided in this application, the overall deviation between the lighting baking data and the real lighting data is quantified to obtain the lighting error; the directional weight matrix is used to perform differential weighting on the lighting errors in different directions or positions to obtain a loss function, wherein the directional weight matrix is used to adjust the importance of lighting in each direction in the loss function. By quantifying the lighting error and using the directional weight matrix, the loss function can more accurately reflect the lighting simulation effect of the first configuration, which helps to discover problems in the lighting simulation, thereby achieving the purpose of making the lighting baking data closer to the real lighting data and improving the quality of the lighting simulation, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0100] As an optional solution, before quantifying the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error, the method should include:

[0101] S2-1, obtaining spherical harmonic basis functions obtained by convolving the lighting baking data, wherein the spherical harmonic basis functions are used to encode lighting direction information into mathematical coefficients;

[0102] S2-2, obtaining a barycentric coordinate matrix, wherein the barycentric coordinate matrix is used to represent the illumination interpolation between vertices;

[0103] S2-3, obtaining a weight matrix, wherein the weight matrix is used to represent the association relationship corresponding to the first configuration, and the weight matrix allows adjustment during the process of minimizing the loss function;

[0104] S2-4, obtaining a baked spherical harmonic coefficient matrix, where the spherical harmonic coefficient matrix is used to store lighting information pre-calculated by at least one lighting probe;

[0105] S2-5, obtain the product of the spherical harmonic function basis function, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix to obtain the predicted lighting data;

[0106] S2-6, obtaining the difference between the predicted illumination data and the actual illumination data to obtain the overall deviation degree.

[0107] In an optional embodiment, spherical harmonics basis functions: a set of specific mathematical functions that have the ability to convert lighting direction information into mathematical coefficients.

[0108] In an optional embodiment, a convolution operation is performed on the lighting bake data, using a specific convolution kernel to extract key features of the lighting data, thereby generating spherical harmonic basis functions. The convolution operation can be considered a filtering process that can highlight specific patterns in the lighting data.

[0109] In an optional embodiment, a barycentric coordinate matrix is a matrix used to describe the interpolation relationship between lighting vertices. In a triangular mesh model, the lighting at any point can be calculated by interpolating the lighting of the three vertices of the triangle in which it is located. The barycentric coordinate matrix clearly defines this interpolation relationship.

[0110] In an optional embodiment, the matrix may be constructed by constructing a barycentric coordinate matrix based on the topological structure and vertex coordinates of the triangular mesh, wherein the element values in the matrix represent the weights of the vertices in the interpolation process.

[0111] In an optional embodiment, the weight matrix can be used to represent the association relationship corresponding to the first configuration and is adjustable during the process of minimizing the loss function. The element values of the weight matrix can be dynamically adjusted according to different configurations and optimization objectives.

[0112] In an optional embodiment, the initialization matrix may be configured by initializing the weight matrix according to the initial parameters of the first configuration.

[0113] In an optional embodiment, the adjustment mechanism may be to adjust the weight matrix using a suitable optimization algorithm (such as gradient descent method) according to the change of the loss function during the optimization process so as to minimize the loss function.

[0114] In an optional embodiment, the spherical harmonic coefficient matrix can be used to store lighting information pre-calculated by at least one lighting probe. The elements in the spherical harmonic coefficient matrix are the coefficients of the spherical harmonic function basis functions, representing the lighting intensity in different directions.

[0115] In an optional embodiment, the pre-calculation process may be to collect lighting data using lighting probes during the scene baking stage, and use spherical harmonic functions to encode lighting direction information into spherical harmonic coefficients, which are finally stored in a spherical harmonic coefficient matrix.

[0116] In an optional embodiment, the product operation may be a matrix multiplication operation involving the spherical harmonic basis functions, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix.

[0117] In an optional embodiment, the predicted lighting data may be the result of the above-mentioned matrix product operation, representing lighting data predicted based on the current configuration and lighting information.

[0118] In an optional embodiment, the matrix multiplication may be performed by sequentially calculating the product of each matrix according to the rules of matrix multiplication. For example, the product of the spherical harmonics basis functions and the spherical harmonic coefficient matrix may be first calculated to obtain an intermediate result, which is then multiplied by the barycentric coordinate matrix and the weight matrix to ultimately obtain the predicted illumination data.

[0119] In an optional embodiment, the overall deviation degree may be the degree of difference between the predicted illumination data and the actual illumination data, which is used to measure the accuracy of the illumination simulation.

[0120] In an optional embodiment, the difference calculation may be to calculate the difference between the predicted illumination data and the actual illumination data, and a suitable distance measurement method may be used, such as Euclidean distance, cosine similarity, etc.

[0121] It should be noted that before quantifying the overall deviation between the baked lighting data and the actual lighting data (lighting error), this embodiment requires a series of key preparatory tasks. Specifically, the spherical harmonic basis functions, barycentric coordinate matrix, weight matrix, and baked spherical harmonic coefficient matrix obtained by convolution of the baked lighting data are obtained. The product of these matrices is then calculated to obtain the predicted lighting data. Finally, the overall deviation is calculated by calculating the difference between the predicted lighting data and the actual lighting data.

[0122] To further illustrate, consider a simple indoor scene containing a cube model and two light probes. A convolution operation is performed on the lighting data collected by the light probes to obtain spherical harmonic basis functions, such as the first three spherical harmonic basis functions. Based on the topological structure of the cube model's triangle mesh, a barycentric coordinate matrix is constructed to represent the lighting interpolation relationship between vertices.

[0123] The weight matrix is then initialized, assuming that the influence of the two light probes is initially equal. During the optimization process, the weight matrix is adjusted to minimize the loss function based on the changes in the loss function. The scene's lighting information is pre-computed using light probes and encoded as spherical harmonic coefficients, which are stored in the spherical harmonic coefficient matrix. Matrix multiplication is performed on the spherical harmonic basis functions, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix to obtain predicted lighting data. The difference between the predicted lighting data and the actual lighting data is calculated to determine the overall deviation.

[0124] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0125] Through the embodiments provided by the present application, spherical harmonic basis functions obtained after convolution of lighting baking data are obtained, wherein the spherical harmonic basis functions are used to encode lighting direction information into mathematical coefficients; a barycentric coordinate matrix is obtained, wherein the barycentric coordinate matrix is used to represent lighting interpolation between vertices; a weight matrix is obtained, wherein the weight matrix is used to represent the association relationship corresponding to the first configuration, and the weight matrix allows adjustment in the process of minimizing the loss function; a baked spherical harmonic coefficient matrix is obtained, wherein the spherical harmonic coefficient matrix is used to store lighting information pre-calculated by at least one lighting probe; the product of the spherical harmonic basis functions, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix is obtained to obtain predicted lighting data; the difference between the predicted lighting data and the actual lighting data is obtained to obtain the overall deviation degree. By obtaining the spherical harmonic basis functions, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix, and calculating the predicted lighting data, the lighting distribution in the scene can be simulated more accurately, and the lighting error can be effectively reduced, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0126] As an optional solution, the first configuration is optimized by minimizing the loss function to obtain the second configuration, including:

[0127] S3-1, obtaining the gradient of the loss function with respect to the weight matrix, wherein the gradient is used to indicate adjustment of the weight matrix to reduce the error between the lighting baking data and the actual lighting data;

[0128] S3-2, using the gradient, iteratively update the weight matrix until the error between the lighting baking data and the real lighting data meets the convergence condition;

[0129] S3-3, determining the configuration corresponding to the weight matrix when the convergence condition is satisfied as the second configuration.

[0130] In an optional embodiment, the gradient can be a vector composed of partial derivatives of the loss function with respect to the weight matrix, indicating the direction and rate of change of the loss function under the current value of the weight matrix.

[0131] In an optional embodiment, the iterative process may be to gradually approach the optimal weight matrix that minimizes the loss function by repeatedly updating the weight matrix.

[0132] In an optional embodiment, the convergence condition may be a condition for determining whether the error between the lighting baking data and the real lighting data meets certain standards, such as the error being less than a certain threshold or the rate of change of the error being less than a certain threshold.

[0133] It should be noted that this embodiment aims to optimize the first configuration by minimizing the loss function, thereby obtaining the second configuration. The specific process is as follows: first, the gradient of the loss function with respect to the weight matrix is obtained. This gradient provides a direction for adjusting the weight matrix to reduce the error between the lighting bake data and the real lighting data; then, the weight matrix is iteratively updated using this gradient until the error meets the convergence criteria; finally, the configuration corresponding to the weight matrix that meets the convergence criteria is determined as the second configuration. This process effectively improves the accuracy of the lighting simulation, making the lighting bake data closer to the real lighting data.

[0134] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0135] Through the embodiments provided in the present application, the gradient of the loss function with respect to the weight matrix is obtained, wherein the gradient is used to indicate the adjustment of the weight matrix to reduce the error between the lighting baking data and the real lighting data; the gradient is used to iteratively update the weight matrix until the error between the lighting baking data and the real lighting data meets the convergence condition; the configuration corresponding to the weight matrix when the convergence condition is met is determined as the second configuration, and the first configuration is optimized by minimizing the loss function. The obtained second configuration can make the lighting baking data more accurately approximate the real lighting data, thereby improving the accuracy of the lighting simulation, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0136] As an optional solution, obtain the gradient of the loss function with respect to the weight matrix, including:

[0137] S4-1, obtaining the product of the spherical harmonic function basis function, the barycentric coordinate matrix, and the spherical harmonic coefficient matrix to obtain a first value;

[0138] S4-2, obtaining the product of the overall deviation degree and the double directional weight matrix to obtain a second value;

[0139] S4-3, obtaining the product of the first value and the second value to obtain a gradient.

[0140] In an optional embodiment, the matrix multiplication may be performed by sequentially multiplying the spherical harmonics basis function matrix, the barycentric coordinate matrix, and the spherical harmonics coefficient matrix according to the rules of matrix multiplication. For example, assuming the spherical harmonics basis function matrix is A, the barycentric coordinate matrix is B, and the spherical harmonics coefficient matrix is C, then the first value M1 = A×B×C.

[0141] In an optional embodiment, calculating the doubled directional weight matrix may be to multiply each element of the directional weight matrix by 2 to obtain the doubled directional weight matrix. For example, assuming that the directional weight matrix is D, the doubled directional weight matrix is 2D.

[0142] In an optional embodiment, the scalar and matrix multiplication may be to multiply the overall deviation degree (scalar) by twice the directional weight matrix to obtain a second value M2=k×2D, where k is the overall deviation degree.

[0143] It should be noted that this embodiment aims to obtain the gradient of the loss function with respect to the weight matrix. The core idea is to obtain this gradient through a series of matrix operations. The specific steps are: first, obtain the first value by taking the product of the spherical harmonic function basis function, the barycentric coordinate matrix, and the spherical harmonic coefficient matrix; then obtain the second value by taking the product of the overall deviation degree and the double direction weight matrix; finally, multiply the first value by the second value to obtain the gradient. This solution is of great significance in fields such as lighting simulation and computer graphics, helping to optimize model parameters and improve the accuracy and effect of simulation.

[0144] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0145] Through the embodiments provided in the present application, the product of the spherical harmonic function basis function, the barycentric coordinate matrix, and the spherical harmonic coefficient matrix is obtained to obtain a first value; the product of the overall deviation degree and the double directional weight matrix is obtained to obtain a second value; the product of the first value and the second value is obtained to obtain a gradient. By accurately obtaining the gradient of the loss function with respect to the weight matrix, the weight matrix can be adjusted more effectively and the convergence speed of the optimization process can be accelerated. The accurate gradient information helps to optimize the model parameters and make the lighting simulation results closer to the real lighting, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0146] As an optional solution, in the process of obtaining lighting baking data of the virtual network model in at least one lighting scene based on at least one lighting probe of the first configuration, the method further includes:

[0147] Obtaining, based on the at least one lighting probe of the first configuration, first baked data of the virtual network model in a first lighting scene, wherein the at least one lighting scene includes the first lighting scene, and the lighting baked data includes the first baked data;

[0148] In the process of obtaining real lighting data of the virtual network model in at least one lighting scene, the method further includes:

[0149] First lighting data of the virtual network model in a first lighting scene is obtained, wherein the real lighting data includes the first lighting data.

[0150] It should be noted that this embodiment focuses on the process of acquiring lighting baking data and real lighting data for a virtual network model in a lighting probe environment based on a specific first configuration. The core of this process is to first obtain the first baked data of the virtual network model in a specific lighting scenario (the first lighting scenario) through the lighting probe. This data falls into the category of lighting baking data. Subsequently, during the process of acquiring real lighting data, the first lighting data of the model in the same scenario is obtained. This data also falls into the category of real lighting data.

[0151] To further illustrate, it is optional to assume that in a game development scenario, it is necessary to obtain lighting baking data and real lighting data of a virtual network model (a character model) under different lighting scenarios.

[0152] According to the first configuration, 10 Light Probes are placed around the character model, with sampling accuracy set to high. In the first lighting scenario (a bright outdoor scene), the Light Probes collect ambient lighting information, such as a light intensity of 1000 lux and a color of white. This collected lighting information is processed to generate the first baked data, including the light intensity and color distribution of different parts of the character model.

[0153] Next, a physically based lighting model (PBR) is selected for lighting calculations. Based on the PBR model, the character model's material properties (such as metallicity and roughness), light source position and intensity, and other information are combined to calculate its lighting intensity and color in the first lighting scenario. The calculated lighting intensity and color information are organized into the first lighting data, such as the lighting intensity of the character model's head is 800 lux and the color is light yellow.

[0154] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0155] Through the embodiments provided in the present application, based on at least one lighting probe of the first configuration, first baked data of the virtual network model in the first lighting scene is obtained, wherein the at least one lighting scene includes the first lighting scene, and the lighting baking data includes the first baked data; the first lighting data of the virtual network model in the first lighting scene is obtained, wherein the real lighting data includes the first lighting data. By accurately obtaining the lighting baking data and the real lighting data, the lighting effect of the virtual scene can be improved, and the lighting of the characters and objects can be made more realistic and natural, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0156] As an optional solution, the at least one lighting scene includes a first lighting scene and a second lighting scene.

[0157] Obtaining lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration includes:

[0158] Obtaining, based on the at least one lighting probe of the first configuration, first baked data and second baked data of the virtual network model in a second lighting scenario, wherein the lighting baked data includes the second baked data;

[0159] Obtaining real lighting data of the virtual network model in at least one lighting scenario, including:

[0160] Acquire first lighting data and second lighting data of the virtual network model in a second lighting scene, wherein the real lighting data includes the second lighting data;

[0161] Get the loss function corresponding to the first configuration, including:

[0162] S5-1, obtaining a first error between first baking data and first lighting data;

[0163] S5-2, obtaining a second error between the second baking data and the second lighting data;

[0164] S5-3, integrating the first error and the second error to obtain a loss function.

[0165] In an optional embodiment, the integrated error may be obtained by integrating the first error and the second error to obtain a loss function.

[0166] To further illustrate, the first error and the second error can optionally be weighted together to form a loss function. Specifically, by assigning a weight to each error term, multiple error terms are combined to form an overall error metric. The size of the weight reflects the importance of each error term in the overall evaluation.

[0167] Alternatively, the first error and the second error can be integrated using a custom function to obtain a loss function. Specifically, based on specific needs, a custom integration function can be designed to combine the first error and the second error in a nonlinear manner. For example, an exponential function or a logarithmic function can be used to adjust the weights of the errors.

[0168] It should be noted that this example focuses on obtaining lighting data and calculating loss functions for a virtual network model under different lighting scenarios. First, it is determined that the lighting scenarios include the first and second lighting scenarios. Next, a first configuration of lighting probes is used to obtain baked lighting data for the model under these two scenarios. Simultaneously, the model obtains real lighting data for both scenarios. Finally, based on the error between the baked and real lighting data, the loss function corresponding to the first configuration is integrated.

[0169] To further illustrate, it is optional to assume that in a game development scenario, it is necessary to obtain the lighting baking data and real lighting data of a virtual network model (a character model) in two different lighting scenes, and calculate the loss function.

[0170] The first lighting scene is a bright outdoor daytime scene, the light source is the sun, the light intensity is 1000 lux, and the color is white; the second lighting scene is a dim indoor night scene, the light source is a lamp, the light intensity is 200 lux, and the color is warm yellow.

[0171] According to the first configuration, 10 light probes are placed around the character model, and the sampling accuracy is set to high precision. The light probes collect lighting information of the surrounding environment in the first lighting scene and the second lighting scene. The collected lighting information is processed and integrated to generate the first baked data (for example, the light intensity of the character model's head is 800 lux and the color is light yellow) and the second baked data (for example, the light intensity of the character model's head is 150 lux and the color is warm yellow).

[0172] A physically based lighting model (PBR) is selected for lighting calculations. Based on the PBR model, the character model's material properties (such as metallicity and roughness), light source position and intensity, and other information are combined to calculate its lighting intensity and color in the first and second lighting scenarios.

[0173] Generate real lighting data: Organize the calculated lighting intensity and color information into first lighting data (e.g., the lighting intensity on the character model's head is 750 lux and the color is light yellow) and second lighting data (e.g., the lighting intensity on the character model's head is 140 lux and the color is warm yellow).

[0174] The first error is calculated to be 50 lux and the second error is 10 lux. Assuming a simple weighted summation method is used, the loss function is 60 (the weights can be adjusted according to actual needs).

[0175] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0176] Through the embodiments provided by the present application, based on at least one lighting probe of the first configuration, first baked data and second baked data of the virtual network model in the second lighting scene are obtained, wherein the lighting baked data include the second baked data; the first lighting data and the second lighting data of the virtual network model in the second lighting scene are obtained, wherein the real lighting data include the second lighting data; a first error between the first baked data and the first lighting data is obtained; a second error between the second baked data and the second lighting data is obtained; the first error and the second error are integrated to obtain a loss function. By obtaining more accurate lighting baked data and real lighting data, and calculating the loss function for optimization, the accuracy of the lighting effect of the virtual scene can be improved, and the lighting of the characters and objects can be made more realistic and natural, thereby achieving the technical effect of improving the configuration accuracy of the lighting probe.

[0177] As an optional solution, the at least one lighting scene includes a third lighting scene and a fourth scene. Before obtaining lighting baking data of the virtual network model in the at least one lighting scene based on the at least one lighting probe of the first configuration, the method further includes:

[0178] S6-1, constructing a standard scene, wherein the standard scene has a first lighting condition and a second lighting condition;

[0179] S6-2, simulating a first appearance of the virtual grid model under a first lighting condition and a second appearance of the virtual grid model under a second lighting condition by rotating the virtual grid model in a standard scene;

[0180] S6-3, determining the first representation as the representation of the virtual grid model in the third lighting scene, and determining the second representation as the representation of the virtual grid model in the fourth lighting scene.

[0181] It should be noted that before obtaining the lighting baking data of the virtual network model under at least one lighting scene (the third lighting scene and the fourth scene), a series of preparatory work is performed first. First, a standard scene is constructed, which contains two different lighting conditions, the first lighting condition and the second lighting condition. Then the virtual grid model is rotated in this standard scene, and the performance of the virtual grid model under two lighting conditions, namely the first performance and the second performance, is simulated by rotation. Finally, the first performance is used as the performance of the virtual grid model in the third lighting scene, and the second performance is used as the performance of the virtual grid model in the fourth lighting scene. The purpose of this embodiment is to provide a basis for the subsequent acquisition of lighting baking data, and by simulating the performance under different lighting conditions, the state of the virtual network model in a specific lighting scene can be more accurately determined, thereby providing a more reasonable basis for the configuration of lighting probes and the acquisition of lighting baking data.

[0182] To further illustrate, it is optional to assume that this embodiment is to obtain lighting baking data for a character model (virtual network model) in a virtual game. There are multiple different lighting scenes in the game, and here we focus on the third and fourth lighting scenes.

[0183] This embodiment creates a standard scene in a game, in which there are two different lighting conditions, one is bright daytime lighting (a first lighting condition), and the other is dim nighttime lighting (a second lighting condition).

[0184] Place the character model (virtual mesh model) in the standard scene and rotate it. During the rotation, observe and record the character model's performance under daylight (primary performance), such as the character's shadow shape and highlight position. Also, record the character model's performance under nightlight (secondary performance), such as the distribution of light and shadow on the character.

[0185] According to the simulation results, the performance of the character model under daylight is determined as the performance of the character model in the third lighting scene (such as a daytime outdoor scene), and the performance of the character model under nightlight is determined as the performance of the character model in the fourth lighting scene (such as a nighttime indoor scene).

[0186] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0187] Through the embodiments provided in this application, a standard scene is constructed, wherein the standard scene has a first lighting condition and a second lighting condition. By rotating the virtual grid model in the standard scene, a first performance of the virtual grid model under the first lighting condition and a second performance of the virtual grid model under the second lighting condition are simulated. The first performance is determined as the performance of the virtual grid model in a third lighting condition, and the second performance is determined as the performance of the virtual grid model in a fourth lighting condition. This can more accurately simulate the performance of the virtual network model under different lighting conditions, providing a more reliable basis for the subsequent acquisition of lighting baking data. This embodiment can take into account the impact of various lighting conditions on the virtual network model, making the acquired lighting baking data more realistic and accurate, thereby improving the lighting effect and realism of the virtual scene. At the same time, by simulating the performance under different lighting conditions, it can also provide guidance for the configuration of lighting probes, optimize the number, location, and sampling accuracy of light probes, and further improve the efficiency and quality of lighting baking.

[0188] As an optional solution, before obtaining the first configuration corresponding to at least one light probe associated with the virtual mesh model, the method further includes:

[0189] Obtain at least two sampling points on the virtual grid model;

[0190] At least one illumination probe is configured based on the at least two sampling points to obtain a first configuration.

[0191] In an optional embodiment, the sampling points may be points with specific position information selected on the virtual grid model for use in subsequent distance calculation and illumination probe configuration analysis.

[0192] Specifically, this embodiment selects at least two representative points from the virtual grid model as sampling points. This step determines the basic data for subsequent analysis.

[0193] To illustrate further, let's assume a simple virtual game scene, which is a small indoor room composed of multiple small blocks (grid units) with walls, furniture, etc. At least two sampling points are randomly selected on the virtual grid model of this room.

[0194] For example, select a sampling point A at the corner of the room, select a sampling point B on the ground in the center of the room, and select a sampling point C on the wall next to the window of the room.

[0195] It should be noted that before obtaining the first configuration corresponding to at least one Light Probe associated with the virtual grid model, a series of preparatory steps are performed. First, at least two sampling points are selected on the virtual grid model. The selection of these sampling points is intended to provide reference information for the subsequent configuration of the Light Probe. Then, based on information such as the position and lighting characteristics of these sampling points, at least one Light Probe is configured, including determining parameters such as the position, direction, and sampling accuracy of the Light Probe, thereby obtaining the first configuration. The purpose of this series of operations is to enable the Light Probe to more accurately collect lighting information of the environment in which the virtual grid model is located, providing reliable data support for subsequent lighting calculations and simulations.

[0196] To further illustrate, it is optionally assumed that this embodiment performs lighting simulation on a character model (virtual grid model) in a virtual game scene, and that a lighting probe is required to collect lighting information.

[0197] In this embodiment, a plurality of key positions on the character model are selected as sampling points, such as vertices of the head, shoulders, chest, arms, legs, etc. These sampling points can represent the lighting conditions of different parts of the character model.

[0198] Based on the locations and lighting characteristics of these sampling points, this embodiment configures the Light Probes in the scene. For example, Light Probes are placed in appropriate locations around the character model so that they cover the area where all sampling points are located. The Light Probes are oriented toward the sampling points to better collect lighting information from them. The Light Probes' sampling accuracy is set based on the lighting variations at the sampling points, for example, setting a higher sampling accuracy in areas with large lighting variations and a lower sampling accuracy in areas with small lighting variations. This ultimately results in a first configuration that enables Light Probes to accurately collect lighting information from the character model's environment.

[0199] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0200] Through the embodiments provided in this application, at least two sampling points are obtained on the virtual grid model; based on the at least two sampling points, at least one lighting probe is configured to obtain a first configuration. Based on the characteristics of the virtual grid model and lighting requirements, sampling points can be reasonably selected and lighting probes can be configured to obtain a more accurate first configuration. This enables the lighting probes to better collect lighting information about the environment in which the virtual grid model resides, improving the accuracy and realism of the lighting simulation. At the same time, by optimizing the configuration of sampling points and lighting probes, the efficiency of lighting calculations can also be improved, the consumption of computing resources can be reduced, and better support can be provided for real-time rendering and interaction of virtual scenes.

[0201] As an optional solution, and for ease of understanding, the above light probe configuration method is applied to lighting rendering scenes in 3D games. This light probe-based global illumination (GI) solution is used to improve game image quality and realism. This embodiment uses machine learning methods to reversely optimize the distribution of light probes based on the loss of light baking in complex standard scenes, which is more rigorous and reasonable.

[0202] Specifically, the process of this embodiment is as follows:

[0203] S7-1, clustering of sampling points on the triangular mesh surface: clustering the sampling points of the triangular mesh using a K-Mediods-based clustering method, and initializing the light probes with the cluster centers (Mediod);

[0204] The formula of K-Mediods clustering algorithm is as follows:

[0205]

[0206] Here, E represents the total energy or the sum of squared distances of the entire dataset, which is used to measure the quality of the clustering results. K is the number of clusters, that is, the number of clusters into which the dataset is desired to be divided. Cj represents the jth cluster, which contains all the sample points within that cluster. p is any sample point in cluster Cj. oi is the medoid (center point) of the i-th cluster, representing that cluster. dist(p, oi) represents the distance between sample point p and the center point oi of its cluster.

[0207] In an optional embodiment, the lighting probe distribution requirements are to avoid light leakage (sampling points on opposite sides are associated with different lighting probes to minimize light leakage, etc.), reasonable association (sampling points on the same side and close to each other are associated with the same lighting probe), and lighting continuity (the distance calculation formula should be continuous for the triangular mesh surface to avoid lighting discontinuity). The requirements are summarized as same-surface association (sampling points on the same surface are associated with the same probe point and the distance is short), opposite-surface association (sampling points on opposite surfaces are associated with different probe points and the distance is long), and GI continuity (ensuring the continuity of global illumination).

[0208] This embodiment proposes a sampling point pathfinding distance algorithm as the distance calculation formula of the clustering algorithm. The algorithm appropriately increases the distance of invisible sampling points while ensuring continuity. Specifically, the Euclidean distance is used between visible sampling points of different triangular mesh surfaces, the mesh surface distance (Mesh Surface Distance) is used for sampling points of the same triangular mesh surface, and the distance of invisible sampling points is calculated using the pathfinding algorithm between sampling points.

[0209] To illustrate further, the optional Figure 4 As shown in the figure, using only the Euclidean distance results in the invisible sampling points (white point and point A) being close to each other, which is prone to light leakage problems.

[0210] Optional e.g. Figure 5 As shown in Figure 2, using only the geometric surface distance between invisible sampling points will result in discontinuity between visible and invisible sampling points (point A).

[0211] Optional e.g. Figure 6 As shown in Figure 3, the Euclidean distance is used between visible sampling points, and the invisible sampling points are separated by a pathfinding algorithm to achieve the desired goal while ensuring smoothness.

[0212] S7-2, initialize the association between vertices and light probes: This step is the middle link of the entire process. After the light probes are obtained through clustering, the vertices of the triangle mesh are initially associated with the light probes.

[0213] To further illustrate, an optional method is to calculate the distance between the sampling point and the nearest n detection points, and then invert and normalize the distance from the sampling point to the detection point and add it to the vertex as the weight, as shown in the following formula:

[0214]

[0215] Among them, p1, p2, ..., p m are sampling points in the same triangle, is a vertex about the exploration point o i The weight of is the sampling point p to the detection point o i The weight of , bary(p) represents the coordinates of the center of gravity of the sampling point p.

[0216] In order to ensure smoothness, each vertex needs to be associated with multiple probe points with the largest weight. In practice, each vertex is associated with two probe points, such as Figure 7 Each sampling point shown is only associated with the nearest probe point, resulting in a narrow transition area and an uneven transition. Figure 8 The more probe points are associated with each sample point shown, the larger the transition area will be and the smoother the result will be.

[0217] S7-3, gradient descent optimization of the weights of vertices and light probes in the standard scene: In the standard scene, the gradient descent algorithm is used to optimize the weights of vertices and light probes, and finally the association relationship between vertices and light probes is obtained.

[0218] In an optional embodiment, step S7-2 has obtained the association between the initialized vertices and the light probes, and this association can be represented by a matrix W. It is necessary to complete the generation of this association in the model space (solving W), which will ensure that the model has good performance in various lighting scenarios. The loss function is defined as follows:

[0219]

[0220] Among them, N sce represents the number of scenes with different lighting conditions, It represents the fitting error of the GI of the i-th scene object under the current association relationship W.

[0221] Based on the lighting bake, the current loss is defined:

[0222] E loss =w||T(Y)·B·W·SH-I|| 2

[0223] Where w is the directional weight matrix, T(Y) is the spherical harmonic basis function after convolution (T can also be called the transfer function), B is the barycentric coordinate matrix, SH is the spherical harmonic coefficient matrix obtained after baking, and I is the actual lighting data.

[0224] Solve the gradient of the loss with respect to W:

[0225]

[0226] It can be seen from this that gradient descent can be used to optimize W.

[0227] It should be noted that simply using gradient descent cannot obtain the ideal light probe distribution result within a limited time. Therefore, the methods of steps S7-1 and S7-2 are adopted as geometric priors, and on this basis, the weights (the weights of light probes in the mesh vertices) are further optimized by gradient descent.

[0228] In order to ensure that the probes distributed in the model space can make the model instance have a good performance in any lighting environment, it is necessary to build scenes with different complex lighting, but this will consume too many resources.

[0229] Therefore, this embodiment adopts the design of standard scene and constructs a single standard scene with complex lighting conditions, such as Figure 9 As shown, by rotating the object 904 in the standard scene 902, multiple sets of lighting information 906 are obtained. This only requires building a complex standard scene, improving resource utilization. Based on the geometric prior, the final relationship between the vertex and the light probe can be quickly optimized.

[0230] Optionally, the content of this embodiment is applicable to other embodiments mentioned above, or combined with other embodiments mentioned above.

[0231] The embodiments provided herein utilize geometric priors to minimize light leaks and significantly shorten optimization time. By optimizing the distribution of light probes based on the loss of baked illumination in complex standard scenes through machine learning, we can better represent the GI information of objects. Illumination based solely on geometric information often results in smoother results.

[0232] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0233] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0234] According to another aspect of the embodiment of the present application, a configuration optimization device for an illumination probe is provided for implementing the configuration optimization method of the illumination probe. Figure 10 As shown, the device includes:

[0235] A first obtaining unit 1002 is configured to obtain a first configuration corresponding to at least one Light Probe associated with the virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and Light Probes in the at least one Light Probe obtained using a geometric prior.

[0236] A second acquisition unit 1004 is configured to acquire lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration;

[0237] The third acquisition unit 1006 is configured to acquire real lighting data of the virtual network model in at least one lighting scene;

[0238] A fourth obtaining unit 1008 is configured to obtain a loss function corresponding to the first configuration, wherein the loss function is used to represent an error between the lighting baking data and the actual lighting data;

[0239] The optimization unit 1010 is configured to optimize the first configuration by minimizing the loss function to obtain a second configuration.

[0240] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0241] As an optional solution, the fourth obtaining unit 1008 includes:

[0242] The quantization module is used to quantify the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error;

[0243] The weighting module is used to use the directional weight matrix to differentially weight the illumination errors in different directions or positions to obtain a loss function. The directional weight matrix is used to adjust the importance of illumination in each direction in the loss function.

[0244] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0245] As an optional solution, the device may include:

[0246] A first acquisition module is configured to obtain spherical harmonic basis functions obtained by convolving the lighting bake data with the actual lighting data before quantifying the overall deviation between the lighting bake data and the actual lighting data to obtain the lighting error. The spherical harmonic basis functions are used to encode lighting direction information into mathematical coefficients.

[0247] The second acquisition module is used to obtain the barycentric coordinate matrix before quantifying the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error, wherein the barycentric coordinate matrix is used to represent the lighting interpolation between vertices;

[0248] A third acquisition module is configured to obtain a weight matrix before quantifying the overall deviation between the lighting baking data and the actual lighting data to obtain the lighting error. The weight matrix is used to represent the association relationship corresponding to the first configuration, and the weight matrix allows adjustment during the process of minimizing the loss function.

[0249] A fourth acquisition module is used to obtain the baked spherical harmonic coefficient matrix before quantifying the overall deviation between the baked lighting data and the real lighting data to obtain the lighting error, where the spherical harmonic coefficient matrix is used to store the lighting information pre-calculated by at least one lighting probe;

[0250] The fifth acquisition module is used to obtain the product of the spherical harmonics basis functions, the barycentric coordinate matrix, the weight matrix, and the spherical harmonics coefficient matrix to obtain the predicted lighting data before quantifying the overall deviation between the lighting baking data and the actual lighting data to obtain the lighting error;

[0251] The sixth acquisition module is used to obtain the difference between the predicted lighting data and the actual lighting data to obtain the overall deviation degree before quantifying the overall deviation degree between the lighting baking data and the actual lighting data to obtain the lighting error.

[0252] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0253] As an optional solution, the optimization unit 1010 includes:

[0254] A seventh acquisition module is used to obtain the gradient of the loss function with respect to the weight matrix, wherein the gradient is used to indicate adjustment of the weight matrix to reduce the error between the lighting baking data and the actual lighting data;

[0255] The update module is used to iteratively update the weight matrix using the gradient until the error between the lighting baking data and the real lighting data meets the convergence condition;

[0256] The determination module is used to determine the configuration corresponding to the weight matrix when the convergence condition is met as the second configuration.

[0257] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0258] As an optional solution, the seventh acquisition module includes:

[0259] A first acquisition submodule is used to obtain the product of the spherical harmonic function basis function, the barycentric coordinate matrix, and the spherical harmonic coefficient matrix to obtain a first value;

[0260] The second acquisition submodule is used to obtain the product of the overall deviation degree and the double direction weight matrix to obtain a second value;

[0261] The third acquisition submodule is used to obtain the product of the first value and the second value to obtain a gradient.

[0262] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0263] As an optional solution, the device further includes:

[0264] a fifth acquiring unit, configured to acquire, in a process of acquiring lighting baked data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration, first baked data of the virtual network model in a first lighting scene based on the at least one lighting probe of the first configuration, wherein the at least one lighting scene includes the first lighting scene, and the lighting baked data includes the first baked data;

[0265] The device also includes:

[0266] The sixth acquisition unit is used to obtain first illumination data of the virtual network model in a first illumination scene in the process of obtaining real illumination data of the virtual network model in at least one illumination scene, wherein the real illumination data includes the first illumination data.

[0267] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0268] As an optional solution, the at least one lighting scene includes a first lighting scene and a second lighting scene.

[0269] The fifth acquisition unit includes:

[0270] an eighth acquisition module, configured to acquire, based on the at least one lighting probe of the first configuration, first baked data and second baked data of the virtual network model in a second lighting scene, wherein the lighting baked data includes the second baked data;

[0271] The sixth acquisition unit includes:

[0272] a ninth acquisition module, configured to acquire the first illumination data and second illumination data of the virtual network model in a second illumination scene, wherein the real illumination data includes the second illumination data;

[0273] The fourth obtaining unit 1008 includes:

[0274] a tenth obtaining module, configured to obtain a first error between the first baking data and the first lighting data;

[0275] an eleventh obtaining module, configured to obtain a second error between the second baking data and the second lighting data;

[0276] The integration module is used to integrate the first error and the second error to obtain a loss function.

[0277] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0278] As an optional solution, the at least one lighting scene includes a third lighting scene and a fourth lighting scene, and the apparatus further includes:

[0279] a construction unit, configured to construct a standard scene before obtaining lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration, wherein the standard scene has a first lighting condition and a second lighting condition;

[0280] a simulation unit configured to simulate a first appearance of the virtual mesh model under a first lighting condition and a second appearance of the virtual mesh model under a second lighting condition by rotating the virtual mesh model in a standard scene before obtaining lighting baking data of the virtual mesh model under at least one lighting scene based on the at least one lighting probe of the first configuration;

[0281] A representation unit is configured to determine the first representation as the representation of the virtual mesh model in a third lighting scene and the second representation as the representation of the virtual mesh model in a fourth lighting scene before obtaining lighting baking data of the virtual mesh model in at least one lighting scene based on at least one lighting probe of the first configuration.

[0282] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0283] As an optional solution, the device further includes:

[0284] a seventh acquisition unit, configured to acquire at least two sampling points on the virtual grid model before acquiring the first configuration corresponding to at least one lighting probe associated with the virtual grid model;

[0285] The configuration unit is configured to configure the at least one lighting probe associated with the virtual grid model based on at least two sampling points to obtain the first configuration before obtaining the first configuration corresponding to the at least one lighting probe associated with the virtual grid model.

[0286] For specific embodiments, reference may be made to the examples shown in the above-mentioned method for optimizing the configuration of illumination probes, which will not be described in detail in this example.

[0287] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned configuration optimization method of the illumination probe is also provided. The electronic device can be, but is not limited to, Figure 1 The user device 102 or server 112 shown in FIG. 1 is used as an example to illustrate the embodiment. Figure 11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the above method embodiments through the computer program.

[0288] In an optional embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0289] In an optional embodiment, the processor may be configured to execute the following steps via a computer program:

[0290] S8-1, obtaining a first configuration corresponding to at least one Light Probe associated with the virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and Light Probes in the at least one Light Probe obtained using a geometric prior;

[0291] S8-2, based on the at least one lighting probe of the first configuration, obtaining lighting baking data of the virtual network model in at least one lighting scene;

[0292] S8-3, obtaining real lighting data of the virtual network model in at least one lighting scene;

[0293] S8-4, obtaining a loss function corresponding to the first configuration, where the loss function is used to represent the error between the lighting baking data and the actual lighting data;

[0294] S8-5, optimize the first configuration by minimizing the loss function to obtain the second configuration.

[0295] Alternatively, those skilled in the art will appreciate that Figure 11 The structure shown is for illustration only. Figure 11 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 11 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 11 Different configurations shown.

[0296] Among them, the memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the configuration optimization method and device of the lighting probe in the embodiment of the present application. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, that is, realizes the above-mentioned configuration optimization method of the lighting probe. The memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include a memory remotely located relative to the processor 1104, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1102 can be used to store, but is not limited to, information such as the first configuration, lighting baking data, real lighting data, and the second configuration. As an example, such as Figure 11 As shown, the memory 1102 may include, but is not limited to, the first acquisition unit 1002, the second acquisition unit 1004, the third acquisition unit 1006, the fourth acquisition unit 1008, and the optimization unit 1010 in the configuration optimization device for the light probe. Furthermore, the memory 1102 may also include, but is not limited to, other module units in the configuration optimization device for the light probe, which will not be described in detail in this example.

[0297] Optionally, the transmission device 1106 is configured to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one embodiment, the transmission device 1106 includes a network interface controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In one embodiment, the transmission device 1106 is a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0298] In addition, the electronic device further includes: a display 1108 for displaying information such as the first configuration, lighting baking data, real lighting data, and the second configuration; and a connection bus 1110 for connecting various module components in the electronic device.

[0299] In other embodiments, the user device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting multiple nodes via network communication. The nodes may form a peer-to-peer network, and any computing device, such as a server, user device, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0300] According to one aspect of the present application, a computer program product is provided, comprising a computer program / instructions containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium. When the computer program is executed by a central processing unit, the various functions provided in the embodiments of the present application are performed.

[0301] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0302] It should be noted that the computer system of the electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0303] A computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from the storage unit into random access memory (RAM). The RAM also stores various programs and data required for system operation. The CPU, the read-only memory, and the RAM are connected to each other via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.

[0304] The following components are connected to the input / output interface: an input section including a keyboard, mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section including a hard disk; and a communication section including a network interface card such as a local area network card and a modem. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc. are installed in the drive as needed so that computer programs read from them can be installed into the storage section as needed.

[0305] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program is executed by a central processing unit, the various functions defined in the system of the present application are performed.

[0306] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations described above.

[0307] In an optional embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0308] S8-1, obtaining a first configuration corresponding to at least one Light Probe associated with the virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and Light Probes in the at least one Light Probe obtained using a geometric prior;

[0309] S8-2, based on the at least one lighting probe of the first configuration, obtaining lighting baking data of the virtual network model in at least one lighting scene;

[0310] S8-3, obtaining real lighting data of the virtual network model in at least one lighting scene;

[0311] S8-4, obtaining a loss function corresponding to the first configuration, where the loss function is used to represent the error between the lighting baking data and the actual lighting data;

[0312] S8-5, optimize the first configuration by minimizing the loss function to obtain the second configuration.

[0313] Alternatively, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0314] In an optional embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the electronic device through a program, and the program may be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0315] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0316] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0317] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0318] In the several embodiments provided in this application, it should be understood that the disclosed user equipment can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0319] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0320] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0321] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for optimizing the configuration of illumination probes, characterized in that: include: Obtaining a first configuration corresponding to at least one Light Probe associated with a virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and Light Probes in the at least one Light Probe obtained using a geometric prior; Obtaining lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration; Acquiring real lighting data of the virtual network model in the at least one lighting scene; Obtaining a loss function corresponding to the first configuration, wherein the loss function is used to represent an error between the lighting baking data and the real lighting data; The first configuration is optimized by minimizing the loss function to obtain a second configuration.

2. The method according to claim 1, characterized in that The obtaining of the loss function corresponding to the first configuration includes: quantifying the overall deviation between the lighting baking data and the real lighting data to obtain a lighting error; The illumination errors in different directions or positions are differentially weighted using a directional weight matrix to obtain the loss function, wherein the directional weight matrix is used to adjust the importance of illumination in each direction in the loss function.

3. The method according to claim 2, characterized in that Before quantifying the overall deviation between the lighting baking data and the real lighting data to obtain the lighting error, the method may include: Obtaining a spherical harmonic basis function obtained by convolving the lighting baking data, wherein the spherical harmonic basis function is used to encode lighting direction information into mathematical coefficients; Obtaining a barycentric coordinate matrix, wherein the barycentric coordinate matrix is used to represent the lighting interpolation between the vertices; Obtaining a weight matrix, wherein the weight matrix is used to represent the association relationship corresponding to the first configuration, and the weight matrix allows adjustment during the process of minimizing the loss function; Obtaining a baked spherical harmonic coefficient matrix, wherein the spherical harmonic coefficient matrix is used to store lighting information precomputed by the at least one lighting probe; Obtaining the product of the spherical harmonic function basis function, the barycentric coordinate matrix, the weight matrix, and the spherical harmonic coefficient matrix to obtain predicted lighting data; The difference between the predicted illumination data and the actual illumination data is obtained to obtain the overall deviation degree.

4. The method according to claim 3, characterized in that The step of optimizing the first configuration by minimizing the loss function to obtain the second configuration includes: Obtaining a gradient of the loss function with respect to the weight matrix, wherein the gradient is used to indicate adjusting the weight matrix to reduce an error between the lighting baking data and the real lighting data; Iteratively updating the weight matrix using the gradient until the error between the lighting baking data and the real lighting data meets a convergence condition; The configuration corresponding to the weight matrix when the convergence condition is satisfied is determined as the second configuration.

5. The method according to claim 4, characterized in that The obtaining of the gradient of the loss function with respect to the weight matrix includes: Obtaining the product of the spherical harmonic function basis function, the barycentric coordinate matrix, and the spherical harmonic coefficient matrix to obtain a first value; Obtaining the product of the overall deviation degree and twice the directional weight matrix to obtain a second value; The product of the first value and the second value is obtained to obtain the gradient.

6. The method according to claim 1, characterized in that In the process of obtaining lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration, the method further includes: obtaining, based on the at least one lighting probe of the first configuration, first baked data of the virtual network model in a first lighting scene, wherein the at least one lighting scene includes the first lighting scene, and the lighting baked data includes the first baked data; In the process of obtaining real lighting data of the virtual network model in the at least one lighting scene, the method further includes: Acquire first illumination data of the virtual network model in the first illumination scene, wherein the real illumination data includes the first illumination data.

7. The method according to claim 6, characterized in that The at least one lighting scene includes the first lighting scene and the second lighting scene, The obtaining, based on the at least one lighting probe of the first configuration, lighting baking data of the virtual network model in at least one lighting scene includes: Obtaining, based on the at least one light probe of the first configuration, the first baked data and second baked data of the virtual network model in the second lighting scene, wherein the lighting baked data includes the second baked data; The obtaining of real lighting data of the virtual network model in the at least one lighting scene includes: Acquire the first lighting data and second lighting data of the virtual network model in the second lighting scene, wherein the real lighting data includes the second lighting data; The obtaining of the loss function corresponding to the first configuration includes: Obtaining a first error between the first baking data and the first lighting data; Obtaining a second error between the second baking data and the second lighting data; The first error and the second error are integrated to obtain the loss function.

8. The method according to any one of claims 1 to 7, characterized in that The at least one lighting scene includes a third lighting scene and a fourth scene. Before obtaining, by the at least one lighting probe based on the first configuration, lighting baking data of the virtual network model in the at least one lighting scene, the method further includes: Constructing a standard scene, wherein the standard scene has a first lighting condition and a second lighting condition; Simulating a first appearance of the virtual grid model under the first lighting condition and a second appearance of the virtual grid model under the second lighting condition by rotating the virtual grid model in the standard scene; The first representation is determined as a representation of the virtual mesh model in the third lighting scene, and the second representation is determined as a representation of the virtual mesh model in the fourth lighting scene.

9. The method according to any one of claims 1 to 7, characterized in that Before obtaining the first configuration corresponding to at least one light probe associated with the virtual grid model, the method further includes: Obtain at least two sampling points on the virtual grid model; The at least one illumination probe is configured based on the at least two sampling points to obtain the first configuration.

10. A configuration optimization device for an illumination probe, characterized in that: include: a first obtaining unit, configured to obtain a first configuration corresponding to at least one light probe associated with a virtual mesh model, wherein the first configuration is an association relationship between vertices in the virtual mesh model and light probes in the at least one light probe, obtained using a geometric prior; A second acquisition unit is configured to acquire lighting baking data of the virtual network model in at least one lighting scene based on the at least one lighting probe of the first configuration; A third acquisition unit is configured to acquire real lighting data of the virtual network model in the at least one lighting scene; a fourth acquiring unit, configured to acquire a loss function corresponding to the first configuration, wherein the loss function is used to represent an error between the lighting baking data and the real lighting data; An optimization unit is used to optimize the first configuration by minimizing the loss function to obtain a second configuration.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 9 when executed by an electronic device.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 9 through the computer program.

Images