Virtual scene illumination layout method and device, electronic equipment and storage medium

Abstract

The invention discloses a virtual scene illumination layout method and device, electronic equipment and a storage medium, and the method comprises the steps: extracting scene features of a target virtual scene according to the scene data of the target virtual scene; acquiring an illumination constraint condition input under at least one illumination constraint type for the target virtual scene; based on the illumination constraint condition, constructing an illumination constraint function in the target virtual scene; performing noise prediction on the noisy illumination layout scheme by taking scene features as conditions of a diffusion model to obtain a predicted noise quantity; based on each illumination constraint function and a weight parameter for each illumination constraint function, correcting the predicted noise quantity to obtain a corrected noise quantity; and performing iterative denoising on the noisy illumination layout scheme based on the corrected noise amount until a denoising ending condition is met, thereby obtaining a first illumination layout scheme. In this way, an illumination layout scheme which is consistent with the scene structure and high in availability can be obtained, and the efficiency of illumination layout of the virtual scene is improved.

Description

Technical Field

[0001] This disclosure relates to the field of game technology, specifically to a virtual scene lighting layout method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, artificial intelligence (AI) technology is increasingly being applied in fields such as game development, virtual reality, and film and television production, generating corresponding scene images or lighting effects simply by inputting text. Among related technologies, AI can generate basic lightmaps or perform simple style transfer based on text descriptions. However, the resulting lighting layout often fails to meet user expectations, easily leading to problems such as unreasonable shadows, misaligned highlights, or missing light sources, resulting in poor quality. Users (such as designers) still need to invest significant time in manual adjustments and corrections. Summary of the Invention

[0003] This application provides a virtual scene lighting layout method, apparatus, electronic device, and storage medium, which can obtain a lighting layout scheme with high satisfaction and usability, and improve the efficiency of virtual scene lighting layout.

[0004] In a first aspect, embodiments of this application provide a virtual scene lighting layout method, the method comprising: Based on the scene data of the target virtual scene, extract the scene features of the target virtual scene; Obtain the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; Based on the lighting constraints, a lighting constraint function is constructed for the target virtual scene. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. Using the scene features as conditions for the diffusion model, noise prediction is performed on the noisy lighting layout scheme to obtain the predicted noise level. Based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions, the predicted noise is corrected to obtain the corrected noise. Based on the corrected noise level, the noisy lighting layout scheme is iteratively denoised until the denoising termination condition is met, resulting in the first lighting layout scheme.

[0005] Secondly, embodiments of this application provide a virtual scene lighting layout device, the device comprising: The extraction module is used to extract scene features of the target virtual scene based on scene data of the target virtual scene; The acquisition module is used to acquire the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; A construction module is used to construct a lighting constraint function for the target virtual scene based on the lighting constraints. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. The module is used to predict the noise level of a noisy lighting layout scheme by using the scene features as conditions for the diffusion model. Based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions, the predicted noise is corrected to obtain the corrected noise. Based on the corrected noise level, the noisy lighting layout scheme is iteratively denoised until the denoising termination condition is met, resulting in the first lighting layout scheme.

[0006] Thirdly, embodiments of this application also provide an electronic device, including a memory storing multiple instructions; a processor loading instructions from the memory to execute the steps of any of the virtual scene lighting layout methods provided in embodiments of this application.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps of any of the virtual scene lighting layout methods provided in embodiments of this application.

[0008] The solution adopted in this application extracts scene features from scene data during the generation process and uses them as conditional inputs to the diffusion model. This allows the diffusion model to fully understand the scene context during denoising iterations, thereby obtaining a first lighting layout scheme that conforms to the virtual scene content and effectively avoiding common problems such as light source clipping and shadow misalignment. Simultaneously, noise correction based on lighting constraints is embedded in the iterative denoising process. For various types of lighting constraints input by the user (such as artistic intent, gameplay requirements, performance indicators, etc.), corresponding lighting constraint functions are constructed, and the predicted noise is dynamically adjusted based on the weight parameters of each lighting constraint function. This guides the generation process towards convergence in a direction that satisfies multi-dimensional lighting constraints, improving the degree to which the final first lighting layout scheme meets the lighting constraints. Compared to traditional lighting design methods that rely on repeated trial and error or numerical optimization, this effectively reduces the cost of manual intervention, yields a lighting layout scheme with high realism and usability, and effectively improves the lighting layout efficiency of virtual scenes. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a scene diagram of the virtual scene lighting layout system provided in the embodiments of this application; Figure 2 This is a schematic flowchart of an embodiment of the virtual scene lighting layout method provided in this application. Figure 3 This is an exemplary schematic diagram of scene feature extraction provided in an embodiment of this application; Figure 4 This is another exemplary schematic diagram of determining the illumination constraint function provided in the embodiments of this application; Figure 5 This is an exemplary schematic diagram of the optimization process provided in an embodiment of this application; Figure 6 This is another exemplary schematic diagram of the optimization process provided in the embodiments of this application; Figure 7 This is an exemplary schematic diagram of a lighting layout scheme provided in an embodiment of this application; Figure 8 This is another exemplary schematic diagram of the lighting layout scheme provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the virtual scene lighting layout device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. Furthermore, in the description of the embodiments of this application, the terms "first," "second," etc., are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance. Therefore, features defined with "first" or "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0012] This application provides a virtual scene lighting layout method, apparatus, electronic device, and computer-readable storage medium. Specifically, this embodiment will be described from the perspective of a virtual scene lighting layout apparatus, which can be integrated into an electronic device. That is, the virtual scene lighting layout method of this application embodiment can be executed by an electronic device. Optionally, the electronic device may include a terminal device. The terminal device may be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, game console, or personal computer (PC), etc.

[0013] The virtual scene lighting layout method provided in this application can be applied to virtual scene lighting layout systems. This virtual scene lighting layout system may include a terminal device and a server. The terminal can be a device that includes both receiving and transmitting hardware, i.e., a device with receiving and transmitting hardware capable of performing bidirectional communication over a bidirectional communication link. The terminal device and the server can communicate bidirectionally via a network.

[0014] Optionally, the server can be a standalone server, or a network of servers or a server cluster, including but not limited to computers, network hosts, single network servers, multiple sets of network servers, or cloud servers composed of multiple servers. Cloud servers consist of a large number of computers or network servers based on cloud computing.

[0015] The virtual scene lighting layout method in one embodiment of this disclosure can run on a local terminal device or a server. When the virtual scene lighting layout method runs on a server, the method can be implemented and executed according to a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0016] For example, when this virtual scene lighting layout method runs on a terminal, the terminal device stores an application and uses it to render the virtual scene in the scene display. The terminal device is used to interact with the user through a graphical user interface (GUI), such as by downloading, installing, and running the application. The way the terminal device provides the GUI to the user can be varied; for example, it can be rendered and displayed on the terminal device's screen, or it can present the GUI through holographic projection. For instance, the terminal device can include a touchscreen display and a processor. The touchscreen display is used to present the GUI and receive user input commands generated by the GUI, which includes scene displays. The processor is used to run the game, generate the GUI, respond to input commands, and control the display of the GUI on the touchscreen display.

[0017] For example, when this virtual scene lighting layout method runs on a server, the application's main operation and the scene rendering are separate. The storage and execution of the virtual scene lighting layout method are completed on the cloud gaming server. Scene rendering, however, is done on the application's client. The client is primarily used for receiving and sending game data and presenting the scene. For example, the client could be a display device with data transmission capabilities located close to the user, such as a mobile terminal, television, computer, PDA, or personal digital assistant. However, the terminal device for data processing is the cloud server. When the application runs, the user operates the client to send operation commands to the server. The server runs the application according to the operation commands, encodes and compresses scene data, returns it to the client via the network, and finally, the client decodes and outputs the scene.

[0018] Please see Figure 1 , Figure 1 This is a scene diagram illustrating a virtual scene lighting layout system provided in an embodiment of this application. The system may include at least one terminal, at least one server, at least one database, and a network. A user's terminal can connect to different game servers via the network. The terminal is any device with computing hardware capable of supporting and executing software products corresponding to the game. Furthermore, when the system includes multiple terminals, multiple servers, and multiple networks, different terminals can connect to each other through different networks and servers. The network can be a wireless network or a wired network, such as a wireless local area network (WLAN), local area network (LAN), cellular network, 2G network, 3G network, 4G network, 5G network, etc. Additionally, different terminals can also connect to other terminals or servers using their own Bluetooth networks or hotspot networks. For example, multiple users can connect online through different terminals via appropriate networks and synchronize with each other to support multiplayer games. Furthermore, the system may include multiple databases coupled to different servers, and can continuously store game environment-related information in the databases while different users are playing multiplayer games online.

[0019] It should be noted that, Figure 1 The scene diagram of the virtual scene lighting layout system shown is merely an example. The virtual scene lighting layout system and scene described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0020] This application provides a virtual scene lighting layout method, which can be executed by a terminal or a server. This application uses the execution of the virtual scene lighting layout method by a terminal as an example for illustration. The following detailed description is provided in conjunction with the accompanying drawings. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0021] Please see Figure 2 , Figure 2 This is a flowchart illustrating a virtual scene lighting layout method provided in an embodiment of this application. The specific process of the virtual scene lighting layout method can be summarized in steps 210 to 260 as follows: Step 210: Extract scene features of the target virtual scene based on the scene data of the target virtual scene.

[0022] The target virtual scene is the virtual scene for which a lighting layout scheme is to be generated. For example, the target virtual scene may include, but is not limited to, a level map in a game, an interactive environment in a virtual reality application, or multimodal data such as text description information related to the target virtual scene. The target virtual scene may be in two-dimensional or three-dimensional form.

[0023] Scene data is a collection of data describing the content of a target virtual scene. Scene data can take the form of structured metadata or unstructured image data. For example, scene data may include, but is not limited to, 3D geometric meshes, spatial bounding volumes, surface normal maps and material property maps, semantic tag maps, location and category information of key objects, preset camera paths, and preset task areas.

[0024] The scene data of the target virtual scene can be multiple structured data used to represent the target virtual scene, which may include at least one of basic geographic data, material texture data, semantic data, and object data.

[0025] In some embodiments, the basic geographic data may include a height map H(x, y) and / or a voxel representation V(x, y, z). The height map H(x, y) is used to represent the elevation of each location (x, y) on the surface of the target virtual scene; the voxel representation V(x, y, z) ∈ {0, 1} is used to describe the spatial occupancy of complex three-dimensional structures (such as caves, cantilevered terrain).

[0026] In another embodiment, the material texture data may include a material ID map Mid(x, y), a texture blending weight map Wtex(x, y, k), a color map C(x, y), and related textures based on physically-based rendering (PBR), such as a normal map N(x, y), a roughness map R(x, y), and a metallicity map M(x, y). The material ID map Mid(x, y) indicates the material type (e.g., grass, rock, water, sand) used at each surface location (x, y); the texture blending weight map Wtex(x, y, k) represents the blending weight of the k-th texture at location (x, y).

[0027] In another embodiment, semantic data may include a semantic label graph Slabel(x, y), etc. The semantic label graph Slabel(x, y) assigns a semantic category (such as safe zone, bunker area, water area, road, resource point, etc.) to each object and surface point.

[0028] In another embodiment, the object data may include a set of static or dynamic objects O={o1, o2, ..., on} that are not terrain, where each object oi may contain attribute information such as type, three-dimensional position, orientation, state, and collision volume. The type may include vegetation, building debris, vehicles, etc.

[0029] Scene features can be data used to characterize the content characteristics of a target virtual scene. For example, scene features can be vectors or feature maps corresponding to the target virtual scene.

[0030] In one specific implementation, feature encoding or feature extraction can be performed on each of the above-mentioned structured data to obtain scene features. For example, convolutional neural networks (CNNs) or graph neural networks (GNNs) can be used to encode raster or graph structured data such as height maps and semantic label maps to obtain corresponding local feature vectors and global feature vectors.

[0031] In some embodiments, based on scene data of the target virtual scene, at least one scene feature of the target virtual scene is extracted, including: Based on the scene data of the target virtual scene, obtain the object information of virtual objects in the target virtual scene; Based on the object information of virtual objects, extract the object features of virtual objects. The object features include at least one of geometric feature vectors, material feature vectors, and semantic feature vectors. The object features of virtual objects are combined to obtain at least one scene feature of the target virtual scene.

[0032] Virtual objects refer to objects that exist within a target virtual scene. Virtual objects can be static objects, such as buildings and vegetation, or dynamic objects, such as character models and vehicles. Object information includes, but is not limited to, object type (e.g., tree, car, building), 3D position coordinates (x, y, z), orientation (azimuth and pitch angles), size (length, width, height, or radius), material properties (surface reflectivity, roughness, etc.), semantic tags (e.g., obstacles, cover, interactive objects), and behavioral state (for dynamic objects, such as speed, acceleration, etc.).

[0033] It should be understood that the types of information contained in the object information may include more or fewer than those shown above, and this application does not limit this.

[0034] Object features include at least one or more of the following: geometric feature vectors, material feature vectors, and semantic feature vectors. Geometric feature vectors can be determined based on the spatial distribution parameters of the virtual object, such as volume, surface area, centroid coordinates, and bounding box dimensions. Material feature vectors can be determined based on the surface material properties of the virtual object, such as color, texture pattern, and physical properties (e.g., reflectivity, transparency, and refractive index). Semantic feature vectors can be determined based on the semantic tags of the virtual object and its logical relationships with other objects.

[0035] In some embodiments, the object features of each virtual object are concatenated to obtain at least one scene feature of the entire target virtual scene. In practice, a suitable combination can be selected according to actual needs.

[0036] For example, please refer to Figure 3 ,like Figure 3 As shown, scene data can be obtained through the game engine's interface (API), and object information of all objects in the entire game level or a specified area can be read. In an optional embodiment, the target virtual scene can be spatially divided to traverse each object in the virtual scene, such as using data structures like octree or BSP tree to divide the space of the virtual scene in order to quickly query objects in a specific area.

[0037] In an optional embodiment, feature extraction can be performed on each object information, including: extracting the geometric information (e.g., occlusion relationships, etc.) of each 3D model (mesh) in the virtual scene by traversing it, thereby obtaining a geometric feature vector. Here, vertex position refers to the coordinates of the object model in world space, vertex normals determine the surface orientation of the object model and are the basis for lighting calculations, and occlusion relationships can be pre-calculated using techniques such as voxelization or distance fields to form a global occlusion query map. The target virtual scene can be physically based rendering. The PBR (Precision Rendering) property can be obtained by parsing the PBR attributes of the surface material of each object model to obtain a material feature vector. The PBR attributes include albedo, metallicity, and roughness. Albedo refers to the base color of the object's surface, metallicity refers to the degree to which the object's surface exhibits metallic or dielectric (non-metallic) properties, and roughness refers to the microscopic smoothness of the object's surface. Alternatively, the object's identity and function can be identified by using the object tags, layers, or naming conventions built into the game engine. For example, the system can identify player characters, enemies, interactive props, mission objective guides, dangerous areas, etc., and encode the semantic information corresponding to the objects into semantic tags that can be processed by AI models to obtain a semantic feature vector.

[0038] Ultimately, the entire target virtual scene S is represented as a set consisting of multiple objects 0i. Each object corresponds to a feature vector containing the three types of information mentioned above. For example, the entire target virtual scene can be defined as a set S = {o1, o2, ..., on}, where n is the total number of objects in the scene.

[0039] For any object i in the virtual scene, its corresponding feature vector can be represented as Fi, Fi = (Gi, Mi, Li), where: Gi is the geometric feature vector of the object, representing the set of all vertex information of the object, Gi = {vj, nj} = 1, where vj is the vertex coordinate and nj is the vertex normal. Furthermore, a global occlusion function V(p1, p2) ∈ {0, 1} is calculated to determine whether two points p1 and p2 in space are mutually visible. Mi is the material feature vector of the object. For a point p on a surface, its material can be represented as: Mi(p) = (A(p), R(p), T(p)), representing the albedo, roughness, and metallicity of the point, respectively. Li is the semantic label of the object, which can be a one-hot encoded vector, for example, Li = [0, 1, 0, ..., 0] represents the object as an "enemy".

[0040] For example, the feature vector of the entire target virtual scene can be represented as S, which includes the set of feature vectors of all objects in the target virtual scene: S = {F1, F2, ..., Fn}.

[0041] Step 220: Obtain the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene.

[0042] Lighting constraint types are a set of preset categories used to classify lighting design intentions, reflecting the different requirements of the target virtual scene for lighting effects in dimensions such as artistic intent, gameplay needs, or performance indicators. For example, lighting constraint types include, but are not limited to, global illumination, light source direction, and area illumination constraints. Each lighting constraint type corresponds to a set of configurable, parameterized constraint rules used to guide the subsequent generation or adjustment of lighting layout schemes.

[0043] Secondly, lighting constraints can be expressed in natural language, numerical parameters, reference images, lighting probe data, or by specifying control points and regions through a graphical interface. For example, under global illumination lighting constraints, the constraints can include the overall ambient light value (such as a floating-point number in the range of 0.3–0.8); under area illumination lighting constraints, semantic tags of the virtual scene (such as "high-contrast lighting is required for combat zones" and "soft and uniform lighting is required for safe zones") can be combined to specify differentiated light source parameters for different functional areas.

[0044] In some embodiments, the lighting constraint types include at least one of the following: global lighting constraints on the target virtual scene, lighting constraints on a portion of the target virtual scene, lighting constraints on a specified virtual object in the target virtual scene, lighting constraints on a path in the target virtual scene, lighting constraints on a specified viewing angle, and constraints related to light source rendering resources.

[0045] For example, please refer to Figure 4 ,like Figure 4As shown, through a user graphical interface, designers can translate their artistic intentions (e.g., eerie atmosphere), gameplay requirements (e.g., highlighting task props), and performance metrics (e.g., up to 8 dynamic light sources) into formalized lighting constraint functions that machines can understand and execute. For example, a Constraint Definition Language (CDL) allows users to describe lighting requirements declaratively. Specifically, the CDL allows users to directly declare the "desired lighting effect" through high-level statements. For instance, a user can write a specific CDL statement declaring: in the area with the semantic label "combat_zone," the illuminance should be no less than 0.6, and the color temperature should be 4500K. After receiving such user input, this statement can be parsed into structured lighting constraints and mapped to the aforementioned lighting constraint types (e.g., "lighting constraint type for a specific area") and the corresponding parameter space.

[0046] Users (e.g., designers) can define a series of lighting constraint types that apply to a virtual scene through an editor interface or script. Multiple lighting constraint types can include: Constraints on global illumination of the target virtual scene can be simply referred to as global constraints: they apply to the entire target virtual scene. For example, if the user sets the overall atmosphere of the target virtual scene to "gloomy," the system can resolve it as low-saturation, high-contrast, and cool-toned light source parameters; or if the user sets the time of day of the target virtual scene to "dusk," the system can resolve it as a low-angle, warm-toned main light source.

[0047] Constraints on lighting in specific areas of a target virtual scene can be simply referred to as region constraints: these constraints apply to specific areas within the target virtual scene. For example, a user can select a region and define its attributes, such as requiring bright, soft lighting for a "safe zone" and dim, flickering lighting with warning colors (such as red) for a "danger zone".

[0048] Lighting constraints on specified virtual objects in a target virtual scene can be simply referred to as object constraints: these apply to virtual objects with specific semantic tags. For example, a rule can be defined: all "quest item" objects must have a light source within a 3-meter radius around them, and this light source must illuminate their front, creating a highlight effect to attract the player's attention.

[0049] The constraint on lighting of paths in a target virtual scene can be simply called a path constraint: it applies to one or more paths. For example, a user can specify a path from point A to point B (such as the player's main quest path) and require that "the average illuminance on this path must not be lower than a certain threshold" to ensure that the path is clearly visible.

[0050] Lighting constraints on a specified viewing angle can be simply referred to as visibility and composition constraints: these apply to a virtual scene viewed from a specific camera's perspective. For example, a key viewpoint camera can be placed, and it can be required that within this camera's field of view, the outline of the main character be illuminated by a rimlight with a brightness of at least X, to enhance artistic expression.

[0051] Constraints related to lighting rendering resources can be simply referred to as performance constraints: hard metrics used to ensure smooth gameplay. For example, performance constraints may include a maximum of 8 light sources in the target virtual scene and a maximum total area of ​​influence of light sources exceeding Y% of the screen space.

[0052] It should be noted that the above-mentioned lighting constraint types can be used independently or in combination. The system can activate the corresponding lighting constraint type based on user input and map it to the corresponding parameterized representation of the lighting constraint function. By compiling user-defined, readable lighting constraints into a series of mathematical lighting constraint functions, these functions are used to guide the generation process of the diffusion model in the next step.

[0053] Step 230: Based on the lighting constraints, construct the lighting constraint function for the target virtual scene. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints.

[0054] For example, the set of lighting constraints input under the lighting constraint type can be represented as C = {c1, c2, ..., cm}. Each constraint cj is a lighting constraint function. The input of the lighting constraint function is the lighting layout scheme L, and the output is a scalar value representing the degree to which the lighting layout scheme satisfies the lighting constraint (e.g., 0 indicates complete satisfaction, and a positive value indicates a degree of non-satisfaction). The lighting layout scheme L is the set of attribute parameters L = {l1, l2, ..., lp} of all light sources lk in the virtual scene, where the parameters of each light source lk include attribute parameters such as position, color, and intensity.

[0055] The lighting constraint function for object constraints can be determined by the object's visibility: for a target object oi, its visibility can be expressed by formula (1): (1) Where, θ vis Here, A is the desired minimum visibility threshold, I(L, p) is the surface area of ​​the object, and n is the incident light produced by the lighting layout L at point p on the surface. p It is the normal to point p, V pIt is the vector pointing from point p to the camera. Formula (1) calculates the total illumination of the virtual object toward the camera. If the total illumination is not lower than the corresponding minimum visibility threshold, the value of the illumination constraint function is 0, indicating that the illumination requirements are fully met; if the total illumination is lower than the corresponding minimum visibility threshold, the value of the illumination constraint function is positive, and its magnitude reflects the degree of non-compliance.

[0056] The lighting constraint function for a path constraint can be determined by the average brightness of the path: For a path p, its average brightness can be expressed by formula (2): (2) Where, θ path is the minimum average brightness threshold of the path, |p| is the path length, and B(L,p) is the brightness of point p on the path under the lighting layout scheme L. Formula (2) calculates the average brightness of path p under the lighting layout scheme L. If the average brightness is not lower than the minimum average brightness threshold of the corresponding path, the value of the lighting constraint function is 0, indicating that the lighting requirements are fully met; otherwise, if the average brightness is lower than the minimum average brightness threshold of the corresponding path, the value of the lighting constraint function is positive, and its magnitude reflects the degree of non-compliance.

[0057] The lighting constraint function for performance constraints can be determined by the number of light sources, as shown in formula (3): cperf(L)=max(0, |L|—Nmax); (3) Where |L| is the total number of light sources in the lighting layout scheme, and Nmax is the maximum allowed number of light sources. Formula (3) calculates the degree to which the number of light sources in the lighting layout scheme L exceeds the maximum allowed number of light sources. If the total number of light sources |L| does not exceed the maximum allowed number of light sources Nmax, the value of the lighting constraint function is 0, indicating that the performance constraint is met; otherwise, if the total number of light sources |L| exceeds the maximum allowed number of light sources Nmax, the value of the lighting constraint function is positive, and its magnitude is equal to the number of light sources exceeding the limit, reflecting the degree of non-compliance with the performance constraint.

[0058] The global lighting constraint function is composed of a weighted combination of multiple sub-functions, which together describe the overall artistic style of the target virtual scene. For example, the global lighting constraint function can be determined by formula (4): c_horror(L)=w_brightness*f_brightness(L)+w_contrast*f_contrast(L)+w_color*f_color(L); (4) Among them, w_brightness, w_contrast, and w_color are the weights of each sub-function, which can be determined based on experiments or experience.

[0059] L represents the lighting layout scheme of the target virtual scene. The brightness function describes the overall brightness of the virtual scene. For example, the brightness function is: f_brightness(L) = max(0, AvgLum(L) - θ_horror_lum), where the average brightness of all rendered pixels in the target virtual scene is AvgLum(L), and the ideal average brightness threshold is θ_horror_lum. This brightness function formula calculates the degree to which the average brightness of the scene under the lighting layout scheme L exceeds the ideal average brightness threshold θhorror_lum. If the average brightness of the virtual scene AvgLum(L) does not exceed the ideal average brightness threshold, the value of the brightness function is 0, indicating that the lighting is within the comfortable or expected visual range; if the average brightness is higher than θhorror_lum, the value of the brightness function is positive, and its magnitude is equal to the brightness value of the excess portion, reflecting the degree of non-compliance that may cause user discomfort, glare, or preset effects (such as overexposure, unnatural excessive brightness, etc.).

[0060] The contrast function can be determined based on the luminance histogram of the rendered image of the virtual scene. The luminance histogram exhibits a "high at both ends, low in the middle" shape, meaning a large number of pixels are concentrated in extremely bright and extremely dark areas. For example, the contrast function is expressed as: f_contrast(L) = max(0, θ_horror_contrast - Var(Histogram(L))), where θ_horror_contrast is the minimum contrast threshold. This formula calculates the degree of contrast dissatisfaction in the luminance distribution of the virtual scene under the lighting layout scheme L. Specifically, the overall contrast is quantified by analyzing the luminance histogram Histogram(L) of the virtual scene and calculating its variance Var(Histogram(L)): a larger variance indicates a more dispersed luminance distribution and higher image contrast; a smaller variance indicates a flatter image lacking depth. If the variance of the brightness histogram is not less than θhorror_contrast, i.e., Var(Histogram(L))≥θhorror_contrast, then the value of the contrast function is 0, indicating that the contrast meets the requirements of visual clarity and perceptual richness. If the variance of the brightness histogram is less than θhorror_contrast, then the value of the contrast function is positive, and its magnitude reflects the degree of insufficient contrast, indicating that the lighting layout scheme may cause the scene to be dark, details to be difficult to distinguish, or even cause user discomfort or a decrease in immersion.

[0061] A color function can be determined by calculating the average color temperature of all light sources in a virtual scene, as well as the average hue of the overall image. For example, an acceptable color temperature / hue range, such as TargetColdRange, can be defined for a target hue (e.g., a cool hue). For instance, the function form of the color function is: f_color(L) = Distance((AvgTemp(L), AvgHue(L)), TargetColdRange). This color function formula calculates the deviation between the overall color characteristics produced by the lighting layout scheme L and the preset cool hue target range TargetColdRange. Specifically, by extracting the average color temperature AvgTemp(L) and average hue AvgHue(L) of the virtual scene, they are combined into a two-dimensional color feature point (AvgTemp(L), AvgHue(L)), and then the shortest distance (e.g., Euclidean distance or bounded area distance) from this two-dimensional color feature point to the target cool hue region TargetColdRange is calculated. If the color feature point falls inside or on the boundary of TargetColdRange, the value of the color function is 0, indicating that the color style of the current lighting matches the expected cool color atmosphere (such as for creating a calm, nighttime, or terrifying mood). If the color feature point is outside the target area, the value of the color function is positive, and its magnitude reflects the degree to which the color style deviates from the cool tone. It may lean towards warm colors (such as orange-yellow), have too high saturation, or too low color temperature, thereby destroying the expected visual narrative or emotional experience.

[0062] It should be noted that the thresholds involved in the above functions (such as the minimum visibility threshold θvis, the minimum average brightness threshold θpath, the maximum number of light sources Nmax, the ideal average brightness threshold θhorror_lum, the minimum contrast threshold θhorror_contrast, and the target range for cool tones TargetColdRange, etc.) can all be dynamically set and adjusted according to the lighting constraints input by the user. Specifically, users can input their lighting effect requirements through parameter interfaces, natural language commands, style template selection, or interface interactions.

[0063] By summing the above sub-functions with weights, the overall satisfaction level can be comprehensively and quantitatively evaluated, serving as the lighting constraint function for global constraints.

[0064] The region-constrained lighting constraint function operates over a local area of ​​the virtual scene, which can be a specific 3D volume specified by the user. Taking a user-defined "safe zone" as an example, its region-constrained lighting constraint function c_safe(L, R) applies to the lighting layout scheme L and the 3D region with semantic label R. For instance, the region-constrained lighting constraint function randomly or uniformly selects a large number of sampling points p within the region R of the virtual scene. For each sampling point p, the lighting properties of that point are calculated, and then the average of the lighting properties of all sampling points is calculated.

[0065] The lighting constraint function for a region constraint can be determined by multiple sub-functions. For example, the lighting constraint function for a region constraint can be determined by formula (5): c_safe(L,R)=w_region_lum*f_region_lum(L,R)+w_region_shadow*f_region_shadow(L,R); (5) The region brightness function f_region_lum(L, R) calculates the average brightness of all sampled points within region R, ensuring it is not lower than a safe brightness threshold θ_safe_lum. The function form is: f_region_lum(L, R) = max(0, θ_safe_lum - AvgLum_R(L)). This formula calculates the degree to which the average brightness AvgLumR(L) of a specific region R is lower than the safe brightness threshold θsafe_lum under a given lighting layout scheme L. AvgLumR(L) represents the average brightness value of all sampled points or pixels within region R under the current lighting conditions, used to measure the overall illumination level of that region. If the average brightness of region R is not lower than the safety threshold (i.e., AvgLumR(L)≥θsafe_lum), the value of the region brightness function is 0, indicating that the region is well lit and meets the basic safety visibility requirements (such as avoiding tripping, identifying obstacles, and ensuring safe passage). If the average brightness is lower than the threshold (i.e., AvgLumR(L)<θsafe_lum), the value of the region brightness function is positive, and its magnitude reflects the degree of non-compliance of the safety requirements that may be caused by insufficient lighting in the region.

[0066] The region shadow softness function f_region_shadow(L, R): For each sampling point in shadow within region R, the light source attribute of that sampling point is determined. Generally, the larger the point light source size (or the farther away it is), the softer the shadow edge. The region shadow softness constraint is determined by calculating the average sharpness of the shadow within the region. The functional form of the region shadow softness function is: f_region_shadow(L, R) = max(0, AvgSharpness_R(L) - θ_soft_shadow). This formula calculates the degree to which the average sharpness (i.e., the lack of softness) of the shadow edge in a specific region R exceeds the preset soft shadow threshold θsoft_shadow under the lighting layout scheme L. Here, AvgSharpnessR(L) represents the average sharpness index of all shadow edges or shadow transition areas within region R. It can be quantified by methods such as image gradient, edge detection response intensity, or the reciprocal of the penumbra width. The larger the value of AvgSharpnessR(L), the harsher the shadow and the more abrupt the boundary; the smaller the value of AvgSharpnessR(L), the smoother the shadow transition and the softer the visual appearance. If the average shadow sharpness of region R does not exceed the preset soft shadow threshold (i.e., AvgSharpnessR(L) ≤ θsoft_shadow), the value of the region shadow softness function is 0, indicating that the shadow produced by the current illumination is soft enough and meets the requirements. If the average sharpness is higher than the preset soft shadow threshold (i.e., AvgSharpnessR(L) > θsoft_shadow), the value of the region shadow softness function is positive, and its magnitude reflects the negative effects that excessively harsh shadows may bring, such as: enhancing unrealism and causing visual fatigue.

[0067] The lighting constraint function for visibility and composition constraints is used to analyze the lighting effects of the final rendered virtual image from the perspective of a specific virtual camera. Taking the user's requirement that the target object must be illuminated by rimlight as an example, the lighting constraint function c_rimlight(L, O, Cam) operates on the lighting layout L, the target object O (the main character), and a specific camera Cam, etc.

[0068] First, from the perspective of the camera Cam, a virtual, simplified rendering is performed on the target object O illuminated by light L. The rendering result yields the pixel region of the target object O in the virtual scene, resulting in the contour pixel set P_silhouette of the target object O. Each pixel p in the contour pixel set P_silhouette is traversed. For each pixel p, its corresponding 3D surface point s in the scene is found. All rays illuminating s are examined. For each ray, the ray direction vector L_dir and the gaze direction vector V_dir from point s to the camera are calculated. A characteristic of a satisfactory contour light is defined as follows: the ray direction L_dir and the gaze direction V_dir are almost opposite. That is, the dot product dot(L_dir, -V_dir) is close to 1. A contour light score is calculated for each contour point s using visibility and composition constraints. This contour light score is the sum of the intensities of the rays that best meet the contour light conditions among all rays illuminating s.

[0069] The lighting constraint function for visibility and composition constraints calculates the light at all points along the entire contour line and sets a desired minimum contour light score θ_rimlight. The functional form of the lighting constraint function for visibility and composition constraints can be determined by formula (6): c_rimlight(L,O,Cam)=max(0,θ_rimlight-AvgScore_rim(O,Cam,L)). (6) The formula for the illumination constraint function of visibility and composition constraints calculates the degree to which the average rimlight score AvgScorerim(O, Cam, L) of the target object is lower than the user-defined minimum rimlight score threshold θrimlight, given the lighting layout scheme L, camera position Cam, and target object O. If the average rimlight score of the target object is not lower than the minimum rimlight score (i.e., AvgScorerim(O, Cam, L) ≥ θrimlight), the value of the illumination constraint function of visibility and composition constraints is 0, indicating that the object's outline has been effectively highlighted. If the average rimlight score of the target object is lower than the minimum rimlight score, the value of the illumination constraint function of visibility and composition constraints is positive, and its magnitude quantifies the degree of rimlight non-compliance, which may cause the target object to blend into the background, lack depth, or fail to meet the standards.

[0070] The lighting layout scheme is a set of structured parameters used to describe the configuration of one or more light sources in a target virtual scene. It can be used for rendering or lighting simulation. For example, the lighting layout scheme may include, but is not limited to, the three-dimensional spatial coordinates (x, y, z), luminous flux (unit: lumen), color temperature, emission direction (unit vector or Euler angle), beam angle, attenuation coefficient, light source type (point light source, spotlight, area light source) and other light source parameters.

[0071] Step 240: Using scene features as conditions for the diffusion model, perform noise prediction on the noisy lighting layout scheme to obtain the predicted noise level.

[0072] A diffusion model is a generative deep learning model based on a progressive noise addition and denoising mechanism. It generates new samples that conform to the data distribution by learning to reverse the forward diffusion process. For example, diffusion models may include, but are not limited to, DDPM (Denoising Diffusion Probabilistic Models) and DDIM (Denoising Diffusion Implicit Models). In some embodiments, the backbone structure of the diffusion model can adopt U-Net, Transformer, ResNet or hybrid attention architecture with time step embedding, and achieve controllable generation through conditional input (such as scene features). For example, in the task of generating lighting layout schemes, U-Net with scene features as conditions is used as a denoising network to receive noisy lighting layout schemes and time step encodings, and output the predicted noise level.

[0073] The generation process of the diffusion model is a reverse iterative denoising process, which includes a preset number of time steps, denoted as t=T,T 1,…,1, where T is the maximum number of steps. At any current time step t, a noisy lighting layout scheme refers to the lighting layout scheme at time step t that is superimposed with Gaussian noise. It needs to be clarified that a noisy lighting layout scheme refers to the noisy lighting layout scheme input at the current time step t in the reverse denoising process of the diffusion model, denoted as Lt. Its initial state LT is pure random noise following a standard Gaussian distribution, and the dimension of the initial state LT is consistent with the form of the desired lighting layout scheme (e.g., a structured parameter vector). For example, the initial noisy lighting layout scheme is a tensor of shape (M, d), where M is the number of preset light sources, and d is the parameter dimension of each light source (e.g., 3D position + 1D intensity + 1D color temperature).

[0074] The noisy lighting layout scheme is a representation of the light source parameters in the intermediate state corresponding to step t of the diffusion process, which is superimposed with Gaussian noise. It is used as the input for the denoising iteration of the diffusion model. For example, the noisy lighting layout scheme can be, but is not limited to, a tensor composed of parameters such as the position coordinates of the light source, the light intensity vector, the color temperature value, and the orientation angle that are disturbed by noise. The predicted noise level is an estimate of the original added noise output by the diffusion model under the given conditions of the current noisy lighting layout scheme, time step and scene characteristics. For example, the predicted noise level can be a tensor with the same dimension as the noisy lighting layout scheme.

[0075] In the noise prediction phase of the diffusion model, a pre-trained noise prediction network (typically a U-Net architecture) is invoked to perform the reverse process. This noise prediction network receives three input signals: the noisy lighting layout Lt at the current time step; the current time step t; and scene features.

[0076] In some embodiments, the prediction result output by the noise prediction network is the predicted noise level, denoted as Eθ(Lt, t, Fs), where θ represents the network parameters and Fs represents the scene features. By predicting the noise level, the noise prediction network can calculate a more accurate t in subsequent steps based on the backsampling formula of the diffusion model (such as the DDPM or DDIM update rule). Lt, a noisy lighting layout scheme with 1 time step 1.

[0077] Step 250: Based on each illumination constraint function and its weight parameters, the predicted noise is corrected to obtain the corrected noise.

[0078] Weight parameters are configurable coefficients used to adjust the influence of different lighting constraint functions during noise correction, reflecting the importance and priority of each constraint. For example, weight parameters may include, but are not limited to, global constraints, region constraints, object constraints, path constraints, visibility and graphing constraints, and performance constraints. The weight parameter wj, where wj≥0, indicates the relative importance of the j-th type of lighting constraint in the guidance information. In some embodiments, weight parameters can be manually set by the user through a configuration interface, or determined by the system based on historical data.

[0079] In some embodiments, the correction amount can be calculated based on each illumination constraint function and the weight parameters for each illumination constraint function; the predicted noise amount is then corrected based on the correction amount to obtain the corrected noise amount.

[0080] Specifically, the current time step's illumination layout scheme is first input into each illumination constraint function to calculate the value of each illumination constraint function. Then, the values ​​of the illumination constraint functions are weighted and fused together with the weight parameters corresponding to each illumination constraint function to obtain a correction amount. This correction amount is then linearly or nonlinearly combined with the original prediction noise amount (e.g., through residual addition, summation, etc.) to obtain the corrected noise amount.

[0081] In some implementations, the above correction process can be performed during each iteration of the diffusion model's denoising process to achieve iterative denoising guided by the illumination constraint function. For example, at time step t, the diffusion model first outputs the initial predicted noise amount Eθ(Lt, t, Fs), then the correction amount Δε is calculated using the illumination constraint function and its weight parameters, and finally, Eθ'(Lt, t, Fs, C) = Eθ(Lt, t, Fs) is used. The corrected noise level is obtained in the form of λ·Δε, where λ can be a correction step size or learning rate parameter.

[0082] Step 260: Iteratively denoise the noisy lighting layout scheme based on the corrected noise level until the denoising termination condition is met, and obtain the first lighting layout scheme.

[0083] The denoising termination condition refers to the criterion used to terminate the iterative denoising process. In some embodiments, the denoising termination condition is when the time step decreases to t=0, at which point the output L0 is the final first lighting layout scheme. In other embodiments, the denoising termination condition can be a dynamic criterion, such as: if the change in the noisy lighting layout scheme over several consecutive time steps is lower than a preset threshold, or if the comprehensive evaluation score corresponding to the current noisy lighting layout scheme meets a preset evaluation condition, then the iteration is terminated early to improve computational efficiency. More information about the comprehensive evaluation score can be found in the relevant description below.

[0084] The final first lighting layout scheme is a set of structured light source parameters that can be used to render the target virtual scene.

[0085] In some embodiments of this application, the generation of a high-quality lighting layout scheme from pure noise is achieved by embedding the constrained guided correction noise amount into the standard diffusion reverse process.

[0086] In some embodiments, the noisy lighting layout scheme is iteratively denoised based on the corrected noise level until the denoising termination condition is met, resulting in a first lighting layout scheme, including: The noisy lighting layout scheme is denoised based on the corrected noise level to obtain a new noisy lighting layout scheme. Based on the new noisy lighting layout scheme, the process continues to execute the step of using scene features as the condition for the diffusion model to predict the noise level of the noisy lighting layout scheme and obtain the predicted noise level until the denoising termination condition is met, thus obtaining the first lighting layout scheme.

[0087] The above iterative denoising refers to the process of denoising at time steps t=T,T The process is executed iteratively on time step 1, ..., 1. At each time step, the following flow is executed: Input the noisy lighting layout scheme Lt, time step t, and scene features Fs at the current time step into the noise prediction network to obtain the predicted noise amount Eθ(Lt, t, Fs) at the current time step. Based on each illumination constraint function and its weight parameters, the correction amount is calculated, and the original prediction noise amount is corrected by the correction amount to obtain the corrected noise amount Eθ'(Lt, t, Fs, C). Update Lt to Lt using Eθ'(Lt, t, Fs, C) 1, namely, the new noisy lighting layout scheme; And Lt 1. Input noise prediction network based on time step t-1 and scene features Fs; The first lighting layout scheme is obtained until the denoising termination condition is met.

[0088] The denoising process involves updating the noisy lighting layout scheme of the current time step according to the corrected noise level and a preset diffusion backsampling algorithm, so as to generate a noisy lighting layout scheme of the previous time step that is closer to the real lighting effect. For example, the denoising process may include, but is not limited to, Markov denoising steps based on DDPM, etc., and the embodiments of this application do not limit this.

[0089] The new noisy lighting layout scheme is obtained after denoising and corresponds to time step t. The lighting layout scheme of 1 has a lower noise level than the previous step and is closer to the desired lighting layout scheme. For example, the new noisy lighting layout scheme may include, but is not limited to, a set of light source parameters with the same dimensions as the initial noisy lighting layout scheme but with more reasonable values. In some embodiments, the new noisy lighting layout scheme can be directly used as the input for the next round of noise prediction. For example, after denoising from t=50 to t=49 in time step, the resulting L49 is the new noisy lighting layout scheme.

[0090] It should be noted that the total number of time steps T refers to the discrete step number of the diffusion process from the real lighting layout scheme L0' to the pure random noise LT. Its value can be preset by the user before training the diffusion model, or the total number of time steps can be used to optimize the parameters of the neural network. For example, the total number of time steps T can be 100, 500, 1000, etc. In some embodiments, the total number of time steps T can also be determined through experimental tuning.

[0091] In some embodiments, the predicted noise is corrected based on each illumination constraint function and its weight parameters to obtain the corrected noise, including: Based on each illumination constraint function and its weight parameters, the first loss of the illumination layout scheme corresponding to the predicted noise is determined. Determine the gradient corresponding to the first loss, and correct the predicted noise based on the gradient to obtain the corrected noise.

[0092] In some embodiments, the predicted noise level is an estimate of the noise in the current noisy lighting layout scheme, output by the diffusion model at the current time step t, using the noisy lighting layout scheme Lt, the time step identifier t, and the scene features Fs as input data.

[0093] Therefore, the lighting layout scheme corresponding to the predicted noise level refers to the noisy lighting layout scheme at the current time step.

[0094] In some embodiments, based on each illumination constraint function and its weight parameters, a first loss for the illumination layout scheme corresponding to the predicted noise is determined, including: Based on each illumination constraint function and the illumination layout scheme corresponding to the predicted noise, determine the second loss corresponding to each illumination constraint function; The first loss is determined based on the second loss corresponding to each illumination constraint function and the weight parameters of each illumination constraint function.

[0095] The second loss is the quantized value output by a single lighting constraint function under the noisy lighting layout scheme at the current time step, which is used to characterize the degree of deviation of the noisy lighting layout scheme at the current time step from a specific lighting constraint type.

[0096] Different lighting constraint types correspond to different second losses. For example, the noisy lighting layout scheme Lt at the current time step is input into the lighting constraint function corresponding to each lighting constraint type to obtain the value of the lighting constraint function corresponding to each lighting constraint type, which is used as the second loss corresponding to each lighting constraint function.

[0097] In some embodiments, a weighted linear combination method can be used to multiply the output values ​​of each lighting constraint function by their corresponding weight parameters and then sum them to obtain the first loss. Specifically, there are N types of lighting constraints that are enabled. The noisy lighting layout scheme Lt of the current time step is input into the lighting constraint function corresponding to the j-th constraint type to obtain the value of the j-th lighting constraint function, i.e., Cj(Lt), whose weight parameter is wj. Then the first loss J(Lt) can be expressed as shown in formula (7): J(Lt)=∑wj*Cj(Lt); (7) Where Lt is the noisy lighting layout scheme based on the predicted noise level at the current time step, and wj represents the weight parameter corresponding to the lighting constraint function Cj().

[0098] In some embodiments, the gradient corresponding to the first loss is determined, and the predicted noise is corrected based on the gradient to obtain the corrected noise, including: Determine the gradient of the first loss relative to the lighting layout scheme corresponding to the predicted noise level; The gradient is weighted based on the preset guiding strength parameters to obtain the correction amount of the predicted noise. The predicted noise level is corrected based on the correction amount to obtain the corrected noise level.

[0099] The guidance strength parameter is a configurable scalar used to control the magnitude of the influence of the constraint guidance condition on noise correction. It adjusts the strength of the gradient in the correction process to balance the diversity of results and constraint satisfaction. For example, the guidance strength parameter may include, but is not limited to, a fixed guidance factor s, a dynamic guidance coefficient s(t) that decays with time step, or a scaling factor that is adaptively adjusted based on the value of the illumination constraint function. In some embodiments, the guidance strength parameter can be set by the user in the user interface, or it can be automatically adjusted by the system according to the denoising stage. For example, it can be set to 2.0 in the early high noise stage for strong constraint guidance, and then reduced to 0.5 in the later stage to preserve the details of the changes.

[0100] For example, by employing an improved Guided Diffusion Model, a high-quality first lighting layout scheme that satisfies all defined constraints is gradually generated from a completely random noisy lighting layout scheme in an iterative denoising manner.

[0101] The diffusion model is a generative model, and its process can be divided into two parts: Forward Process: This process is performed during the training phase. It starts with a real, high-quality lighting layout scheme (called L0'), and gradually adds Gaussian noise to it over T time steps until a purely random noise LT is obtained at t=T.

[0102] Reverse Process: This process takes place during the inference (generation) phase. The diffusion model learns how to reverse the above process. Starting with random noise LT, at each time step t, it predicts and removes the noise added in the previous step, thereby progressively recovering a clear and reasonable lighting layout scheme L0, where L0 is the same as or similar to L0'.

[0103] At each step of the reverse process, not only is the diffusion model predicted for the amount of noise, but the gradient of the noisy lighting layout scheme Lt at the current time step is also calculated with respect to all lighting constraint functions Cj. This gradient indicates the guiding information for adjusting Lt to reduce the first loss. The guiding information guides the diffusion model to perform iterative denoising, thereby guiding the entire generation process in a direction that satisfies all lighting constraints.

[0104] For example, a standard diffusion model learns a noise prediction network E(Lt,t) that predicts the noise added to L0' at time step t, i.e., the amount of noise.

[0105] To introduce constraints, a guiding term, or correction term, is added to the prediction at each time step. The first loss function J(L) is the weighted sum of all illumination constraint functions, as shown in Equation (8): (8) Here, wj represents the weight of each constraint, which can be adjusted flexibly.

[0106] In each time step t of the reverse process, the gradient of the function J (also called the first loss) of the first loss with respect to the noisy lighting layout scheme Lt at the current time step is calculated, as shown in Equation (9): gt=▽ Lt J(Lt); (9) The gradient gt is a vector pointing in the direction that can reduce the first loss J(L) the fastest (i.e., satisfy the constraint condition).

[0107] The gradient is fused into the noise prediction of the diffusion model to obtain an estimated value of the guided noise amount, that is, the corrected noise amount, as shown in formula (10): Eθ' (Lt, t, Fs, C) = Eθ (Lt, t, Fs) — γ*gt; (10) Here, γ is the GuidanceScale hyperparameter, which controls the force intensity of the illumination constraint function.

[0108] The diffusion model uses guided noise estimation to calculate the noisy lighting layout scheme Lt-1 of the previous time step, thereby ensuring that the generation direction of the noisy lighting layout scheme of the previous step not only conforms to the general lighting laws learned by the diffusion model, but also tends to meet the specific needs defined by the user.

[0109] In some embodiments, the method further includes: Based on the first lighting layout scheme, determine the evaluation index value of the target virtual scene in at least one evaluation dimension; The comprehensive evaluation score of the first lighting layout scheme is determined based on the evaluation index values ​​corresponding to each evaluation dimension and the preset weights corresponding to each evaluation dimension. Based on the comprehensive evaluation score, the weight parameters of each lighting constraint function are updated to obtain the updated weight parameters of each lighting constraint function. Based on the updated weight parameters of each illumination constraint function, the step of correcting the predicted noise based on each illumination constraint function and its weight parameters is re-executed to obtain the corrected noise, thereby obtaining the second illumination layout scheme. The evaluation dimensions include at least one of the following: interaction dimension, visual dimension, and resource consumption dimension.

[0110] In some embodiments, the target virtual scene can be rendered or simulated based on a first lighting layout scheme, and a corresponding evaluation model can be invoked to evaluate the rendered lighting effect from multiple preset evaluation dimensions to obtain corresponding evaluation index values. For example, the evaluation can be performed from at least one of the following evaluation dimensions: interaction dimension, visual dimension, and resource consumption dimension.

[0111] In some embodiments, the rendered lighting effect can be evaluated to obtain an evaluation index value corresponding to the rendered lighting effect. For example, the rendered lighting effect can be evaluated from multiple evaluation dimensions such as interaction dimension, visual dimension, and resource consumption dimension to obtain at least one of the following: interaction dimension score, visual dimension score, and resource consumption dimension score. In one possible approach, the evaluation index value corresponding to the rendered lighting effect may include: the interaction dimension score, the visual dimension score, and the resource consumption dimension score corresponding to the rendered lighting effect.

[0112] In some embodiments, an evaluation model can be invoked to assess the rendered lighting effect and obtain an evaluation index value corresponding to the rendered lighting effect. The evaluation model can be a machine learning model, language model, etc., used for evaluation.

[0113] In some embodiments, the quality of the rendered lighting effect is determined based on the evaluation index value corresponding to the rendered lighting effect.

[0114] For example, lighting quality conditions can be preset. For instance, the lighting quality conditions could be: the interaction dimension score is greater than the interaction dimension threshold, the visual dimension score is greater than the visual dimension threshold, and the resource consumption dimension score is less than the resource consumption dimension threshold.

[0115] Specifically, it can be determined whether the interaction dimension score corresponding to the rendered lighting effect is greater than the interaction dimension threshold, whether the visual dimension score of the rendered lighting effect is greater than the visual dimension threshold, and whether the resource consumption dimension score corresponding to the rendered lighting effect is greater than the resource consumption dimension threshold. When the interaction dimension score corresponding to the rendered lighting effect is greater than the interaction dimension threshold, the visual dimension score of the rendered lighting effect is greater than the visual dimension threshold, and the resource consumption dimension score corresponding to the rendered lighting effect is less than the resource consumption dimension threshold, it can be determined that the rendered lighting effect meets the lighting effect quality conditions. At this time, the process can be terminated, and the first lighting layout scheme can be returned to the terminal device.

[0116] Otherwise, if it is determined that the rendered lighting effect does not meet the lighting effect quality conditions, the weight parameters corresponding to each lighting constraint type can be readjusted based on the evaluation index value corresponding to the rendered lighting effect.

[0117] For example, a first lighting layout scheme that meets the lighting effect quality conditions can be used as optimization data for training a diffusion model to optimize the diffusion model.

[0118] For example, embodiments of this application can also judge the rendered lighting effect based on multiple preset evaluation dimensions, which can reduce the occurrence of unsafe or unreasonable lighting layout schemes and expand the application scenarios of the lighting layout scheme generation system. For example, the weight parameters of the original lighting constraint function can be dynamically adjusted according to the contribution of each evaluation dimension in the comprehensive evaluation score. For example, if the interaction dimension score is low, such as the weight of the lighting constraint type corresponding to the current "playability function" being low, the weight of that lighting constraint type can be increased; if the resource consumption dimension exceeds the standard, the weight of constraints related to light source rendering resources can be increased. One possible approach is: when the evaluation index value of a certain evaluation dimension is lower than a preset threshold, the weight of the corresponding lighting constraint type is increased proportionally to strengthen the guidance intensity of that lighting constraint type in the next round. Different evaluation dimensions correspond to different lighting constraint functions that need to be adjusted, and their correspondence can be determined based on prior knowledge or historical data.

[0119] Based on the updated weight parameters of each lighting constraint function, the step of correcting the predicted noise based on each lighting constraint function and its weight parameters is re-executed to obtain the corrected noise level, thus yielding the second lighting layout scheme. For example, the overall function can be reconstructed based on the updated weight parameters, and the reverse process of the diffusion model can be restarted: starting with initial noise, noise prediction is performed using scene features as conditions. The predicted noise level is then corrected by combining the updated weight parameters corresponding to each lighting constraint type, resulting in the corrected noise level. Iterative denoising using the corrected noise level continues until a new lighting layout scheme, i.e., the second lighting layout scheme, is obtained. This second lighting layout scheme focuses more on aspects that were insufficiently performed in the previous evaluation in this new round, thereby achieving a targeted improvement in quality.

[0120] In some embodiments, the weight parameters of each lighting constraint function are updated based on the comprehensive evaluation score to obtain the updated weight parameters of each lighting constraint function, including: Based on the comprehensive evaluation score, calculate the gradient of the comprehensive evaluation score with respect to the weight parameters of each lighting constraint function; Based on the gradient and the preset learning rate parameters, the weight parameters of each lighting constraint function are updated to obtain the updated weight parameters of each lighting constraint function.

[0121] For example, after obtaining the first lighting layout scheme, in order to ensure the quality of the final result, the first lighting layout scheme is evaluated and processed, and feedback and optimization are performed based on the evaluation results.

[0122] For example, evaluation dimensions include: gameplay clarity, artistic expression, and performance budget.

[0123] Interaction dimension, also known as gameplay clarity, can be determined based on factors such as path visibility, visual salience, and threat identifiability.

[0124] One possible approach is to simulate a player moving along a preset path and check whether the brightness of key nodes on the path meets the standard to obtain path visibility.

[0125] One possible approach is to calculate the visual salience of all objects marked as task objectives or interactive objects from the player's common perspective, ensuring that they can be easily discovered.

[0126] One possible approach is to check if enemy units have sufficient lighting to determine threat visibility and prevent players from suffering unfair attacks due to poor visibility.

[0127] The visual dimension, also known as artistic expression, can be determined based on contrast, compositional information, and matching with reference styles.

[0128] One possible approach is to analyze the histogram of the rendered image of the virtual scene from a key viewpoint, check whether the contrast of the histogram is appropriate, and whether the hue conforms to the range set by global constraints (such as "gloomy" or "warm") to determine the contrast.

[0129] One possible approach is to use visual algorithms to analyze the visual flow of the image, determine whether the lighting effectively guides the viewer's eye to the center of the composition or key elements, and thus determine the compositional information.

[0130] One possible approach is to use user-provided reference images to calculate the stylistic similarity between the rendered image of the virtual scene corresponding to the lighting layout scheme and the reference image, thereby determining the composition information.

[0131] The resource consumption dimension, also known as the performance budget, can be determined based on the number and type of light sources, lighting complexity, memory usage, etc.

[0132] One possible approach is to count the number of dynamic light sources and shadow-casting light sources to determine if they exceed performance constraints, thereby identifying the light source count and type.

[0133] One possible approach is to analyze the overlapping areas of the light source's influence range (overdraw), assess its potential impact on GPU rendering costs, and determine the lighting complexity.

[0134] One possible approach is to estimate the video memory usage of lightmaps, volumetric lights, etc., and thus determine the memory usage.

[0135] Please see Figure 5 ,like Figure 5 As shown, after the evaluation, an evaluation report will be output, displaying the advantages and disadvantages of the current lighting layout scheme to the user terminal through data and visualization methods such as heat maps. If the evaluation results do not meet the preset lighting quality conditions, two optimization strategies can be adopted: Automatic optimization strategy: The system automatically adjusts the weight parameters of each lighting constraint function (for example, if the path is too dark, increase the weight parameters of the path brightness constraint, etc.), and then re-executes the reverse process of the diffusion model to perform a new round of iterative denoising.

[0136] Manual intervention strategy: The system highlights the problem areas of the target virtual scene to the user and provides specific optimization suggestions (e.g., "Warning: Area A is too dark. It is recommended to add a supplementary light or increase the ambient light intensity of area A"). The system then iterates on the proposed changes after the user confirms the modifications.

[0137] For the automatic optimization strategy, a comprehensive evaluation function sscore(L) is defined to quantify the overall quality of the generated lighting layout scheme L. The value of the comprehensive evaluation function is the comprehensive evaluation score, as shown in formula (11): sscore(L)=wgameplay*vgameplay(L)+wart*vart(L)—wperf*pperf(L); (11) Wherein: vgameplay is the overall score for gameplay clarity. vart is the overall score for artistic presentation. pperf is the penalty score for performance overhead; the higher the overhead, the higher the penalty score. wgameplay, wart, and wperf are the weights of each metric.

[0138] One possible update method is to set the goal of iterative optimization to find a lighting layout scheme L* that maximizes the overall evaluation score while satisfying all hard constraints (such as constraints related to light source rendering resources), as shown in formulas (12) and (13): (12) Subject to cj(L)=o for all hard constraints cj∈Chard; (13) In the automatic optimization loop, a gradient-based optimization algorithm (such as gradient ascent) is used to adjust the weight parameters wj corresponding to each lighting constraint type in order to maximize the comprehensive evaluation score of the lighting layout scheme generated in the next round, as shown in formula (14): wk+1=wk+η*▽ wk *s score (Generate(Fs,C(wk)));(14) Where k is the number of iterations, η is the learning rate, and Generate() represents the entire process of generating the lighting layout scheme.

[0139] In some embodiments, the method further includes: The target virtual scene is rendered using the target lighting layout scheme, which is determined based on the first lighting layout scheme and / or the second lighting layout scheme.

[0140] Rendering a target virtual scene using a target lighting layout scheme means applying the lighting layout results obtained through the aforementioned methods to a graphics rendering engine to perform real-time lighting calculations or offline lighting baking, ultimately completing the rendering of the target virtual scene.

[0141] The first lighting layout scheme refers to the preliminary lighting layout scheme generated for the target virtual scene under the initially set lighting constraints.

[0142] The second lighting layout scheme is an optimized lighting layout scheme output after further evaluation based on the first lighting layout scheme.

[0143] In one implementation scenario, when the first lighting layout scheme meets the lighting effect quality conditions, the target lighting layout scheme is the first lighting layout scheme.

[0144] In another implementation scenario, when the first lighting layout scheme does not meet the lighting effect quality conditions, the generated lighting layout scheme is the second lighting layout scheme.

[0145] In another implementation, the target lighting layout scheme used for rendering can be determined based on a first lighting layout scheme and a second lighting layout scheme. For example, the light source parameters in the first and second lighting layout schemes can be combined in various ways to obtain the target lighting layout scheme, such as weighted averaging. The target lighting layout scheme contains specific parameters for configuring lighting configuration items such as the position, intensity, and type of light sources in the target virtual scene. These specific parameters are passed to the graphics rendering engine for performing real-time lighting calculations or offline lighting baking.

[0146] In some embodiments of this application, please refer to Figure 6 ,like Figure 6 As shown, the first lighting layout scheme generated by the diffusion model will be evaluated through at least one preset evaluation dimension. The evaluation results (such as automatic optimization or manual intervention) are directly used to guide the generation of the next round of lighting layout schemes, thereby ensuring that the system can continuously and automatically converge to an ideal solution that meets the needs of all users, and thus obtain the final lighting layout scheme. The final lighting layout scheme can be applied to the game engine to render the final game screen.

[0147] To better understand the above solutions, we will explain them in detail below using two different types of game scenario examples.

[0148] Case 1: Please refer to Figure 7 ,like Figure 7 The image shows a "corridor" in a linear horror puzzle game.

[0149] Scene Description: A long, narrow corridor, dilapidated and dimly lit. Players need to walk from point A (the entrance) at one end of the corridor to point B (a locked door) at the other end, and find the key item "Key" hidden at point C (in a side ward) along the way. Several enemies (marked "Enemy") may jump out from the corridor.

[0150] Illumination constraints: Artistic intention: To create an atmosphere of oppression, terror, and unease.

[0151] Gameplay requirements: The player's main path A->B must be faintly visible, not completely dark.

[0152] Point C, where the key item is hidden, needs to be subtly illuminated to entice players to explore.

[0153] Enemy areas (such as Enemy 1, Enemy 2, etc.) need to be relatively dark to maximize the scare effect.

[0154] Performance requirements: Since it is an indoor linear barrier, the performance requirements are relatively relaxed, but the number of dynamic light sources should be less than 10.

[0155] After obtaining the above input information, the system automatically parses the scene description, identifies the geometry and material of the corridor walls and floor, and identifies “Player_Start_Point” (A), “Locked_Door” (B), “Key_Item” (C) and multiple “Enemy” objects based on the object labels.

[0156] After receiving the above input information, the system transforms the lighting constraints into lighting constraint functions: The global constraints are interpreted as: low global illumination intensity, high contrast, and cool color tone.

[0157] Path constraints define the minimum brightness along the path from A to B.

[0158] Object constraint 1: There must be a sickly green glow near point C (the key).

[0159] Object constraint 2: Requires all enemies to be virtually invisible in their default positions.

[0160] Performance constraint: Number of dynamic light sources count = 10.

[0161] The initial random light source layout might illuminate the entire corridor. Guided by the lighting constraints corresponding to each lighting constraint type, the light sources gradually decrease and dim, and the overall tone of the virtual scene cools down. To satisfy path constraints, the system might generate one or more relatively weak downlights with limited illumination range on the ceiling, or simulate faint moonlight filtering through a window to illuminate the path.

[0162] During the generation of the lighting layout scheme, to satisfy object constraint 1, the system generates a self-illuminating point light source in the room at point C, simulating the light emitted by a damaged device. To satisfy object constraint 2, the system ensures that all light illuminating enemy areas is blocked by geometry, or that the light intensity in these enemy areas is reduced to near zero. When the verification module runs, it simulates the player's perspective and generates a "gameplay clarity heatmap," showing that the brightness of the A->B path and point C meets the requirements, while the enemy areas remain dark.

[0163] The method described in the embodiments of this application allows users to automatically generate a complex lighting effect that would typically require hours or even days of repeated adjustments and baking by defining one or more high-level rules, perfectly balancing the creation of a terrifying atmosphere and gameplay requirements.

[0164] Case 2: Please refer to Figure 8 ,like Figure 8 As shown, this is an outpost in an open-world game.

[0165] Scene Description: An outpost nestled in a forest, comprising a central tower, several tents, a quest NPC, and a treasure chest hidden behind a waterfall. Players can enter the area from any direction. The time is set at dusk.

[0166] Illumination constraints: Artistic intent: To create a tranquil, mysterious, and fantastical twilight scene.

[0167] Gameplay requirements: The central tower is a regional landmark and needs to be highlighted. The NPCs for the quest are clearly visible and give off a warm and friendly vibe; The treasure chest behind the waterfall needs visual cues to indicate its existence.

[0168] Performance metrics: As part of an open-world game, performance is a primary consideration. Therefore, it is necessary to strictly control the performance overhead of lighting, such as shadow casting.

[0169] After obtaining the above input information, the system automatically parses the scene description and identifies semantic objects such as "Main_Tower", "Tent", "Quest_NPC", "Hidden_Chest" and "Waterfall".

[0170] After receiving the above input information, the system transforms the lighting constraints into lighting constraint functions: Global constraints were applied to create a low-angle, warm orange parallel light as the sun, with a corresponding purple-toned ambient light.

[0171] Object constraint 1 requires outlining the main tower with a golden outline light so that it stands out even in the backlight of dusk.

[0172] The object constraint requires providing a soft, warm white fill light for the quest NPC, ensuring it remains clear and friendly from any angle.

[0173] Object constraint 3 requires the generation of some cyan volumetric lights (GodRays) in the waterfall area, hinting at a secret behind it.

[0174] Performance constraint 1: The number of shadow casting sources, count=1. Only the main light source (sun) is allowed to cast real-time shadows.

[0175] Performance Constraint 2: Optimize Light Source Overlap (Maximum Overlap Percentage = 15%). This constraint limits the overlap of light sources within a given space.

[0176] During the generation of the lighting layout scheme, to meet global constraints, the system first sets the time to "dusk" and establishes basic lighting. To satisfy object constraint 1, i.e., key buildings (such as the main tower), the system generates a spotlight as a contour light above and behind the main tower, facing the main viewing direction, and disables its shadow casting to save performance. To satisfy object constraint 2, i.e., focusing on NPCs, the system generates a low-intensity, large-area point light source in front of the NPCs as fill light, also without casting shadows. To satisfy object constraint 3, i.e., creating a secret cues effect, the system places a spotlight with volumetric lighting effects behind the waterfall, facing outwards; the light passing through the waterfall's particle effects creates a visual cue of "holy light." During the verification module's runtime, the scene is checked from multiple typical entrance directions to ensure the main tower's outline is clear and the NPCs are always visible. Simultaneously, the performance analyzer confirms that there is only one light source casting shadows, and the lighting overlap is less than 15%, meeting the performance budget.

[0177] In the above embodiments, the embodiments of this application can quickly achieve an artistically beautiful twilight atmosphere. More importantly, it can meet the performance requirements of an open world while satisfying multiple complex gameplay needs. It possesses the ability to automatically achieve the optimal balance between artistic intent, gameplay requirements, and performance metrics.

[0178] With the rapid development of modern video games, players have placed higher demands on the realism, immersion, and artistic expression of game worlds. Among the many elements that constitute the visual experience of a game, lighting is crucial. It forms the basis for simulating the real physical world and is a means of guiding player emotions, narrating the story, and highlighting key gameplay elements. However, in the current mainstream AAA game development process, game lighting designers typically need to manually place and finely adjust hundreds or even thousands of light sources in a 3D scene, including main light sources (such as the sun), ambient light, point light sources, spotlights, etc. The color, intensity, radius, attenuation mode, shadow type, and other parameters of each light source need to be repeatedly adjusted and coordinated with other light sources to achieve the desired overall visual effect. However, this approach suffers from long iteration cycles and high trial-and-error costs. Any minor adjustment to the lighting scheme often requires re-executing a time-consuming lighting baking process. Especially in large, high-precision scenes, a single baking process can take several hours or even tens of hours, severely restricting the flexibility of artists to quickly try different lighting layout schemes. Secondly, there is a significant difference between the artistic expression and technical implementation of this approach. Designers usually put forward abstract and emotional needs, such as "creating a gloomy and terrifying atmosphere," "making key props visible at a glance," or "presenting the tranquility of morning mist." Translating these user intentions into specific combinations of light source parameters relies on the designer's professional experience, and there is a lack of efficient tools to achieve a direct mapping from user semantics to light source parameters.

[0179] Furthermore, lighting design requires a delicate balance between several conflicting objectives: it must fulfill the artistic intent, ensure gameplay clarity (such as path recognition and the visibility of interactive objects), and strictly control performance overhead to avoid frame rate drops due to excessive dynamic light sources or complex shadow calculations. Achieving multi-objective coordination is difficult in complex game scenes. Finally, in large-scale projects, maintaining consistency in lighting styles among multiple collaborators is challenging, and changes to scene content often necessitate large-scale rework, lacking scalability and robustness.

[0180] Currently, one approach involves manually placing and adjusting light sources: designers configure each light source individually in the engine editor. Static parts rely on global illumination techniques (such as path tracing or lightmap baking) for pre-calculation, while dynamic parts use real-time light sources. This method is inefficient, slow to iterate, and highly dependent on human experience, making it difficult to achieve systematic optimization between art, gameplay, and performance. Another approach is Image-Based Lighting (IBL), which provides uniform ambient lighting through high dynamic range panoramic images (such as skyboxes), quickly enhancing the overall realism of the scene. However, IBL can only handle macroscopic ambient light and cannot solve the problem of local artificial light source placement or refined artistic lighting (such as contour lighting or dramatic fill lighting) within the scene, still requiring a lot of manual adjustment of light sources. Additionally, some procedural lighting generation tools or plugins are used, such as automatically adjusting sun parameters based on time or batch generating point light sources on specific models. While these tools can handle repetitive tasks, their logic is based on simple rules or templates, lacking an understanding of contextual semantics, artistic style, or gameplay requirements. The generated results are often rough and still require a lot of manual correction. Using deep learning for lighting style transfer or post-processing, such as applying the lighting style of a reference image to the rendered image, only works at the two-dimensional image level. It is a kind of visual filter and cannot generate a realistic three-dimensional light source layout. Therefore, it cannot produce correct spatial lighting interactions such as projection and specular highlights, nor can it support the needs of gameplay based on spatial awareness.

[0181] While related technologies have improved efficiency in some aspects, they have never fundamentally solved the problem of how to automatically transform high-level design intentions, including artistic intentions and gameplay requirements, into an optimized lighting layout that takes into account physical rationality, visual expressiveness, and operational performance.

[0182] The solution adopted in this application addresses the common problem that traditional methods often rely on human experience or simple rules for lighting design, making it difficult to balance physical realism and artistic expression. By leveraging the powerful generative capabilities of the diffusion model and introducing quantifiable lighting constraint functions, the generated lighting layout can both conform to optical laws and satisfy the specific intentions set by the user. During the generation process, scene features are first extracted from the scene data and used as conditional inputs to the diffusion model. This allows the diffusion model to fully understand the scene context during denoising iterations, thereby obtaining light source positions, intensities, and directions that conform to the virtual scene content, effectively avoiding common problems such as light source clipping and shadow misalignment. Meanwhile, noise correction based on lighting constraints is embedded in the iterative denoising process. For various types of lighting constraints input by users (such as artistic intent, gameplay requirements, performance indicators, etc.), corresponding lighting constraint functions are constructed, and the predicted noise is dynamically adjusted in combination with corresponding weight parameters. This can guide the generation process to converge in the direction of satisfying multi-dimensional lighting constraints, improve the degree to which the final lighting layout scheme meets the lighting constraints, and effectively reduce the cost of manual intervention compared with traditional lighting design methods that rely on repeated trial and error or numerical optimization. This results in a lighting layout scheme with higher realism and usability, and effectively improves the lighting layout efficiency of virtual scenes.

[0183] This embodiment also provides a virtual scene lighting layout device, which can be integrated into a terminal device. For example, as... Figure 9 As shown, the virtual scene lighting layout device may include: Extraction module 901 is used to extract scene features of the target virtual scene based on scene data of the target virtual scene; The acquisition module 902 is used to acquire the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; Module 903 is used to construct a lighting constraint function for the target virtual scene based on lighting constraints. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. Module 904 is obtained, which uses scene features as conditions for the diffusion model to predict noise in noisy lighting layout schemes and obtain the predicted noise amount. Based on each illumination constraint function and its weight parameters, the predicted noise is corrected to obtain the corrected noise level.

[0184] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0185] The solution adopted in this application addresses the common problem that traditional methods often rely on human experience or simple rules for lighting design, making it difficult to balance physical realism and artistic expression. By leveraging the powerful generative capabilities of the diffusion model and introducing quantifiable lighting constraint functions, the generated lighting layout can both conform to optical laws and satisfy the specific intentions set by the user. During the generation process, scene features are first extracted from the scene data and used as conditional inputs to the diffusion model. This allows the diffusion model to fully understand the scene context during denoising iterations, thereby obtaining light source positions, intensities, and directions that conform to the virtual scene content, effectively avoiding common problems such as light source clipping and shadow misalignment. Meanwhile, noise correction based on lighting constraints is embedded in the iterative denoising process. For various types of lighting constraints input by users (such as artistic intent, gameplay requirements, performance indicators, etc.), corresponding lighting constraint functions are constructed, and the predicted noise is dynamically adjusted in combination with corresponding weight parameters. This can guide the generation process to converge in the direction of satisfying multi-dimensional lighting constraints, improve the degree to which the final lighting layout scheme meets the lighting constraints, and effectively reduce the cost of manual intervention compared with traditional lighting design methods that rely on repeated trial and error or numerical optimization. This results in a lighting layout scheme with higher realism and usability, and effectively improves the lighting layout efficiency of virtual scenes.

[0186] Accordingly, this application also provides an electronic device, which can be a terminal, such as a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. Alternatively, the electronic device can be a server.

[0187] like Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1000 includes a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, and a computer program stored on the memory 1002 and executable on the processor. The processor 1001 and the memory 1002 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0188] The processor 1001 is the control center of the electronic device 1000. It connects various parts of the electronic device 1000 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1002, and by calling data stored in the memory 1002, it executes various functions of the electronic device 1000 and processes data, thereby providing overall monitoring of the electronic device 1000. The processor 1001 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0189] In this embodiment of the application, the processor 1001 in the electronic device 1000 loads the instructions corresponding to the processes of one or more application programs into the memory 1002 according to the following steps, and the processor 1001 runs the application programs stored in the memory 1002 to realize various functions. For example, the method includes: Extract scene features of the target virtual scene based on the scene data of the target virtual scene; Obtain the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; Based on the lighting constraints, a lighting constraint function is constructed for the target virtual scene. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. Using scene features as conditions for the diffusion model, noise prediction is performed on noisy lighting layout schemes to obtain the predicted noise level. Based on each illumination constraint function and its weight parameters, the predicted noise is corrected to obtain the corrected noise level. The noisy lighting layout scheme is iteratively denoised based on the corrected noise level until the denoising termination condition is met, resulting in the first lighting layout scheme.

[0190] The solution adopted in this application addresses the common problem that traditional methods often rely on human experience or simple rules for lighting design, making it difficult to balance physical realism and artistic expression. By leveraging the powerful generative capabilities of the diffusion model and introducing quantifiable lighting constraint functions, the generated lighting layout can both conform to optical laws and satisfy the specific intentions set by the user. During the generation process, scene features are first extracted from the scene data and used as conditional inputs to the diffusion model. This allows the diffusion model to fully understand the scene context during denoising iterations, thereby obtaining light source positions, intensities, and directions that conform to the virtual scene content, effectively avoiding common problems such as light source clipping and shadow misalignment. Meanwhile, noise correction based on lighting constraints is embedded in the iterative denoising process. For various types of lighting constraints input by users (such as artistic intent, gameplay requirements, performance indicators, etc.), corresponding lighting constraint functions are constructed, and the predicted noise is dynamically adjusted in combination with corresponding weight parameters. This can guide the generation process to converge in the direction of satisfying multi-dimensional lighting constraints, improve the degree to which the final lighting layout scheme meets the lighting constraints, and effectively reduce the cost of manual intervention compared with traditional lighting design methods that rely on repeated trial and error or numerical optimization. This results in a lighting layout scheme with higher realism and usability, and effectively improves the lighting layout efficiency of virtual scenes.

[0191] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0192] Optional, such as Figure 10 As shown, the electronic device 1000 further includes: a touch display screen 1003, a radio frequency circuit 1004, an audio circuit 1005, an input unit 1006, and a power supply 1007. The processor 1001 is electrically connected to the touch display screen 1003, the radio frequency circuit 1004, the audio circuit 1005, the input unit 1006, and the power supply 1007. Those skilled in the art will understand that... Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0193] The touch display screen 1003 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1003 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1001. It can also receive and execute commands from the processor 1001. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1001 to determine the type of touch event. Subsequently, the processor 1001 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1003 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1003 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 1003 can also be used as part of the input unit 1006 to achieve input functions.

[0194] The radio frequency circuit 1004 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0195] Audio circuit 1005 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1005 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1005, converted back into audio data, and processed by processor 1001 before being transmitted via radio frequency circuit 1004 to, for example, another electronic device, or output to memory 1002 for further processing. Audio circuit 1005 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0196] The input unit 1006 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0197] Power supply 1007 is used to supply power to various components of electronic device 1000. Optionally, power supply 1007 can be logically connected to processor 1001 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1007 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0198] although Figure 10 As not shown in the diagram, the electronic device 1000 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0199] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0200] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0201] Therefore, embodiments of this application provide a computer-readable storage medium storing multiple computer programs that can be loaded by a processor to execute any of the virtual scene lighting layout methods provided in this application. The computer program can execute the following steps of the virtual scene lighting layout method: the method includes: Extract scene features of the target virtual scene based on the scene data of the target virtual scene; Obtain the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; Based on the lighting constraints, a lighting constraint function is constructed for the target virtual scene. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. Using scene features as conditions for the diffusion model, noise prediction is performed on noisy lighting layout schemes to obtain the predicted noise level. Based on each illumination constraint function and its weight parameters, the predicted noise is corrected to obtain the corrected noise level. The noisy lighting layout scheme is iteratively denoised based on the corrected noise level until the denoising termination condition is met, resulting in the first lighting layout scheme.

[0202] The solution adopted in this application addresses the common problem that traditional methods often rely on human experience or simple rules for lighting design, making it difficult to balance physical realism and artistic expression. By leveraging the powerful generative capabilities of the diffusion model and introducing quantifiable lighting constraint functions, the generated lighting layout can both conform to optical laws and satisfy the specific intentions set by the user. During the generation process, scene features are first extracted from the scene data and used as conditional inputs to the diffusion model. This allows the diffusion model to fully understand the scene context during denoising iterations, thereby obtaining light source positions, intensities, and directions that conform to the virtual scene content, effectively avoiding common problems such as light source clipping and shadow misalignment. Meanwhile, noise correction based on lighting constraints is embedded in the iterative denoising process. For various types of lighting constraints input by users (such as artistic intent, gameplay requirements, performance indicators, etc.), corresponding lighting constraint functions are constructed, and the predicted noise is dynamically adjusted in combination with corresponding weight parameters. This can guide the generation process to converge in the direction of satisfying multi-dimensional lighting constraints, improve the degree to which the final lighting layout scheme meets the lighting constraints, and effectively reduce the cost of manual intervention compared with traditional lighting design methods that rely on repeated trial and error or numerical optimization. This results in a lighting layout scheme with higher realism and usability, and effectively improves the lighting layout efficiency of virtual scenes.

[0203] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0204] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0205] Since the computer program stored in the computer-readable storage medium can execute any of the virtual scene lighting layout methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the virtual scene lighting layout methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0206] According to one aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0207] In the above embodiments of the virtual scene lighting layout device, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the virtual scene lighting layout device, computer-readable storage medium, computer program product, electronic device, and their corresponding units described above can be referred to the description of the virtual scene lighting layout method in the above embodiments, and will not be repeated here.

[0208] The foregoing has provided a detailed description of a virtual scene lighting layout method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A virtual scene lighting layout method, characterized in that, The method includes: Based on the scene data of the target virtual scene, extract the scene features of the target virtual scene; Obtain the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; Based on the lighting constraints, a lighting constraint function is constructed for the target virtual scene. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. Using the scene features as conditions for the diffusion model, noise prediction is performed on the noisy lighting layout scheme to obtain the predicted noise level. Based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions, the predicted noise is corrected to obtain the corrected noise. Based on the corrected noise level, the noisy lighting layout scheme is iteratively denoised until the denoising termination condition is met, resulting in the first lighting layout scheme.

2. The method according to claim 1, characterized in that, The step of iteratively denoising the noisy lighting layout scheme based on the corrected noise level until the denoising termination condition is met to obtain the first lighting layout scheme includes: Based on the corrected noise level, the noisy lighting layout scheme is denoised to obtain a new noisy lighting layout scheme. Based on the new noisy lighting layout scheme, the step of using the scene features as the diffusion model to predict noise in the noisy lighting layout scheme and obtain the predicted noise amount continues until the denoising termination condition is met, thus obtaining the first lighting layout scheme.

3. The method according to claim 2, characterized in that, The method of correcting the predicted noise based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions to obtain the corrected noise includes: Based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions, the first loss of the illumination layout scheme corresponding to the predicted noise is determined. Determine the gradient corresponding to the first loss, and obtain the corrected noise amount based on the gradient.

4. The method according to claim 3, characterized in that, Determining the gradient corresponding to the first loss, and obtaining the corrected noise amount based on the gradient, includes: Determine the gradient of the first loss relative to the lighting layout scheme corresponding to the predicted noise amount; The gradient is weighted based on a preset guiding strength parameter to obtain the correction amount of the predicted noise. The predicted noise level is corrected based on the correction amount to obtain the corrected noise level.

5. The method according to claim 3, characterized in that, The step of determining the first loss of the lighting layout scheme corresponding to the predicted noise based on each of the lighting constraint functions and the weight parameters of each of the lighting constraint functions includes: Based on each of the illumination constraint functions and the illumination layout scheme corresponding to the predicted noise, determine the second loss corresponding to each of the illumination constraint functions; The first loss is determined based on the second loss corresponding to each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions.

6. The method according to claim 5, characterized in that, The method further includes: Based on the first lighting layout scheme, the evaluation index value of the target virtual scene in at least one evaluation dimension is determined; The comprehensive evaluation score of the first lighting layout scheme is determined based on the evaluation index values ​​corresponding to each evaluation dimension and the preset weights corresponding to each evaluation dimension. Based on the comprehensive evaluation score, the weight parameters of each of the lighting constraint functions are updated to obtain the updated weight parameters of each of the lighting constraint functions. Based on the updated weight parameters of each of the lighting constraint functions, the step of re-exercising the predicted noise based on each of the lighting constraint functions and the weight parameters of each of the lighting constraint functions to obtain the corrected noise is performed to obtain the second lighting layout scheme. The evaluation dimensions include at least one of the following: interaction dimension, visual dimension, and resource consumption dimension.

7. The method according to claim 6, characterized in that, The step of updating the weight parameters of each of the lighting constraint functions based on the comprehensive evaluation score to obtain the updated weight parameters of each of the lighting constraint functions includes: Based on the comprehensive evaluation score, calculate the gradient of the comprehensive evaluation score with respect to the weight parameters of each of the lighting constraint functions; Based on the gradient and the preset learning rate parameter, the weight parameters of each of the illumination constraint functions are updated to obtain the updated weight parameters of each of the illumination constraint functions.

8. The method according to any one of claims 1 to 7, characterized in that, The step of extracting at least one scene feature of the target virtual scene based on the scene data of the target virtual scene includes: Based on the scene data of the target virtual scene, obtain the object information of the virtual objects in the target virtual scene; Based on the object information of the virtual object, the object features of the virtual object are extracted, and the object features include at least one of geometric feature vector, material feature vector, and semantic feature vector. The object features of the virtual object are combined to obtain at least one scene feature of the target virtual scene.

9. The method according to any one of claims 1 to 7, characterized in that, The lighting constraint types include at least one of the following: global lighting constraints on the target virtual scene, lighting constraints on a portion of the target virtual scene, lighting constraints on a specified virtual object in the target virtual scene, lighting constraints on a path in the target virtual scene, lighting constraints on a specified viewing angle, and constraints related to light source rendering resources.

10. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The target virtual scene is rendered using a target lighting layout scheme, which is determined based on the first lighting layout scheme and / or the second lighting layout scheme.

11. A virtual scene lighting layout device, characterized in that, The device includes: The extraction module is used to extract scene features of the target virtual scene based on scene data of the target virtual scene; The acquisition module is used to acquire the input lighting constraint conditions under at least one lighting constraint type for the target virtual scene; A construction module is used to construct a lighting constraint function for the target virtual scene based on the lighting constraints. The lighting constraint function is used to determine the degree to which the lighting layout scheme satisfies the lighting constraints. The module is used to predict the noise level of a noisy lighting layout scheme by using the scene features as conditions for the diffusion model. Based on each of the illumination constraint functions and the weight parameters of each of the illumination constraint functions, the predicted noise is corrected to obtain the corrected noise. Based on the corrected noise level, the noisy lighting layout scheme is iteratively denoised until the denoising termination condition is met, resulting in the first lighting layout scheme.

12. An electronic device, characterized in that, The system includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the virtual scene lighting layout method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the virtual scene lighting layout method as described in any one of claims 1 to 10.

Images