Illumination rendering method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an illumination rendering method and device, computer equipment, a storage medium and a computer program product. The method can be applied to technologies of artificial intelligence, intelligent traffic and the like, and comprises the following steps: performing light effect detection on a rendered image to obtain a detection result; when the detection result shows that the first object vertex which does not meet the illumination rendering condition exists in the rendered image, obtaining illumination information of a second object vertex which meets the illumination rendering condition in the rendered image along an illumination path where the first object vertex is located; training a neural network based on the illumination information of the second object vertex to obtain a trained neural network; through the trained neural network, performing illumination information extraction on object vertexes including the first object vertex in the to-be-rendered image to obtain illumination information of the object vertexes; and performing illumination rendering on the to-be-rendered image according to the illumination information of each object vertex. By adopting the method, the illumination rendering effect can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a method, device, computer device, storage medium, and computer program product for light rendering. Background Art

[0002] With the development of image processing technology, light rendering technology has been widely used in fields such as games, movies, and virtual reality to achieve light effects in related scenes, thereby greatly improving the authenticity of the scenes.

[0003] In traditional light rendering solutions, radiation caching technology is usually used to obtain cached light information, and then the cached light information is used for light rendering. Since in some application scenarios, the positions and postures of objects are dynamically changing, or the viewing angles during the display of images are dynamically changing, the light information on the object surface will also change dynamically. At this time, using the cached light information for light rendering will result in poor light rendering effects. Summary of the Invention

[0004] Based on this, in order to solve the above technical problems, it is necessary to provide a light rendering method, device, computer device, computer-readable storage medium, and computer program product that can effectively improve the light rendering effect.

[0005] In a first aspect, the present application provides a light rendering method, and the method includes:

[0006] Before performing light rendering on the image to be rendered, perform light effect detection on the already-rendered image to obtain a detection result; the already-rendered image is a previous frame image of the image to be rendered;

[0007] When the detection result indicates that there are first object vertices in the already-rendered image that do not meet the light rendering conditions, along the light path where the first object vertices are located, obtain the light information of the second object vertices in the already-rendered image that meet the light rendering conditions;

[0008] Train a neural network based on the light information of the second object vertices to obtain a trained neural network;

[0009] Extract the light information of each object vertex including the first object vertex in the image to be rendered through the trained neural network to obtain the light information of each object vertex;

[0010] Perform light rendering on the image to be rendered according to the light information of each object vertex.

[0011] In a second aspect, the present application also provides a light rendering device, and the device includes:

[0012] A detection module, configured to perform a light effect detection on a rendered image before performing light rendering on an image to be rendered, so as to obtain a detection result; the rendered image is a previous frame image of the image to be rendered;

[0013] A search module, configured to, when the detection result indicates that there is a first object vertex in the rendered image that does not meet the light rendering condition, obtain light information of a second object vertex in the rendered image that meets the light rendering condition along a light path where the first object vertex is located;

[0014] A training module, configured to train a neural network based on the light information of the second object vertex to obtain a trained neural network;

[0015] An extraction module, configured to extract light information of each object vertex including the first object vertex in the image to be rendered through the trained neural network, so as to obtain the light information of each object vertex;

[0016] A rendering module, configured to perform light rendering on the image to be rendered according to the light information of each object vertex.

[0017] In one embodiment, the first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point; the search module is configured to determine, in the rendered image, a first-level intersection point that intersects with a direct light ray of a light source; generate a first secondary light ray at the first-level intersection point, and determine, in the rendered image, a point that intersects with the first secondary light ray to obtain a second-level intersection point; generate a second secondary light ray at the second-level intersection point, and determine, in the rendered image, a point that intersects with the second secondary light ray to obtain a third-level intersection point; generate a light path including the first-level intersection point, the second-level intersection point, and the third-level intersection point; and obtain the light information of the second object vertex in the rendered image that meets the light rendering condition along the light path.

[0018] In one embodiment, the search module is further configured to generate an initial path according to the first-level intersection point, the second-level intersection point, and the third-level intersection point; extend the initial light path to obtain an extended path; and search, along the extended path, for a second object vertex in the rendered image that meets the light rendering condition.

[0019] In one embodiment, the apparatus further includes:

[0020] A first acquisition module, configured to acquire a pre-configured first light reflection count;

[0021] A determination module, configured to determine a vertex search count according to the first light reflection count and the number of the first object vertices;

[0022] The search module is further configured to search for second object vertices that meet the light rendering condition in the rendered image along the light path according to the vertex search quantity.

[0023] In one embodiment, the apparatus further includes:

[0024] A display module, configured to display a configuration page of an interactive application;

[0025] A configuration module, configured to, in response to a first configuration operation triggered on the configuration page, configure the number of light reflections in the configuration page; and in response to a second configuration operation triggered on the configuration page, configure the on / off state of the radiation cache.

[0026] In one embodiment, the detection module is further configured to obtain the configured on / off state of the radiation cache; when the on / off state of the radiation cache is the radiation cache on state, perform a light effect detection on the rendered image to obtain a detection result;

[0027] The first acquisition module is further configured to, when the on / off state of the radiation cache is the radiation cache off state, acquire the light information for each object vertex in the to-be-rendered image that is cached;

[0028] The rendering module is further configured to perform light rendering on the to-be-rendered image based on the cached light information for each object vertex in the to-be-rendered image.

[0029] In one embodiment, the extraction module is further configured to, when the detection result indicates that each object vertex in the rendered image meets the light rendering condition, extract the light information for each object vertex in the to-be-rendered image through the neural network;

[0030] The rendering module is further configured to perform light rendering on the to-be-rendered image according to the light information for each object vertex in the to-be-rendered image.

[0031] In one embodiment, the rendering module is further configured to train at least two sub-networks based on the light information of the second object vertices to obtain at least two trained sub-networks; wherein, the at least two sub-networks are obtained by splitting the structure of the neural network.

[0032] In one embodiment, the number of the second object vertices is at least two; the apparatus further includes:

[0033] A second acquisition module, configured to acquire the geometric information of at least two of the second object vertices;

[0034] The training module is further configured to input the lighting information and geometric information of each of the second object vertices into each of the sub-networks, so that each sub-network generates predicted lighting information for each of the second object vertices based on the input geometric information; and optimize the parameters of each sub-network respectively based on the loss value between the predicted lighting information and the lighting information of each of the second object vertices.

[0035] In one embodiment, the extraction module is further configured to obtain pre-configured numbers of secondary light reflections, light refractions, and light scatterings; extract lighting information for each object vertex including the first object vertex in the to-be-rendered image according to the numbers of secondary light reflections, light refractions, and light scatterings; wherein the lighting information includes direct lighting information, reflected light information, refracted light information, and scattered light information.

[0036] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned lighting rendering method are implemented.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the above-mentioned lighting rendering method are implemented.

[0038] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned lighting rendering method are implemented.

[0039] Before performing lighting rendering on the to-be-rendered image, the above-mentioned lighting rendering method, device, computer device, storage medium, and computer program product first perform light effect detection on the already-rendered image. When it is detected that there are first object vertices in the already-rendered image that do not meet the lighting rendering conditions, obtain the lighting information of second object vertices on the same lighting path as the first object vertices and that meet the lighting rendering conditions, and use the obtained lighting information for the training of the neural network, which can enable the neural network to learn to extract lighting information that meets the lighting rendering conditions for the first object vertices. Therefore, even if the objects or viewpoints in the scene change dynamically, the trained neural network can still extract lighting information that meets the lighting rendering conditions, and using this lighting information can effectively improve the lighting rendering effect of the to-be-rendered image; in addition, only using the lighting information of second object vertices on the same lighting path as the first object vertices and that meet the lighting rendering conditions to train the neural network can effectively speed up the training speed and meet the real-time requirements of lighting rendering. Description of the Drawings

[0040] Figure 1 It is an application environment diagram of the lighting rendering method in an embodiment;

[0041] Figure 2 It is a schematic flowchart of the lighting rendering method in an embodiment;

[0042] Figure 3 It is a schematic diagram of the lighting path in an embodiment;

[0043] Figure 4 It is a schematic diagram of the lighting path in another embodiment;

[0044] Figure 5a It is a schematic structural diagram of a neural network in an embodiment;

[0045] Figure 5b It is a schematic structural diagram of splitting the neural network into sub - networks in an embodiment;

[0046] Figure 6 It is a schematic diagram of dividing the rendered image into multiple regions in an embodiment;

[0047] Figure 7 It is a schematic comparison diagram of the rendering results using different lighting rendering methods in an embodiment;

[0048] Figure 8 It is a schematic comparison diagram of the rendering results using different lighting rendering methods in another embodiment;

[0049] Figure 9 It is a schematic comparison diagram of the rendering results using different lighting rendering methods in another embodiment;

[0050] Figure 10 It is a schematic flowchart of the step of extracting lighting information in an embodiment;

[0051] Figure 11 It is a schematic page diagram of configuring lighting rendering parameters in an embodiment;

[0052] Figure 12 It is a structural block diagram of a lighting rendering device in an embodiment;

[0053] Figure 13 It is a structural block diagram of a lighting rendering device in an embodiment;

[0054] Figure 14 It is an internal structural diagram of a computer device in an embodiment. Detailed implementation manners

[0055] In order to make the objectives, technical solutions and advantages of this application more clear and understandable, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0056] It should be noted that in the following description, the terms "first", "second" and "third" only distinguish similar objects and do not represent a specific order for the objects. Understandably, "first", "second" and "third" can be interchanged in a specific order or sequence when permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0057] Before elaborating on the solution of this application, the technologies involved in this application will be explained as follows:

[0058] Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. Artificial intelligence basic technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, the pre-trained model, also known as the large model or the basic model, can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, machine learning / deep learning, autonomous driving, and intelligent transportation.

[0059] Computer Vision (CV) is a science that studies how to enable machines to "see". More specifically, it refers to machine vision that uses cameras and computers to replace human eyes for object recognition and measurement, and further performs graphics processing to make the computer-processed images more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Large model technology has brought important changes to the development of computer vision technology. Pretrained models in the visual field such as swin-transformer, ViT, V-MOE, and MAE can be quickly and widely applied to downstream specific tasks after fine-tuning. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, etc., and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0060] Machine Learning (ML) is an interdisciplinary field that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0061] The lighting rendering method provided in the embodiments of this application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process, such as rendered or to-be-rendered image data. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers.

[0062] Among them, the terminal 102 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an Internet of Things device, and a portable wearable device. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0063] The server 104 can be an independent physical server or a service node in a blockchain system. A peer-to-peer network is formed among the service nodes in the blockchain system. The peer-to-peer protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP).

[0064] In addition, the server 104 can also be a server cluster composed of multiple physical servers, and can be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0065] The terminal 102 and the server 104 can be connected through communication connection methods such as Bluetooth, USB (Universal Serial Bus), or network. This application does not limit this here.

[0066] In one embodiment, as Figure 2 shown, a lighting rendering method is provided. This method can be executed by the server or the terminal in Figure 1 , or jointly executed by the server and the terminal. Taking the execution by the terminal in Figure 1 as an example, the method includes the following steps:

[0067] S202, before performing lighting rendering on the image to be rendered, perform light effect detection on the already rendered image to obtain a detection result.

[0068] Among them, the image to be rendered can be at least one image that needs to be rendered, and can be an image that needs to be displayed currently and has not been rendered, or a video frame that needs to be played currently and has not been rendered. In practical applications, the image to be rendered can be a game image to be rendered in a game scene, or a virtual reality image to be rendered in a virtual reality scene, or an augmented reality image to be rendered in an augmented reality scene, or an extended reality image to be rendered in an extended reality scene.

[0069] The rendered image can be at least one image that has been completely rendered and is a previous frame image of the image to be rendered. Specifically, the rendered image can be an image that is being displayed (or has been displayed) after completion of rendering, or a video frame that is being played (or has been played) after completion of rendering. In practical applications, the rendered image can be a rendered game image in a game scene, or a rendered virtual reality image in a virtual reality scene, or a rendered augmented reality image in an augmented reality scene, or a rendered extended reality image in an extended reality scene. It should be noted that the image to be rendered and the rendered image can be images of different frames in the same scene, such as images of different frames in the same game scene.

[0070] The light effect can be the rendering effect of light, which can be referred to as the light rendering effect or the light effect. Therefore, light effect detection can refer to the detection of the light rendering effect. The detection result can be a result used to indicate whether the light rendering effect of the whole or part of the rendered image is good or bad. For example, when an abnormal image parameter such as over-brightness, over-darkness, or excessive noise appears in a certain object (such as a virtual character or virtual object in a game) in the rendered image, it indicates that the light rendering effect of the object is poor and will affect the visual effect. Thus, the object (including each object vertex of the object) does not meet the light rendering condition.

[0071] It should be noted that the light rendering method of the present application can be applied to game scenes, virtual reality scenes, augmented reality scenes, and extended reality scenes, etc. And in the above scenes, during the process of image rendering (including light rendering), the light information of the corresponding object vertices in the rendered image can be used to guide the training of the neural network, and then the trained neural network is used to extract the light information of each object vertex in the image to be rendered, so as to render the light effect of the image to be rendered. That is to say, the light rendering method of the present application can directly apply the initialized neural network to the image rendering process, and perform network training during the rendering process, without the need to pre-train the neural network. In addition, before applying the neural network to image rendering, the neural network can also be pre-trained. In this case, accurate light rendering can also be performed on the first few frames of images.

[0072] In one embodiment, in order to ensure the lighting rendering effect of subsequent images to be rendered, the terminal may, before performing lighting rendering on the image to be rendered, first perform a light effect detection on the rendered image, such as detecting at least one image parameter among the number of noise points, brightness or contrast with the surrounding environment of each object in the rendered image, to obtain a detection result; thereby, it can be determined whether the lighting rendering effect of the rendered image meets the requirements (i.e., lighting rendering conditions) according to the detection result, so as to decide whether it is necessary to train the neural network, which is beneficial to improving the lighting rendering effect of the image to be rendered and also beneficial to ensuring the smoothness of image display during the rendering process.

[0073] For example, in a game scenario, during the process of a user playing a game, assuming that the terminal has completed the rendering of game video frames a to c, in order to ensure the lighting rendering effect of subsequent game video frames (such as the game video frame d to be rendered), at this time, a light effect detection will be performed on game video frames a to c to obtain a detection result, so as to determine whether it is necessary to train the neural network according to the detection result, thereby ensuring the lighting rendering effect of game video frame d.

[0074] In one embodiment, the terminal may first determine whether it is necessary to perform a light effect detection on the rendered image. When light effect detection is required, the terminal performs a light effect detection on the rendered image to obtain a detection result. Specifically, the terminal may first obtain the configured on / off state of the radiation cache; determine whether light effect detection is required according to the on / off state of the radiation cache. When the on / off state of the radiation cache is the radiation cache on state, perform a light effect detection on the rendered image to obtain a detection result. By configuring the on / off state of the radiation cache to determine whether to use the neural radiation cache (NRC) method for lighting rendering, it is beneficial to select different rendering methods under different requirements to meet the needs of different users. For example, for users with high requirements for image quality, they can choose to turn on the radiation cache, which is beneficial to rendering high-fidelity images.

[0075] In addition, for users with high requirements for high refresh rate (or smoothness), they can choose to turn off the neural radiation cache. Therefore, when the on / off state of the radiation cache is the radiation cache off state, the terminal can obtain the cached lighting information for each object vertex in the image to be rendered; based on the cached lighting information for each object vertex in the image to be rendered, perform lighting rendering on the image to be rendered, thereby effectively ensuring the high smoothness requirement of the screen even under high refresh rate conditions.

[0076] S204, when the detection result indicates that there is a first object vertex in the rendered image that does not meet the lighting rendering conditions, along the lighting path where the first object vertex is located, obtain the lighting information of the second object vertex in the rendered image that meets the lighting rendering conditions.

[0077] Among them, the first object vertex can be a vertex of the object. The number of the first object vertices can be m, where m is a positive integer greater than or equal to 1. In addition, the second object vertex can also be a vertex of the object. The number of the second object vertices can be n, where n is a positive integer greater than or equal to 1. As Figure 3 shown, the first object vertex can be , and , and the second object vertex can be and .

[0078] The object mentioned above can be a person, an animal, a plant, an item, a building, etc. shown in the image. In some scenarios, the object is presented in the image by collecting the image of a real object in the real world; in some special scenarios, the object can be a virtual object. For example, in a game scenario, the object can be a game object in the game scenario, including virtual characters (such as game characters controlled by users), virtual animals (such as game monsters or game characters controlled by users), virtual plants, virtual items, and virtual buildings, etc.

[0079] The light path can be a path formed by the propagation of light between objects, including a path formed by at least one propagation method such as direct light, reflection, refraction, or scattering between objects. Correspondingly, the light path where the first object vertex is located can be a light path containing the first object vertex, and can also contain other object vertices (such as the second object vertex). As Figure 3 shown, the light path can be composed of , , , and . The light path contains the first object vertices , and , and also contains the second object vertices and .

[0080] The light rendering condition can also be called the light rendering requirement. Correspondingly, the first object vertex not meeting the light rendering condition can mean that the light rendering effect of the first object vertex does not meet the light rendering condition, such as meaning that at least one of the number of noise points is greater than the preset number, the brightness is greater than the preset brightness, the darkness is greater than the preset darkness, or the contrast is less than the preset contrast.

[0081] The lighting information can be information used to represent the radiance, color, and lighting type of light. Among them, the lighting information of different lighting types can include direct lighting information, reflected light information, refracted light information, and scattered light information; the radiance can be a physical quantity describing the intensity and direction distribution of light, used to measure the light energy passing through a unit area of the surface of a certain object in a given direction.

[0082] It should be noted that the first object vertex does not meet the lighting rendering condition, indicating that the first object vertex has not been learned by the neural network, that is, the neural network has not learned how to extract the lighting information that meets the lighting rendering condition of the first object vertex. For example, the neural network has not learned how to extract the lighting information that meets the lighting rendering condition of the first object vertex from the initial path (i.e., the initial lighting path) formed by the first object vertex.

[0083] In one embodiment, when the detection result indicates that there is a first object vertex in the rendered image that does not meet the lighting rendering condition, the terminal can use a path tracing algorithm to obtain the lighting information of the second object vertex that meets the lighting rendering condition, that is, simulate the reflection, refraction, and scattering of light among objects in the environment in the rendered image, and obtain the lighting information of each vertex on the object surface on the lighting path where the first object vertex is located. Therefore, accurate lighting information can be obtained through the above path tracing algorithm, and complex lighting effects can be accurately simulated during the rendering process, such as simulating shadows, indirect lighting, colors, and transparencies in complex game scenes.

[0084] Among them, the first object vertex does not meet the lighting rendering condition, indicating that the first object vertex has not been learned by the neural network. At this time, the lighting information of the second object vertex that meets the lighting rendering condition on the same lighting path as the first object vertex will be obtained, so as to use the lighting information of the second object vertex to train the neural network so that the neural network can learn the first object vertex.

[0085] In another embodiment, when the detection result indicates that all object vertices in the rendered image meet the lighting rendering condition, the lighting information of each object vertex in the image to be rendered is extracted through the neural network; according to the lighting information of each object vertex in the image to be rendered, lighting rendering is performed on the image to be rendered. Therefore, when all object vertices meet the lighting rendering condition, there is no need to train the neural network, and the lighting information of each object vertex extracted by the neural network is directly used to perform lighting rendering on the image to be rendered, which can effectively ensure the smoothness of each frame of image display.

[0086] Among them, when all object vertices meet the lighting rendering conditions, it means that the first object vertices are learned by the neural network, that is, the neural network learns how to extract the lighting information that meets the lighting rendering conditions of the first object vertices. For example, the neural network has learned how to extract the lighting information that meets the lighting rendering conditions of the first object vertices from the initial path.

[0087] Therefore, when all object vertices meet the lighting rendering conditions, the terminal can directly use the lighting information of each object vertex in the rendered image to determine the lighting information of each object vertex in the to-be-rendered image without training the neural network, and then use the lighting information of each object vertex in the to-be-rendered image to perform lighting rendering on the to-be-rendered image. For example, using Figure 4 the object vertices in are the object vertices in the rendered image 、 and , according to each object vertex in the rendered image 、 and infer the lighting information of each object vertex in the to-be-rendered image, and then perform lighting rendering on the to-be-rendered image; for a game scene, this can ensure the smoothness of each frame of the game screen display; in addition, for virtual reality scenes, augmented reality scenes, and extended reality scenes, it can also ensure the smoothness of each frame of the scene screen display.

[0088] S206. Train the neural network based on the lighting information of the second object vertices to obtain the trained neural network.

[0089] Among them, the neural network can be a Multi-Layer Perceptron (MLP) network. For example, as Figure 5a shown, Figure 5a in the exemplified neural network, is the input layer, is the hidden layer (i.e., the perception layer), is the output layer, and ReLU is the activation function.

[0090] The trained neural network can be a neural network trained based on the lighting information of the second object vertices of the previous frame of the rendered image of the to-be-rendered image, or a neural network trained based on the lighting information of the second object vertices of multiple rendered images.

[0091] In one embodiment, the terminal can obtain the geometric information of the second object vertices, and use the geometric information and illumination information of the second object vertices to train a neural network to obtain a trained neural network. It should be emphasized that when training the neural network, only the illumination information and geometric information of the second object vertices on the illumination path where the first object vertices are located are used for training, rather than using the geometric information and illumination information of all object vertices in the rendered image for training, which can effectively improve the training speed and greatly shorten the training time, so as to meet the real-time rendering requirements of the image to be rendered.

[0092] Among them, the geometric information may include at least one of the position, surface roughness, propagation direction of light at the vertex, surface normal vector at the vertex, diffuse vector, or specular vector of the second object vertices.

[0093] The position can be represented by three-dimensional coordinates. In order to enable the multi-layer perceptron to obtain a better illumination information extraction effect, before inputting into the neural network, each coordinate value in the three-dimensional coordinates is extended to twelve dimensions through trigonometric function encoding, so as to obtain a thirty-six-dimensional position. The surface roughness is a one-dimensional vector and is extended to four dimensions through identity encoding. The propagation direction of light at the geometric vertex is a two-dimensional direction vector and is extended to eight dimensions through identity encoding. The surface normal vector at the geometric vertex is a two-dimensional vector and is extended to eight dimensions through identity encoding. Both the diffuse vector and the specular vector at the geometric vertex are three-dimensional vectors and are not encoded. After processing in the above manner, sixty-two-dimensional geometric information is obtained, and then it is supplemented to sixty-four dimensions with placeholders. The sixty-four-dimensional geometric information and the illumination information are input into the neural network for training. Therefore, after processing the geometric information of the second object vertices in the above manner, the multi-layer perceptron can obtain a better illumination information extraction effect.

[0094] In one embodiment, in order to further improve the real-time performance during the rendering process, the neural network can be pre-structurally split into at least two sub-networks, and then the at least two sub-networks are trained based on the illumination information of the second object vertices, which can greatly shorten the network training time and quickly obtain at least two trained sub-networks, so as to effectively meet the real-time rendering requirements of the image to be rendered.

[0095] Among them, the trained sub-network can refer to the sub-network obtained after training, and can also be called the trained sub-network.

[0096] When splitting the structure of a neural network, the neurons in each network layer of the neural network are respectively divided into multiple groups to obtain multiple groups of neurons for each network layer; then, the groups of neurons in each network layer are respectively combined to obtain multiple sub-networks. For example, the network layers of a neural network include an input layer, multiple hidden layers, and an output layer. Refer to Figure 6 the structure diagram above. Each network layer of this neural network has a relatively large number of neurons. Let's assume that the number of neurons in each network layer is N = m × n. At this time, the neurons in the input layer, the neurons in each hidden layer, and the neurons in the output layer can be respectively divided into m groups (i.e., the 1st to mth groups of neurons), and the number of neurons in each group is n; then, the ith group of neurons in the input layer, the ith group of neurons in each hidden layer, and the ith group of neurons in the output layer are combined until the combination between the corresponding groups of neurons in each network layer is completed, so that m sub-networks can be obtained. i, m, n, and N are all positive integers, and i ≤ N, m < N, n < N. It should be noted that the structures and calculation methods of each sub-network are the same. After training, the parameter values of each sub-network can be different. For example, the parameter a of the trained sub-network 1 and the parameter a of the sub-network 2 can be different in size. It should be noted that the above "multiple" can be two or more than two; in addition, "multiple groups" can be two groups or more than two groups.

[0097] Pre-splitting the structure of the neural network to obtain at least two sub-networks reduces the number of neurons in the network, can effectively reduce the data dimensions of the input and output, reduces the complexity of the entire network, and greatly reduces the computational amount; in addition, using the Cuda operator can enable the split sub-networks to have high-performance concurrent processing capabilities, so it can ensure that the training and inference of the network can be carried out in real time.

[0098] It should be noted that Cuda can be a high-performance computing platform that allows the use of a graphics processing unit (GPU) for parallel computing, thereby accelerating the processing speed of computing tasks. Correspondingly, the Cuda operator can refer to: an operator that uses the parallel computing ability of the graphics processing unit (GPU) to allocate computing tasks to multiple computing units (i.e., the cores of Cuda) for execution. For example, by using the Cuda operator, the lighting information and geometric information of the vertices of each second object can be allocated to the corresponding sub-networks for parallel processing, which can accelerate the training speed of the sub-networks.

[0099] In one embodiment, the number of second object vertices is at least two; thus, before training, the terminal can obtain the geometric information of at least two second object vertices; then, input the illumination information and geometric information of each second object vertex into each sub-network, so that each sub-network generates the predicted illumination information of each second object vertex based on the input geometric information; based on the loss value between the predicted illumination information and the illumination information of each second object vertex, optimize the parameters of each sub-network respectively. In addition, before inputting to the sub-network, the terminal can also encode the geometric information of each second object vertex to expand the dimension of the geometric information, which is beneficial to improving the illumination information extraction effect of each perceptron in the sub-network, and splitting the neural network into multiple sub-networks for training can greatly reduce the training time-consuming and effectively ensure the real-time requirement.

[0100] For example, the terminal can obtain geometric information including at least one of position, surface roughness, propagation direction of light at the vertex, surface normal vector at the vertex, scattering vector or specular reflection vector, and then respectively use the geometric information of each second object vertex as training data, and the illumination information of each second object vertex as labels, and then input these training data and illumination data into different sub-networks respectively. For example, training data 1 and label 1 are input into sub-network 1, training data 2 and label 2 are input into sub-network 2, training data 3 and label 3 are input into sub-network 3, training data 4 and label 4 are input into sub-network 4, as Figure 5b shown, so that the sub-network generates the predicted illumination information of the second object vertex based on the input geometric information, calculates the loss value between the predicted illumination information and the label, and then backpropagates the obtained loss value in the sub-network to realize the parameter optimization of the sub-network. Among them, Figure 5b in represents the illumination feature output by the i-th perception layer, is a learnable matrix parameter for processing the illumination feature output by the i-th perception layer, .

[0101] In another embodiment, the terminal can divide multiple regions in the rendered image, select multiple object vertices in each region, then determine the illumination path where the first object vertex is located among these object vertices, then continue to screen the second object vertices that meet the illumination rendering conditions among these vertices, and then obtain the corresponding illumination information for training. For example, as Figure 6 shown, divide the rendered image into 6×6 different grids, select multiple object vertices in each grid respectively, and then select the second object vertices that are on the same illumination path as the first object vertex that meets the illumination rendering conditions and meet the illumination rendering, and then use the illumination information corresponding to the second object vertex for training.

[0102] S208. Use the trained neural network to extract the lighting information of each object vertex including the first object vertex in the image to be rendered, so as to obtain the lighting information of each object vertex.

[0103] Among them, the trained neural network can perform lighting information processing on each object vertex, which can be that the trained neural network processes the geometric information of each object vertex to generate the lighting information of each object vertex.

[0104] In one embodiment, when the neural network is split into at least two sub-networks for training to obtain at least two trained sub-networks, the terminal uses the at least two trained sub-networks to extract the lighting information of each object vertex including the first object vertex in the image to be rendered, so as to obtain the lighting information of each object vertex. Considering that the number of neurons in the trained sub-network is small and the data dimensions of the input and output are low, the complexity of the network is effectively reduced. Therefore, extracting the lighting information through the trained sub-network can effectively speed up the extraction of the lighting information and is beneficial to ensuring the real-time performance of lighting rendering.

[0105] For example, taking a game scene as an example, the terminal can obtain the geometric information of the object vertices corresponding to each game object and environmental object in the game video frame to be rendered, and input these geometric information into different trained sub-networks respectively, which can be referred to Figure 5b so that these trained sub-networks generate the lighting information of the corresponding object vertices according to the input geometric information of the object vertices.

[0106] In one embodiment, when extracting the lighting information, the terminal can combine the configured light propagation parameters to extract the lighting information of each object vertex in the image to be rendered, so as to control the number of times of reflected light, refracted light and scattered light extracted, and avoid excessive calculation. Specifically, the terminal can obtain the pre-configured second number of light reflections, number of light refractions and number of light scatterings; for each object vertex including the first object vertex in the image to be rendered, extract the lighting information according to the second number of light reflections, number of light refractions and number of light scatterings, so that the number of times of light reflection, refraction and scattering can be controlled when extracting the lighting information.

[0107] Among them, the second number of light reflections, number of light refractions and number of light scatterings can be the limit values for the reflection, refraction and scattering of light between object surfaces during the real-time inference of the neural network, and this value is used to control when the light reflection, refraction and scattering stop. In addition, the lighting information includes direct lighting information, reflected light information, refracted light information and scattered light information.

[0108] S210. Perform lighting rendering on the image to be rendered according to the lighting information of each object vertex.

[0109] Among them, light rendering can be an act of rendering the light effect in real time. In addition, real-time rendering is a rendering method in computer graphics to obtain a display image within a limited time. The goal is to generate a display image with better visual effects in the shortest possible time, so as to improve the smoothness of the picture in interactive applications (such as game applications, virtual reality applications, and augmented reality applications, etc.).

[0110] In one embodiment, the terminal can utilize a light rendering function (such as a spherical harmonic function), and perform light rendering on each object in the image to be rendered according to the light information of each object vertex, and obtain the rendering result of the global illumination of each object; since this light information is generated by a neural network (or a sub-network of a neural network), it avoids the complex integral calculation in traditional path tracing, reduces the amount of calculation, and thus can effectively shorten the rendering speed and ensure the real-time requirement of rendering.

[0111] It should be noted that for light rendering in the traditional solution, usually the irradiance caching technology is used to obtain the cached light information, and then the cached light information is used for light rendering. Since in some application scenarios, the positions and postures of each object are dynamically changing, or the viewing angle during the display image process is dynamically changing, which leads to the dynamic change of the light information on the object surface. At this time, using the cached light information for light rendering will cause the problem of poor light rendering effect, as shown in Figure 7 Figure (a) of Figure 8 Figure (a) of Figure 9 Figure (a) of Figure 7 Figure (b) of Figure 8 Figure (b) of Figure 9 Figure (b) of

[0112] In the above embodiments, before performing light rendering on the image to be rendered, the light effect of the rendered image is first detected. When it is detected that there is a first object vertex in the rendered image that does not meet the light rendering condition, the light information of the second object vertex on the same light path as the first object vertex and meeting the light rendering condition is obtained. Using the obtained light information for the training of the neural network can enable the neural network to learn to extract the light information that meets the light rendering condition for the first object vertex. Therefore, even if the objects or viewpoints in the scene change dynamically, the trained neural network can still extract the light information that meets the light rendering condition. Using this light information can effectively improve the light rendering effect of the image to be rendered. In addition, only using the light information of the second object vertex on the same light path as the first object vertex and meeting the light rendering condition to train the neural network can effectively accelerate the training speed and meet the real-time requirement of light rendering.

[0113] In one embodiment, the terminal can determine the light path where the first object vertex is located in the rendered image; then, along the light path where the first object vertex is located, obtain the light information of the second object vertex in the rendered image that meets the light rendering condition.

[0114] In another embodiment, the first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point. Therefore, the steps of obtaining the light information of the second object vertex can refer to 10, and the specific steps include:

[0115] S1002, determine the first-level intersection point in the rendered image that intersects with the direct light of the light source.

[0116] Among them, the first-level intersection point can refer to the vertex when the direct light of the light source intersects with the object. For example, the point where the light directly emitted by the light source (i.e., the direct light) intersects with the game object a in the game scene is the first-level intersection point. It can be understood that the direct light of the light source intersecting with the game object a in the game scene can mean that the direct light of the light source shines on the game object a.

[0117] S1004, generate a first secondary light at the first-level intersection point, and determine the point that intersects with the first secondary light in the rendered image to obtain the second-level intersection point.

[0118] Among them, the first secondary light can be the light generated when at least one of the light phenomena of reflection, refraction, or scattering occurs after the direct light of the light source shines on the object. For example, when the direct light of the light source shines on the game object a in the game scene, a reflection phenomenon will occur on the direct light on the game object a, and at this time, a reflected light will be generated, and this reflected light is the first secondary light.

[0119] The second - level intersection point is the vertex when the first - level secondary ray intersects with an object. For example, when the first - level secondary ray intersects with other game objects (such as game object b) in the game scene, the intersection point is the second - level intersection point.

[0120] S1006, Generate a second - level secondary ray at the second - level intersection point, and determine the point that intersects with the second - level secondary ray in the rendered image to obtain the third - level intersection point.

[0121] Among them, the second - level secondary ray can be the ray generated when at least one of the light phenomena such as reflection, refraction, or scattering occurs after the first - level secondary ray irradiates the object. For example, when the first - level secondary ray irradiates game object b in the game scene, a reflection phenomenon occurs on game object b for the first - level secondary ray, and at this time, a reflected ray is generated, and this reflected ray is the second - level secondary ray.

[0122] The third - level intersection point is the vertex when the second - level secondary ray intersects with an object. For example, when the second - level secondary ray intersects with other game objects (such as game object c) in the game scene, the intersection point is the third - level intersection point.

[0123] S1008, Generate a lighting path including the first - level intersection point, the second - level intersection point, and the third - level intersection point.

[0124] It should be noted that the lighting path can include other object nodes in addition to the first - level intersection point, the second - level intersection point, and the third - level intersection point.

[0125] For example, when the primary ray (i.e., the direct ray of the light source) intersects with object 1 in the scene, the first - level intersection point is obtained. At this time, the lighting information at this first intersection point can also be calculated, which includes the direct lighting information from the light source and the reflected light information, refracted light information, and scattered light information on the object surface; then, one or more secondary rays are generated from this first - level intersection point and continue to pass through the scene in a random direction. When intersecting with object 2 in the scene, the second - level intersection point is obtained. At this time, the lighting information at this second intersection point can also be calculated; and so on, recursively pursuing the secondary rays until reaching a predetermined maximum depth or the ray is absorbed, thereby obtaining a lighting path including the first - level intersection point, the second - level intersection point, and the third - level intersection point. Among them, the above - mentioned scene can be a game scene, a virtual reality scene, or an augmented reality scene, etc.

[0126] In one embodiment, the illumination path may include an initial segment path and an extended path. Therefore, during the process of generating the illumination path, the terminal may generate the initial segment path based on the first-level intersection point, the second-level intersection point, and the third-level intersection point, and perform path extension on the initial illumination path to obtain the extended path. In the above solution, the first-level intersection point, the second-level intersection point, and the third-level intersection point that do not meet the illumination rendering conditions are found in a segmented manner first, and then the path of the initial segment formed by the first-level intersection point, the second-level intersection point, and the third-level intersection point is extended, so that the extended path composed of object nodes that meet the illumination rendering conditions can be found, and thus the object nodes that meet the illumination rendering conditions and those that do not can be distinguished, facilitating the accurate and rapid search for the illumination information of the object nodes that meet the illumination rendering conditions.

[0127] For example, continuing with the above example, after obtaining the initial segment path composed of the first-level intersection point, the second-level intersection point, and the third-level intersection point, one or more secondary rays are generated at the third-level intersection point and continue to pass through the scene in a random direction. When intersecting with other objects in the scene, the fourth-level intersection point is obtained. At this time, the illumination information at the fourth intersection point can also be calculated. And so on, recursively pursuing the secondary rays until reaching the predetermined maximum depth or the ray is absorbed, thereby obtaining the extended path.

[0128] S1010. Along the illumination path, obtain the illumination information of the second object vertices that meet the illumination rendering conditions in the rendered image.

[0129] In one embodiment, the terminal may first obtain the pre-configured first number of ray reflections, and determine the vertex search quantity according to the first number of ray reflections and the number of first object vertices. Therefore, when searching for the illumination information, along this illumination path, the second object vertices that meet the illumination rendering conditions in the rendered image can be searched according to the vertex search quantity, so that the number of times of reflected light extraction can be controlled, avoiding excessive calculation amount, which is beneficial to accelerating the speed of network training and better meeting the real-time requirement of rendering.

[0130] Among them, the first number of ray reflections may be a limit value for the reflection of light between object surfaces during the real-time training of the neural network, and this value is used to control when the ray reflection stops. The first number of ray reflections can be pre-configured by the user according to the actual situation. For example, when the user requires high picture quality, a larger first number of ray reflections can be configured, that is, the larger the first number of ray reflections, the higher the picture quality. Another example is that when the user requires high fluency, a smaller first number of ray reflections can be configured, that is, the smaller the first number of ray reflections, the higher the fluency of the picture.

[0131] For example, in a game scenario, the terminal can first obtain the number of light reflection times pre-configured by the user on the game configuration page, and use the difference between the first number of light reflection times and the number of first object vertices as the vertex search quantity. Therefore, when searching for lighting information, the lighting information of the second object vertices whose difference in the rendered game video frames meets the lighting rendering condition is obtained according to the vertex search quantity, so that the number of reflected lights extracted during the game process can be controlled, avoiding excessive computational load, which can help accelerate the speed of network training and better meet the real-time requirements of the game.

[0132] In one embodiment, considering that the lighting path can include an initial path and an extended path, the initial path is generated by the first-level intersection point, the second-level intersection point, and the third-level intersection point, and the extended path is a path extended based on the initial path. Therefore, when searching for lighting information, the terminal can obtain the lighting information of the second object vertices that meet the lighting rendering condition in the rendered image along the extended path. By distinguishing the object nodes that meet the lighting rendering condition from those that do not, the lighting information of the object nodes that meet the lighting rendering condition can be accurately and quickly found.

[0133] In the above embodiment, the first object vertices that do not meet the lighting rendering condition are found by means of path tracing, and the lighting information of the first object vertices that meet the lighting rendering condition is obtained on the lighting path where the first object vertices are located, which can more accurately simulate complex lighting effects, such as accurately simulating soft shadows, indirect lighting, diffuse reflection, and transparency, etc., so that accurate lighting information that meets the lighting rendering condition can be obtained.

[0134] In one embodiment, before performing lighting rendering, relevant lighting rendering parameters can be configured. When performing lighting rendering, lighting information extraction and lighting rendering are performed according to the configured lighting rendering parameters, which is beneficial to improving the rendering effect and meeting the image quality or fluency requirements of different users. Specifically, the terminal can display the configuration page of the interactive application; in response to the first configuration operation triggered on the configuration page, configure the number of light reflection times on the configuration page; in response to the second configuration operation triggered on the configuration page, configure the on / off state of the radiation cache.

[0135] Among them, the number of light reflection times can be the limit value for the reflection of light between object surfaces during neural network training or real-time inference, and this value is used to control when the light reflection stops.

[0136] The radiation cache on / off state can be the on / off state of the neural radiation cache (NRC). When the radiation cache is in the on state, the radiation cache method is used for lighting rendering, that is, the method of the present application is used for lighting rendering, such as first performing a lighting effect detection on the rendered image, and then determining whether the neural network needs to be trained based on the detection results, so as to use the neural network to generate lighting information for the vertices of each object in the image to be rendered, and finally using the lighting information to perform lighting rendering on the image to be rendered.

[0137] In addition, you can also configure other lighting rendering parameters in the configuration page, such as whether to display the light information of the current pixel (such as Ray stats), whether to use the radiation cache to synthesize the current pixel (such as Visualize NRC), reset the parameters in the neural network (which can be set through the reset network button), and the learning rate of the neural network, etc. Figure 11 Here, Figure 11 The parameters involved are described as follows:

[0138] Enable NRC: Indicates whether the neural radiation cache is turned on or off, which is used to control whether the neural radiation cache lighting rendering method is turned on;

[0139] Max inference bounces: The maximum number of times light is reflected, refracted, and scattered between surfaces in the scene when the neural network is doing real-time inference;

[0140] Max training suffix bounces: The maximum number of times light is reflected, refracted, and scattered between the surfaces of objects in the scene when the neural network is training;

[0141] Max RR suffix bounces: The maximum number of times light is allowed to bounce between the surfaces of objects in the scene when the RR roulette technique is applied during path tracing.

[0142] Terminate threshold inference: The limit value for ray bouncing during real-time inference of the neural network. This value is used to control when ray bouncing is terminated.

[0143] Terminate threshold suffix: When training a neural network, the limit value for ray bouncing is used to control when ray bouncing is terminated.

[0144] Ray stats: Indicates whether to display the ray information of the current pixel. If checked, the ray information of the current pixel will be displayed; otherwise, it is turned off.

[0145] Visualize NRC: Indicates whether to use the NRC method for rendering when rendering the current pixel.

[0146] Visualize mode checkbox: Provides a radiance bias mode that only shows the results of the original path tracing and a composited radiance mode that shows the final NRC results.

[0147] Reset network button in Network Params: Can reset the parameters in the neural network.

[0148] The learning rate in Network Params is the learning rate of the neural network.

[0149] It should be understood that although each step in the flowchart involved in the above-described embodiments is shown in sequence according to the indication of the arrow, these steps do not necessarily need to be executed in the order indicated by the arrow. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0150] Based on the same inventive concept, the embodiments of the present application also provide a light rendering device for implementing the above-mentioned light rendering method. The implementation solutions provided by this device for solving problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the following light rendering devices can refer to the limitations on the light rendering method in the above text, and will not be repeated here.

[0151] In one embodiment, as Figure 12 shown, a light rendering device is provided, including: a detection module 1202, a search module 1204, a training module 1206, an extraction module 1208, and a rendering module 1210, where:

[0152] The detection module 1202 is configured to perform a light effect detection on a rendered image before performing light rendering on the image to be rendered, so as to obtain a detection result; the rendered image is a previous frame image of the image to be rendered.

[0153] The search module 1204 is configured to, when the detection result indicates that there is a first object vertex in the rendered image that does not meet the light rendering condition, obtain the light information of a second object vertex in the rendered image that meets the light rendering condition along the light path where the first object vertex is located.

[0154] The training module 1206 is configured to train a neural network based on the light information of the second object vertex to obtain a trained neural network.

[0155] The extraction module 1208 is configured to extract the light information of each object vertex including the first object vertex in the image to be rendered through the trained neural network, so as to obtain the light information of each object vertex.

[0156] The rendering module 1210 is configured to perform light rendering on the image to be rendered according to the light information of each object vertex.

[0157] In the above embodiment, before performing light rendering on the image to be rendered, first perform a light effect detection on the rendered image. When it is detected that there is a first object vertex in the rendered image that does not meet the light rendering condition, obtain the light information of a second object vertex on the same light path as the first object vertex and meeting the light rendering condition, and use the obtained light information for training the neural network, which can enable the neural network to learn to extract the light information that meets the light rendering condition for the first object vertex. Therefore, even if the objects or viewpoints in the scene change dynamically, the trained neural network can still extract the light information that meets the light rendering condition, and using this light information can effectively improve the light rendering effect of the image to be rendered; in addition, only using the light information of the second object vertex on the same light path as the first object vertex and meeting the light rendering condition to train the neural network can effectively accelerate the training speed and meet the real-time requirement of light rendering.

[0158] In one of the embodiments, the first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point.

[0159] The lookup module 1204 is further configured to determine a first-level intersection point in the rendered image that intersects with the direct light of the light source; generate a first secondary ray at the first-level intersection point, and determine a point in the rendered image that intersects with the first secondary ray to obtain a second-level intersection point; generate a second secondary ray at the second-level intersection point, and determine a point in the rendered image that intersects with the second secondary ray to obtain a third-level intersection point; generate an illumination path including the first-level intersection point, the second-level intersection point, and the third-level intersection point; and along the illumination path, obtain the illumination information of the second object vertices in the rendered image that meet the illumination rendering conditions.

[0160] In one embodiment, the lookup module 1204 is further configured to generate an initial path based on the first-level intersection point, the second-level intersection point, and the third-level intersection point; extend the initial illumination path to obtain an extended path; and along the extended path, look up the second object vertices in the rendered image that meet the illumination rendering conditions.

[0161] In one embodiment, as Figure 13 shown, the apparatus further includes:

[0162] A first acquisition module 1212, configured to acquire a pre-configured first light reflection count;

[0163] A determination module 1214, configured to determine the vertex lookup count according to the first light reflection count and the number of first object vertices;

[0164] The lookup module 1204 is further configured to look up the second object vertices in the rendered image that meet the illumination rendering conditions along the illumination path according to the vertex lookup count.

[0165] In one embodiment, as Figure 13 shown, the apparatus further includes:

[0166] A display module 1216, configured to display a configuration page of an interactive application;

[0167] A configuration module 1218, configured to, in response to a first configuration operation triggered on the configuration page, configure the light reflection count in the configuration page; and in response to a second configuration operation triggered on the configuration page, configure the radiation cache on / off state in the configuration page.

[0168] In one embodiment, the detection module 1202 is further configured to acquire the configured radiation cache on / off state; when the radiation cache on / off state is the radiation cache on state, perform a light effect detection on the rendered image to obtain a detection result;

[0169] The first acquisition module 1212 is further configured to, when the radiation cache on / off state is the radiation cache off state, acquire the cached illumination information for each object vertex in the to-be-rendered image;

[0170] The rendering module 1210 is further configured to perform lighting rendering on the image to be rendered based on the cached lighting information of each object vertex in the image to be rendered.

[0171] In one embodiment, the extraction module 1208 is further configured to, when the detection result indicates that each object vertex in the rendered image satisfies the lighting rendering condition, extract the lighting information of each object vertex in the image to be rendered through a neural network;

[0172] The rendering module 1210 is further configured to perform lighting rendering on the image to be rendered according to the lighting information of each object vertex in the image to be rendered.

[0173] In one embodiment, the rendering module 1210 is further configured to train at least two sub-networks based on the lighting information of the second object vertices to obtain at least two trained sub-networks; wherein, the at least two sub-networks are obtained by splitting the structure of the neural network.

[0174] In one embodiment, the number of the second object vertices is at least two; as Figure 13 shown, the apparatus further includes:

[0175] The second acquisition module 1220 is configured to acquire the geometric information of at least two second object vertices;

[0176] The training module 1206 is further configured to input the lighting information and geometric information of each second object vertex into each sub-network, so that each sub-network generates the predicted lighting information of each second object vertex based on the input geometric information; and optimize the parameters of each sub-network respectively based on the loss value between the predicted lighting information and the lighting information of each second object vertex.

[0177] In one embodiment, the extraction module 1208 is further configured to acquire pre-configured second light reflection times, light refraction times, and light scattering times; extract the lighting information of each object vertex including the first object vertex in the image to be rendered according to the second light reflection times, light refraction times, and light scattering times; wherein, the lighting information includes direct lighting information, reflected light information, refracted light information, and scattered light information.

[0178] In the above embodiment, by using the path tracing method to find the first object vertex that does not satisfy the lighting rendering condition and acquiring the lighting information of the first object vertex that satisfies the lighting rendering condition on the lighting path where the first object vertex is located, a more accurate simulation of complex lighting effects can be achieved, such as accurately simulating soft shadows, indirect lighting, diffuse reflection, and transparency.

[0179] Each module in the above light rendering device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0180] In one embodiment, a computer device is provided. The computer device can be a terminal or a server. Taking the terminal as an example for illustration, its internal structural diagram can be as Figure 14 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a light rendering method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0181] Those skilled in the art can understand that Figure 14 the structure shown in

[0182] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0183] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above light rendering method are implemented.

[0184] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the steps of the above-mentioned lighting rendering method.

[0185] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0186] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned various methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0187] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0188] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A lighting rendering method, characterized in that, The method includes: Before performing light rendering on the image to be rendered, performing light effect detection on the rendered image to obtain a detection result; the rendered image is a previous frame image of the image to be rendered; When the detection result indicates that there is a first object vertex in the rendered image that does not meet the light rendering condition, along the light path where the first object vertex is located, obtaining the light information of the second object vertex in the rendered image that meets the light rendering condition; Training a neural network based on the light information of the second object vertex to obtain a trained neural network; Through the trained neural network, extracting the light information of each object vertex including the first object vertex in the image to be rendered to obtain the light information of each object vertex; Performing light rendering on the image to be rendered according to the light information of each object vertex.

2. The method according to claim 1, wherein The first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point; the obtaining the light information of the second object vertex in the rendered image that meets the light rendering condition along the light path where the first object vertex is located includes: Determining a first-level intersection point in the rendered image that intersects with the direct light of the light source; Generating a first secondary light ray at the first-level intersection point, and determining a point in the rendered image that intersects with the first secondary light ray to obtain a second-level intersection point; Generating a second secondary light ray at the second-level intersection point, and determining a point in the rendered image that intersects with the second secondary light ray to obtain a third-level intersection point; Generating a light path including the first-level intersection point, the second-level intersection point, and the third-level intersection point; Along the light path, obtaining the light information of the second object vertex in the rendered image that meets the light rendering condition.

3. The method according to claim 2, characterized in that, The light path includes an initial segment path and an extended path; the generating a light path including the first-level intersection point, the second-level intersection point, and the third-level intersection point includes: Generating an initial segment path according to the first-level intersection point, the second-level intersection point, and the third-level intersection point; Extending the initial segment light path to obtain an extended path; The finding the second object vertex in the rendered image that meets the light rendering condition along the light path includes: Along the extended path, finding the second object vertex in the rendered image that meets the light rendering condition.

4. The method according to claim 2, wherein The method further includes: Obtaining a pre-configured first light reflection count; Determining the vertex search count according to the first light reflection count and the number of the first object vertices; The finding the second object vertex in the rendered image that meets the light rendering condition along the light path includes: Along the light path, finding the second object vertex in the rendered image that meets the light rendering condition according to the vertex search count.

5. The method according to claim 4, wherein The method further includes: Displaying a configuration page of an interactive application; In response to a first configuration operation triggered on the configuration page, configuring the light reflection count on the configuration page; In response to a second configuration operation triggered on the configuration page, configuring the radiation cache on / off state on the configuration page.

6. The method according to any one of claims 1 to 5, characterized in that, Performing light effect detection on the rendered image, the obtained detection results include: Obtaining the on / off state of the configured radiation cache; When the on / off state of the radiation cache is the radiation cache on state, performing light effect detection on the rendered image to obtain detection results; The method further includes: when the on / off state of the radiation cache is the radiation cache off state, obtaining the cached lighting information for each object vertex in the to-be-rendered image; Performing lighting rendering on the to-be-rendered image based on the cached lighting information for each object vertex in the to-be-rendered image.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the detection results indicate that each object vertex in the rendered image satisfies the lighting rendering condition, extracting the lighting information for each object vertex in the to-be-rendered image through the neural network; Performing lighting rendering on the to-be-rendered image according to the lighting information for each object vertex in the to-be-rendered image.

8. The method according to any one of claims 1 to 5, characterized in that, Training the neural network based on the lighting information of the second object vertex, the obtained trained neural network includes: Training at least two sub-networks based on the lighting information of the second object vertex to obtain at least two trained sub-networks; Wherein, at least two of the sub-networks are obtained by splitting the structure of the neural network.

9. The method according to claim 8, wherein The number of the second object vertices is at least two; the method further includes: Obtaining the geometric information of at least two of the second object vertices; The training of at least two sub-networks based on the lighting information of the second object vertex includes: Inputting the lighting information and geometric information of each second object vertex into each sub-network, so that each sub-network generates the predicted lighting information of each second object vertex based on the input geometric information; Optimizing the parameters of each sub-network respectively based on the loss value between the predicted lighting information of each second object vertex and the lighting information.

10. The method according to any one of claims 1 to 5, characterized in that, The extraction of the lighting information for each object vertex including the first object vertex in the to-be-rendered image includes: Obtaining the pre-configured second number of light reflections, number of light refractions, and number of light scatterings; Extracting the lighting information for each object vertex including the first object vertex in the to-be-rendered image according to the second number of light reflections, the number of light refractions, and the number of light scatterings; Wherein, the lighting information includes direct lighting information, reflected light information, refracted light information, and scattered light information.

11. A light rendering device, characterized in that, The device includes: A detection module, configured to perform light effect detection on the rendered image before performing lighting rendering on the to-be-rendered image to obtain detection results; the rendered image is the previous frame image of the to-be-rendered image; A search module, configured to, when the detection results indicate that there is a first object vertex in the rendered image that does not satisfy the lighting rendering condition, obtain the lighting information of the second object vertex in the rendered image that satisfies the lighting rendering condition along the lighting path where the first object vertex is located; A training module, configured to train a neural network based on the lighting information of the second object vertex to obtain a trained neural network; An extraction module, configured to extract lighting information of each object vertex including the first object vertex in the to-be-rendered image through the trained neural network, so as to obtain the lighting information of each object vertex; A rendering module, configured to perform lighting rendering on the to-be-rendered image according to the lighting information of each object vertex.

12. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, comprising a computer program, characterized in that, When this computer program is executed by the processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images