Lighting rendering method and apparatus, and computer device, computer-readable storage medium and computer program product

Abstract

The present application relates to a lighting rendering method and apparatus, and a computer device, a computer-readable storage medium and a computer program product. The method can be applied to technologies such as artificial intelligence and intelligent transportation. The method comprises: performing light effect detection on a rendered image, so as to obtain a detection result (202); when the detection result indicates that a first object vertex that does not satisfy a lighting rendering condition is present in the rendered image, along a lighting path where the first object vertex is located, acquiring lighting information of a second object vertex that satisfies the lighting rendering condition in the rendered image (204); training a neural network on the basis of the lighting information of the second object vertex, so as to obtain a trained neural network (206); by means of the trained neural network, performing lighting information extraction on each object vertex including the first object vertex in an image to be rendered, so as to obtain lighting information of each object vertex (208); and on the basis of the lighting information of each object vertex, performing lighting rendering on the image to be rendered (210).

Description

Lighting rendering method, apparatus, computer device, computer-readable storage medium, and computer program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 25, 2023, with application number 2023118011160, and invention name “Lighting rendering method, device, computer equipment, storage medium and program product”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a lighting rendering method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0003] With the development of image processing technology, lighting rendering technology has been widely used in games, film and television, virtual reality and other fields to achieve lighting effects in related scenes, thereby greatly improving the realism of the scenes.

[0004] Traditional lighting rendering solutions usually use radiation caching technology to obtain cached lighting information, and then use the cached lighting information for lighting rendering. In some application scenarios, the position and posture of each object changes dynamically, or the perspective during image display changes dynamically, which causes the lighting information on the object surface to change dynamically. In this case, using cached lighting information for lighting rendering will lead to poor lighting rendering effects.

[0005] Summary of the Invention

[0006] According to various embodiments of the present application, a lighting rendering method, apparatus, computer device, computer-readable storage medium, and computer program product are provided.

[0007] In a first aspect, the present application provides a lighting rendering method, performed by a computer device, the method comprising:

[0008] Before performing illumination rendering on the image to be rendered, performing a light effect detection on the rendered image to obtain a detection result; the rendered image is a previous frame image of the image to be rendered;

[0009] When the detection result indicates that a first object vertex that does not meet the lighting rendering condition exists in the rendered image, obtaining lighting information of a second object vertex that meets the lighting rendering condition in the rendered image along the lighting path where the first object vertex is located;

[0010] training a neural network based on illumination information of the vertices of the second object to obtain a trained neural network;

[0011] Extracting illumination information from each object vertex, including the first object vertex, in the image to be rendered using the trained neural network to obtain illumination information from each object vertex;

[0012] Lighting rendering is performed on the image to be rendered according to the lighting information of each vertex of the object.

[0013] In a second aspect, the present application further provides a lighting rendering device, comprising:

[0014] A detection module is used to perform a light effect detection on a rendered image before performing light rendering on the image to be rendered, and obtain a detection result; the rendered image is a previous frame image of the image to be rendered;

[0015] a search module configured to, when the detection result indicates that a first object vertex that does not meet the lighting rendering condition exists in the rendered image, obtain lighting information of a second object vertex in the rendered image that meets the lighting rendering condition along the lighting path where the first object vertex is located;

[0016] A training module, configured to train a neural network based on illumination information of vertices of the second object to obtain a trained neural network;

[0017] an extraction module, configured to extract illumination information of each object vertex, including the first object vertex, in the image to be rendered using the trained neural network, to obtain illumination information of each object vertex;

[0018] A rendering module is used to perform lighting rendering on the image to be rendered according to the lighting information of each vertex of the object.

[0019] In one embodiment, the first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point; the search module is used to determine a first-level intersection point that intersects with a direct ray of a light source in the rendered image; generate a first secondary ray at the first-level intersection point, and determine a point that intersects with the first secondary ray in the rendered image to obtain a second-level intersection point; generate a second secondary ray at the second-level intersection point, and determine a point that intersects with the second secondary ray in the rendered image to obtain a third-level intersection point; generate a lighting path including the first-level intersection point, the second-level intersection point, and the third-level intersection point; and obtain lighting information of a second object vertex in the rendered image that meets the lighting rendering condition along the lighting path.

[0020] In one embodiment, the search module is further used to generate an initial path based on the first-level intersection, the second-level intersection and the third-level intersection; extend the initial lighting path to obtain an extended path; and search for a second object vertex in the rendered image that meets the lighting rendering condition along the extended path.

[0021] In one embodiment, the device further comprises:

[0022] A first acquisition module, configured to acquire a preconfigured number of first light reflections;

[0023] a determination module, configured to determine a number of vertices to be searched based on the number of reflections of the first light and the number of vertices of the first object;

[0024] The search module is further configured to search for a second object vertex in the rendered image that meets the lighting rendering condition along the lighting path according to the vertex search quantity.

[0025] In one embodiment, the device further comprises:

[0026] Display module, used to display the configuration page of interactive applications;

[0027] The configuration module is used to configure the number of light reflections in the configuration page in response to a first configuration operation triggered on the configuration page; and to configure the radiation cache on / off state in the configuration page in response to a second configuration operation triggered on the configuration page.

[0028] In one embodiment, the detection module is further configured to obtain a configured radiation cache on / off state; when the radiation cache on / off state is a radiation cache on state, perform a light effect detection on the rendered image to obtain a detection result;

[0029] The first acquisition module is further configured to acquire cached illumination information for each vertex of an object in the image to be rendered when the radiation cache on-off state is the radiation cache off state;

[0030] The rendering module is further configured to perform lighting rendering on the image to be rendered based on the cached lighting information for each vertex of the object in the image to be rendered.

[0031] In one embodiment, the extraction module is further configured to extract illumination information of each object vertex in the image to be rendered through the neural network when the detection result indicates that each object vertex in the rendered image satisfies the illumination rendering condition;

[0032] The rendering module is further configured to perform lighting rendering on the image to be rendered according to lighting information of vertices of each object in the image to be rendered.

[0033] In one embodiment, the rendering module is further used to train at least two sub-networks based on the lighting information of the vertices of the second object to obtain at least two trained sub-networks; wherein, at least two of the sub-networks are obtained by structurally splitting the neural network.

[0034] In one embodiment, the number of vertices of the second object is at least two; and the apparatus further comprises:

[0035] A second acquisition module, configured to acquire geometric information of at least two vertices of the second object;

[0036] The training module is also used to input the lighting information and geometric information of each second object vertex into each sub-network, so that each sub-network generates predicted lighting information of each second object vertex based on the input geometric information; and based on the loss value between the predicted lighting information of each second object vertex and the lighting information, optimize the parameters of each sub-network respectively.

[0037] In one embodiment, the extraction module is further used to obtain a preconfigured second light reflection number, light refraction number, and light scattering number; for each object vertex in the image to be rendered, including the first object vertex, lighting information is extracted according to the second light reflection number, the light refraction number, and the light scattering number; wherein the lighting information includes direct lighting information, reflected light information, refracted light information, and scattered light information.

[0038] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned lighting rendering method when executing the computer program.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned lighting rendering method when executed by a processor.

[0040] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above-mentioned lighting rendering method when executed by a processor.

[0041] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG1 is a diagram illustrating an application environment of a lighting rendering method according to an embodiment;

[0043] FIG2 is a schematic diagram of a flow chart of a lighting rendering method according to an embodiment;

[0044] FIG3 is a schematic diagram of a lighting path in one embodiment;

[0045] FIG4 is a schematic diagram of a light path in another embodiment;

[0046] FIG5 a is a schematic diagram of the structure of a neural network in one embodiment;

[0047] FIG5 b is a schematic diagram showing a structure of a sub-network obtained by structurally splitting a neural network in one embodiment;

[0048] FIG6 is a schematic diagram of dividing a rendered image into multiple regions in one embodiment;

[0049] FIG7 is a schematic diagram showing a comparison of rendering results using different lighting rendering methods in one embodiment;

[0050] FIG8 is a schematic diagram showing a comparison of rendering results using different lighting rendering methods in another embodiment;

[0051] FIG9 is a schematic diagram showing a comparison of rendering results using different lighting rendering methods in another embodiment;

[0052] FIG10 is a schematic flow chart of the step of extracting illumination information in one embodiment;

[0053] FIG11 is a schematic diagram of a page for configuring lighting rendering parameters in one embodiment;

[0054] FIG12 is a structural block diagram of a lighting rendering device according to an embodiment;

[0055] FIG13 is a structural block diagram of a lighting rendering device according to an embodiment;

[0056] FIG14 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] It should be noted that in the following description, the terms "first, second and third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first, second and third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0059] The lighting rendering method provided in the embodiments of the present application can be applied in the application environment shown in FIG1 . In this embodiment, a terminal 102 communicates with a server 104 via a network. A data storage system can store data that the server 104 needs to process, such as rendered or to-be-rendered image data. The data storage system can be integrated with the server 104 or placed in the cloud or on another network server.

[0060] The terminal 102 may be a smartphone, tablet computer, laptop computer, desktop computer, IoT device, or portable wearable device. The IoT device may be a smart speaker, smart TV, smart air conditioner, or smart car device. The portable wearable device may be a smart watch, head-mounted device, or the like.

[0061] The server 104 may be an independent physical server or a service node in a blockchain system. A peer-to-peer network is formed between the service nodes in the blockchain system. The peer-to-peer protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP).

[0062] In addition, server 104 can also be a server cluster composed of multiple physical servers, and can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0063] The terminal 102 and the server 104 may be connected via Bluetooth, USB (Universal Serial Bus), or a network, and this application does not impose any limitation thereto.

[0064] In one embodiment, as shown in FIG2 , a lighting rendering method is provided. The method may be executed by a computer device, such as the server or terminal in FIG1 , or the server and the terminal in collaboration. The method is described using the terminal in FIG1 as an example, and includes the following steps:

[0065] 202 , before performing lighting rendering on the image to be rendered, perform a light effect detection on the rendered image to obtain a detection result.

[0066] The image to be rendered may be at least one image that needs to be rendered, an image that is currently being displayed but not yet rendered, or a video frame that is currently being played but not yet rendered. In some practical applications, the image to be rendered may be a game image to be rendered in a game scene, a virtual reality image to be rendered in a virtual reality scene, an augmented reality image to be rendered in an augmented reality scene, or an extended reality image to be rendered in an extended reality scene.

[0067] The rendered image may be at least one image that has been rendered, and is the previous frame image of the image to be rendered. Specifically, the rendered image may be an image that is being displayed (or has been displayed) after the rendering is completed, or a video frame that is being played (or has been played) after the rendering is completed. In actual applications, the rendered image may be a game image rendered in a game scene, or a virtual reality image rendered in a virtual reality scene, or an augmented reality image rendered in an augmented reality scene, or an extended reality image rendered in an extended reality scene. It should be pointed out that the image to be rendered and the rendered image may be images of different frames in the same scene, such as images of different frames in the same game scene.

[0068] Lighting effect can be the rendering effect of lighting, which can be called lighting rendering effect or lighting effect. Therefore, lighting effect detection can refer to the detection of lighting rendering effect. The detection result can be used to indicate whether the global or local lighting rendering effect of the rendered image is good or poor. For example, if an object in the rendered image (such as a virtual character or virtual object in a game) has abnormal image parameters such as being too bright, too dark, or having too much noise, it indicates that the lighting rendering effect of the object is poor, which will affect the visual effect, and thus the object (including each object vertex of the object) does not meet the lighting rendering conditions.

[0069] It should be pointed out that the lighting rendering method of the present application can be applied to game scenes, virtual reality scenes, augmented reality scenes, and extended reality scenes, and in the above scenes, during the image rendering (including lighting rendering), the lighting information of the corresponding object vertices in the rendered image can be used to guide the training of the neural network, and then the trained neural network can be used to extract the lighting information of each object vertex in the image to be rendered, so as to render the lighting effect of the image to be rendered. In other words, the lighting rendering method of the present application can directly apply the initialized neural network to the image rendering process, and perform network training during the rendering process, without the need to pre-train the neural network. In addition, the neural network can also be pre-trained before being applied to image rendering. In this case, the initial few frames of images can also be accurately light-rendered.

[0070] In one embodiment, in order to ensure the lighting rendering effect of the subsequent image to be rendered, the terminal can first perform a lighting effect detection on the rendered image before performing lighting rendering on the image to be rendered, such as detecting at least one image parameter of the number of noise points, brightness or contrast with the surrounding environment of each object in the rendered image, and obtain a detection result; thereby, it can be judged based on the detection result whether the lighting rendering effect of the rendered image can meet the requirements (i.e., the lighting rendering condition) so as to decide whether the neural network needs to be trained, which is beneficial to improving the lighting rendering effect of the image to be rendered and also beneficial to ensuring the smoothness of the image display during the rendering process.

[0071] For example, in a game scene, when the user is playing the game, assuming that the terminal has completed the rendering of game video frames a to c, in order to ensure the lighting rendering effect of subsequent game video frames (such as the game video frame d to be rendered), the game video frames a to c will be subjected to light effect detection at this time to obtain the detection results, so as to determine whether the neural network needs to be trained based on the detection results, thereby ensuring the lighting rendering effect of the game video frame d.

[0072] In one embodiment, the terminal may first determine whether it is necessary to perform a light effect detection on the rendered image. When a light effect detection is required, the terminal performs a light effect detection on the rendered image to obtain a detection result. Specifically, the terminal may first obtain the configured radiation cache on-off state; determine whether a light effect detection is required based on the radiation cache on-off state, and when the radiation cache on-off state is the radiation cache on-off state, perform a light effect detection on the rendered image to obtain a detection result. By configuring the radiation cache on-off state to determine whether to use the neural radiation cache (NRC) method for lighting rendering, it can be beneficial to select different rendering methods under different needs to meet the needs of different users. For example, users with high image quality requirements can choose to turn on the radiation cache, which is beneficial for rendering high-fidelity images.

[0073] For example, when the radiation cache is on, the brightness, light-dark contrast, shadow effect, and radiance of the lighting in the rendered image are detected. For example, the brightness of the unobstructed area is significantly higher than that of the obstructed area. In addition, when light shines on one side of an object, a shadow will appear in the corresponding area on the other side. Therefore, it is possible to detect whether the brightness of the lighting in these two areas and the corresponding shadow effects are reasonable, thereby obtaining the detection result.

[0074] In addition, users with high refresh rate (or high requirements for smoothness) can choose to turn off the neural radiation cache. Therefore, when the radiation cache is on or off, the terminal can obtain the cached lighting information for each vertex of each object in the image to be rendered; based on the cached lighting information for each vertex of each object in the image to be rendered, the image to be rendered is rendered for lighting, thereby ensuring that the high smoothness requirements of the picture are still effectively ensured under high refresh rate conditions.

[0075] 204 , when the detection result indicates that a first object vertex that does not meet the lighting rendering condition exists in the rendered image, obtain lighting information of a second object vertex that meets the lighting rendering condition in the rendered image along the lighting path where the first object vertex is located.

[0076] The first object vertices may be vertices of an object, and the number of the first object vertices may be m, where m is a positive integer greater than or equal to 1. In addition, the second object vertices may also be vertices of an object, and the number of the second object vertices may be n, where n is a positive integer greater than or equal to 1. As shown in FIG3 , the first object vertices may be y0, y1, and y2, and the second object vertices may be y3 and y4.

[0077] The objects mentioned above can be people, animals, plants, objects, buildings, etc. displayed in the image. In some scenarios, the object is a real object in the real world that is captured and presented in the image; in some special scenarios, the object can be a virtual object. For example, in a game scene, the object can be a game object in the game scene, including virtual people (such as game characters controlled by users), virtual animals (such as game monsters or game characters controlled by users), virtual plants, virtual objects, and virtual buildings, etc.

[0078] The illumination path may be the path formed by light propagating between objects, including paths formed by light propagating between objects by at least one of direct illumination, reflection, refraction, or scattering. Correspondingly, the illumination path for a vertex of a first object may be a lighting path that includes the vertex of the first object, and may also include vertices of other objects (such as a vertex of a second object). As shown in FIG3 , the illumination path may be a path formed by y0, y1, y2, y3, and y4, which includes the first object vertices y0, y1, and y2, and the second object vertices y3 and y4.

[0079] The lighting rendering condition can also be called the lighting rendering requirement. Correspondingly, the first object vertex does not meet the lighting rendering condition, which may mean that the lighting rendering effect of the first object vertex does not meet the lighting rendering condition, such as at least one of the number of noise points is greater than a preset number, the brightness is greater than a preset brightness, the darkness is greater than a preset darkness, or the contrast is less than a preset contrast.

[0080] Lighting information can be used to represent the radiance, color, and type of light. Different types of illumination information can include direct, reflected, refracted, and scattered light. Radiosity is a physical quantity that describes the intensity and directional distribution of light. It measures the amount of light energy per unit area passing through an object's surface in a given direction.

[0081] It should be pointed out that the first object vertex does not meet the lighting rendering conditions, which means that the first object vertex has not been learned by the neural network, that is, the neural network has not learned how to extract the lighting information of the first object vertex that meets the lighting rendering conditions, such as the neural network has not learned how to extract the lighting information of the first object vertex that meets the lighting rendering conditions from the initial path formed by the first object vertex (that is, the initial lighting path).

[0082] In one embodiment, when the detection result indicates that a first object vertex in the rendered image does not meet the lighting rendering conditions, the terminal can use a path tracing algorithm to obtain lighting information for a second object vertex that meets the lighting rendering conditions. This algorithm simulates the reflection, refraction, and scattering of light between objects in the environment in the rendered image to obtain lighting information for each vertex on the object surface along the lighting path where the first object vertex is located. Therefore, the path tracing algorithm can obtain accurate lighting information and accurately simulate complex lighting effects during the rendering process, such as shadows, indirect lighting, color, and transparency in complex game scenes.

[0083] Among them, the first object vertex does not meet the lighting rendering conditions, which means that the first object vertex has not been learned by the neural network. At this time, the lighting information of the second object vertex on the same lighting path as the first object vertex that meets the lighting rendering conditions will be obtained, so as to use the lighting information of the second object vertex to train the neural network so that the neural network can learn the first object vertex.

[0084] In another embodiment, when the detection result indicates that each object vertex in the rendered image meets the lighting rendering conditions, the lighting information of each object vertex in the image to be rendered is extracted through a neural network; and the image to be rendered is then rendered based on the lighting information of each object vertex in the image to be rendered. Therefore, when each object vertex meets the lighting rendering conditions, there is no need to train the neural network, and the lighting information of each object vertex extracted by the neural network is directly used to render the image to be rendered, effectively ensuring the smoothness of each frame of the image.

[0085] Among them, when all object vertices meet the lighting rendering conditions, it means that the first object vertex is learned by the neural network, that is, the neural network learns how to extract the lighting information of the first object vertex that meets the lighting rendering conditions, such as the neural network has learned how to extract the lighting information of the first object vertex that meets the lighting rendering conditions from the initial path.

[0086] Therefore, when all object vertices meet the lighting rendering conditions, the terminal does not need to train the neural network, and can directly use the lighting information of each object vertex in the rendered image to determine the lighting information of each object vertex in the image to be rendered, and then use the lighting information of each object vertex in the image to be rendered to perform lighting rendering on the image to be rendered. For example, the object vertices in Figure 4 are the object vertices x0, x1 and x2 in the rendered image. According to the object vertices x0, x1 and x2 in the rendered image, the lighting information of each object vertex in the image to be rendered is inferred, and then the image to be rendered is rendered. For game scenes, this can ensure the smoothness of the display of each frame of the game screen. In addition, for virtual reality scenes, augmented reality scenes and extended reality scenes, it can also ensure the smoothness of the display of each frame of the scene screen.

[0087] 206 : Train a neural network based on the illumination information of the vertices of the second object to obtain a trained neural network.

[0088] The neural network can be a multi-layer perceptron (MLP) network. For example, as shown in FIG5a, in the neural network shown in FIG5a, M in is the input layer, M hidden is the hidden layer (i.e., perception layer), M out is the output layer and ReLU is the activation function.

[0089] The trained neural network may be a neural network trained based on the lighting information of the second object vertices of the previous frame of the rendered image to be rendered, or a neural network trained based on the lighting information of the second object vertices of multiple rendered images.

[0090] In one embodiment, the terminal can obtain geometric information of the vertices of the second object and use the geometric information and lighting information of the vertices of the second object to train the neural network to obtain a trained neural network. It should be emphasized that when training the neural network, only the lighting information and geometric information of the vertices of the second object on the lighting path where the vertex of the first object is located is used for training, rather than using the geometric information and lighting information of all object vertices in the rendered image for training. This can effectively increase the training speed and greatly shorten the training time, thereby meeting the real-time rendering requirements of the image to be rendered.

[0091] The geometric information may include at least one of the position of the second object's vertex, surface roughness, the direction of light propagation at the vertex, the surface normal vector at the vertex, a diffuse vector, or a specular vector.

[0092] This position can be represented using three-dimensional coordinates. To achieve better illumination information extraction for the multilayer perceptron, each dimension of the three-dimensional coordinates is expanded to twelve dimensions using trigonometric encoding before input into the neural network, resulting in a thirty-six-dimensional position. The surface roughness is a one-dimensional vector, which is expanded to four dimensions using identity encoding. The light propagation direction at this geometric vertex is a two-dimensional direction vector, which is expanded to eight dimensions using identity encoding. The surface normal vector at this geometric vertex is a two-dimensional vector, which is expanded to eight dimensions using identity encoding. The scattering vector and specular reflection vector at this geometric vertex are both three-dimensional vectors and are not encoded. After processing in this manner, sixty-two dimensions of geometric information are obtained. This is then supplemented with placeholders to sixty-four dimensions. This sixty-four-dimensional geometric information, along with the illumination information, is input into the neural network for training. Therefore, processing the geometric information of the second object vertex using this method enables the multilayer perceptron to achieve better illumination information extraction.

[0093] In one embodiment, in order to further improve the real-time performance of the rendering process, the neural network can be structurally split in advance to obtain at least two sub-networks, and then the at least two sub-networks can be trained based on the lighting information of the vertices of the second object. This can greatly shorten the network training time, quickly obtain at least two trained sub-networks, and combine them to obtain a trained neural network, thereby effectively meeting the real-time rendering requirements of the image to be rendered.

[0094] The trained neural network includes at least two trained sub-networks. The trained sub-networks may refer to sub-networks obtained after training is completed, or may be referred to as trained sub-networks.

[0095] When decomposing a neural network, the neurons in each network layer are divided into multiple groups, resulting in multiple groups of neurons in each network layer. Then, the groups of neurons in each network layer are combined to form multiple sub-networks. For example, a neural network layer includes an input layer, multiple hidden layers, and an output layer. Refer to the structure diagram at the top of Figure 6. Each network layer of this neural network has a large number of neurons. Let's assume that the number of neurons in each network layer is N = m × n. In this case, the neurons in the input layer, the neurons in each hidden layer, and the neurons in the output layer can be divided into m groups (i.e., the 1st to mth groups of neurons), each with n neurons. Then, the i-th group of neurons in the input layer, the i-th group of neurons in each hidden layer, and the i-th group of neurons in the output layer are combined until the corresponding groups of neurons in each network layer are combined, resulting in m sub-networks. i, m, n, and N are all positive integers, with i ≤ N, m < N, and n < N. It should be noted that the structure and calculation method of each sub-network are the same. After training, the parameter values ​​of each sub-network may differ. For example, the parameter a of sub-network 1 and the parameter a of sub-network 2 after training may be different. It should be noted that the above-mentioned multiple can be two or more than two; furthermore, multiple groups can be two or more than two groups.

[0096] Pre-splitting the neural network structure to obtain at least two sub-networks reduces the number of neurons in the network, effectively reducing the input and output data dimensions, reducing the complexity of the entire network, and greatly reducing the amount of computation. In addition, the use of the Cuda operator enables the split sub-networks to have high-performance concurrent processing capabilities, thus ensuring that network training and inference can be carried out in real time.

[0097] It should be noted that Cuda can be a high-performance computing platform that allows the use of graphics processing units (GPUs) for parallel computing, thereby accelerating the processing speed of computing tasks. Correspondingly, the Cuda operator can refer to an operator that utilizes the parallel computing capabilities of a graphics processing unit (GPU) to distribute computing tasks to multiple computing units (i.e., Cuda cores) for execution. For example, using the Cuda operator, the lighting information and geometric information of each second object vertex can be distributed to the corresponding sub-network for parallel processing, which can accelerate the training speed of the sub-network.

[0098] In one embodiment, the number of second object vertices is at least two; therefore, before training, the terminal can obtain geometric information of at least two second object vertices; then, the illumination information and geometric information of each second object vertex are input into each sub-network, so that each sub-network generates predicted illumination information of each second object vertex based on the input geometric information; and based on the loss value between the predicted illumination information and the illumination information of each second object vertex, the parameters of each sub-network are optimized separately. In addition, before inputting into the sub-network, the terminal can also encode the geometric information of each second object vertex to expand the dimension of the geometric information, which is conducive to improving the illumination information extraction effect of each sensor in the sub-network. Moreover, splitting the neural network into multiple sub-networks for training can greatly reduce the training time and effectively ensure real-time requirements.

[0099] For example, the terminal can obtain geometric information including at least one of the position, surface roughness, propagation direction of light at the vertex, surface normal vector at the vertex, scattering vector or specular reflection vector, and then use the geometric information of each second object vertex as training data and the illumination information of each second object vertex as a label. Then, these training data and illumination data are input into different sub-networks, such as training data 1 and label 1 are input into sub-network 1, training data 2 and label 2 are input into sub-network 2, training data 3 and label 3 are input into sub-network 3, and training data 4 and label 4 are input into sub-network 4, as shown in FIG5b. Thus, the sub-network generates the predicted illumination information of the second object vertex based on the input geometric information, calculates the loss value between the predicted illumination information and the label, and then back-propagates the obtained loss value in the sub-network to optimize the parameters of the sub-network. Among them, H in FIG5b is i represents the illumination feature output by the i-th perception layer, W i It is a learnable matrix parameter used to process the illumination features output by the i-th perception layer, H i+1 =W i *H i .

[0100] In another embodiment, the terminal may divide the rendered image into multiple regions, select multiple object vertices in each region, and then determine the lighting path of the first object vertex among these object vertices, and then continue to screen the second object vertex that meets the lighting rendering conditions among these vertices, and then obtain the corresponding lighting information for training. For example, as shown in Figure 6, the rendered image is divided into different 6×6 grids, and multiple object vertices are selected in each grid. Then, a second object vertex that is on the same lighting path as the first object vertex that meets the lighting rendering conditions and meets the lighting rendering conditions is selected, and then the lighting information corresponding to the second object vertex is used for training.

[0101] 208 , extracting illumination information from each object vertex including the first object vertex in the rendered image through the trained neural network to obtain illumination information of each object vertex.

[0102] The trained neural network performs illumination information on each vertex of the object, which may be performed by the trained neural network using geometric information of each vertex of the object to generate illumination information of each vertex of the object.

[0103] In one embodiment, when a neural network is split into at least two subnetworks and trained to obtain at least two trained subnetworks, the terminal uses the at least two trained subnetworks to extract illumination information from each object vertex in the image to be rendered, including the vertex of the first object, to obtain illumination information for each object vertex. Given that the trained subnetworks have a smaller number of neurons and lower input and output data dimensions, effectively reducing network complexity, extracting illumination information using the trained subnetworks can effectively accelerate illumination information extraction, facilitating real-time illumination rendering.

[0104] For example, taking the game scene as an example, the terminal can obtain the geometric information of the object vertices corresponding to each game object and environmental object in the game video frame to be rendered, and input these geometric information into different trained sub-networks respectively. You can refer to Figure 5b, so that these trained sub-networks can generate the lighting information of the corresponding object vertices according to the input geometric information of the object vertices.

[0105] In one embodiment, when extracting illumination information, the terminal may combine configured light propagation parameters to extract illumination information for each object vertex in the to-be-rendered image, thereby controlling the number of extracted reflected, refracted, and scattered light, thereby avoiding excessive computational effort. Specifically, the terminal may obtain preconfigured second light reflection, light refraction, and light scattering times; and extract illumination information for each object vertex in the to-be-rendered image, including the first object vertex, based on the second light reflection, light refraction, and light scattering times. This allows the terminal to control the number of light reflections, refractions, and scattering times when extracting illumination information.

[0106] The second number of light reflections, the number of light refractions, and the number of light scatterings can be limit values ​​for light reflection, refraction, and scattering between object surfaces during real-time neural network inference. These values ​​are used to control when light reflection, refraction, and scattering are cut off. Furthermore, illumination information includes direct illumination information, reflected light information, refracted light information, and scattered light information.

[0107] 210 , performing lighting rendering on the image to be rendered according to the lighting information of each object vertex.

[0108] Lighting rendering can be the act of rendering lighting effects in real time. Furthermore, real-time rendering is a rendering method in computer graphics that obtains display images within a limited time. The goal is to generate visually appealing display images in the shortest possible time, thereby improving the smoothness of the image in interactive applications (such as gaming, virtual reality, and augmented reality).

[0109] In one embodiment, the terminal can use a lighting rendering function (such as a spherical harmonic function) and perform lighting rendering on each object in the rendered image according to the lighting information of each object's vertex to obtain the rendering result of the global lighting of each object; since the lighting information is generated by a neural network (or a subnetwork of a neural network), the complex integral calculation in traditional path tracing is avoided, the amount of calculation is reduced, and the rendering speed can be effectively shortened, ensuring the real-time requirements of the rendering.

[0110] It should be noted that the lighting rendering in the traditional scheme usually adopts the radiation cache technology to obtain the cached lighting information, and then uses the cached lighting information for lighting rendering. Since in some application scenarios, the position and posture of each object are dynamically changing, or the perspective during the image display process is dynamically changing, which causes the lighting information on the object surface to change dynamically. At this time, using the cached lighting information for lighting rendering will lead to poor lighting rendering effects, as shown in Figure 7 (a), Figure 8 (a) and Figure 9 (a); and the lighting rendering scheme of the present application can effectively improve the lighting rendering effect, as shown in Figure 7 (b), Figure 8 (b) and Figure 9 (b). Among them, radiation cache is a technology in computer graphics for improving the efficiency of global illumination calculations. Its core idea is to pre-cache pre-calculated lighting information so that the cached lighting information can be quickly queried and used when needed.

[0111] In the above embodiment, before performing lighting rendering on the image to be rendered, the rendered image is first subjected to lighting effect detection. When it is detected that there is a first object vertex in the rendered image that does not meet the lighting rendering conditions, lighting information of a second object vertex on the same lighting path as the first object vertex and that meets the lighting rendering conditions is obtained. The obtained lighting information is used to train the neural network, so that the neural network can learn to extract lighting information that meets the lighting rendering conditions for the first object vertex. Therefore, even if the objects or perspectives in the scene change dynamically, the trained neural network can still extract lighting information that meets the lighting rendering conditions. Utilizing this lighting information can effectively improve the lighting rendering effect of the image to be rendered. In addition, only the lighting information of the second object vertex on the same lighting path as the first object vertex and that meets the lighting rendering conditions needs to be used to train the neural network, which can effectively speed up the training speed and meet the real-time requirements of lighting rendering.

[0112] In one embodiment, the terminal may determine the lighting path of the first object vertex in the rendered image; then, along the lighting path of the first object vertex, obtain lighting information of the second object vertex in the rendered image that meets the lighting rendering conditions.

[0113] In another embodiment, the first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point. Therefore, the step of obtaining the illumination information of the second object vertex can refer to step 10, and the specific steps include:

[0114] S1002: Determine, in the rendered image, a first-level intersection point that intersects with a direct ray of the light source.

[0115] A first-level intersection point can refer to the vertex where a direct ray from a light source intersects an object. For example, the point where a ray directly emitted by a light source (i.e., a direct ray) intersects with game object a in the game scene is a first-level intersection point. It is understood that the intersection of a direct ray from a light source and game object a in the game scene indicates that the direct ray from the light source illuminates game object a.

[0116] S1004: Generate a first secondary ray at the first-level intersection point, and determine a point in the rendered image that intersects with the first secondary ray to obtain a second-level intersection point.

[0117] The first secondary light may be light generated when direct light from a light source strikes an object and undergoes at least one of reflection, refraction, or scattering. For example, when direct light from a light source strikes game object a in a game scene, the direct light is reflected from game object a, generating reflected light. This reflected light is the first secondary light.

[0118] The second-level intersection point is the vertex where the first-level ray intersects with the object. For example, when the first-level ray intersects with other game objects (such as game object b) in the game scene, the intersection point is the second-level intersection point.

[0119] S1006 , generating a second secondary ray at the second-level intersection point, and determining a point intersecting with the second secondary ray in the rendered image to obtain a third-level intersection point.

[0120] The second secondary ray may be a ray generated when the first secondary ray, after striking an object, undergoes at least one of reflection, refraction, or scattering. For example, when the first secondary ray strikes a game object B in a game scene, the first secondary ray is reflected by the game object B, generating a reflected ray. This reflected ray is the second secondary ray.

[0121] The third-level intersection point is the vertex where the second-level ray intersects with the object. For example, if the second-level ray intersects with other game objects (such as game object C) in the game scene, the intersection point is the third-level intersection point.

[0122] S1008 , generating a lighting path including first-level intersection points, second-level intersection points, and third-level intersection points.

[0123] It should be noted that, in addition to the first-level intersection points, the second-level intersection points and the third-level intersection points, the lighting path may also include other object nodes.

[0124] For example, when the primary ray (i.e., the direct ray from the light source) intersects with object 1 in the scene, a first-level intersection is obtained. At this time, the lighting information at the first intersection can also be calculated, which includes the direct lighting information from the light source and the reflected light information, refracted light information, and scattered light information from the object surface. Then, one or more secondary rays are generated from the first-level intersection and continue to pass through the scene in a random direction. When they intersect with object 2 in the scene, a second-level intersection is obtained. At this time, the lighting information at the second intersection can also be calculated. Similarly, the secondary rays are recursively pursued until a predetermined maximum depth is reached or the light is absorbed, thereby obtaining a lighting path including the first-level intersection, the second-level intersection, and the third-level intersection. The above-mentioned scene can be a game scene, a virtual reality scene, an augmented reality scene, etc.

[0125] In one embodiment, the lighting path may include an initial path and an extended path; therefore, in the process of generating the lighting path, the terminal may generate the initial path based on the first-level intersections, the second-level intersections, and the third-level intersections; and extend the initial lighting path to obtain the extended path. In the above scheme, the first-level intersections, the second-level intersections, and the third-level intersections that do not meet the lighting rendering conditions are first searched in a segmented manner, and then the initial path composed of the first-level intersections, the second-level intersections, and the third-level intersections are extended, so that the extended path composed of object nodes that meet the lighting rendering conditions can be found, thereby distinguishing object nodes that meet the lighting rendering conditions from those that do not, so as to accurately and quickly find the lighting information of the object nodes that meet the lighting rendering conditions.

[0126] For example, continuing with the above example, after obtaining the initial path consisting of the first-level intersection, the second-level intersection, and the third-level intersection, one or more secondary rays are generated at the third-level intersection, continuing to pass through the scene in a random direction, and when intersecting with other objects in the scene, a fourth-level intersection is obtained, at which point the lighting information at the fourth intersection can also be calculated; and so on, the secondary rays are recursively pursued until a predetermined maximum depth is reached or the light is absorbed, thereby obtaining an extended path.

[0127] S1010 , obtaining lighting information of a second object vertex that meets a lighting rendering condition in a rendered image along a lighting path.

[0128] In one embodiment, the terminal may first obtain a preconfigured number of first light reflections; determine the number of vertex searches based on the number of first light reflections and the number of first object vertices; therefore, when searching for lighting information, the second object vertex that meets the lighting rendering conditions in the rendered image may be searched along the lighting path according to the number of vertex searches, thereby controlling the number of extracted reflected lights and avoiding excessive computational effort, which may help speed up network training and better meet the real-time requirements of rendering.

[0129] The first number of light reflections may be a limit on the reflection of light between object surfaces during real-time training of a neural network. This value is used to control when the light reflection is terminated. The first number of light reflections may be pre-configured by the user based on actual conditions. For example, if the user prioritizes image quality, a larger number of first light reflections may be configured, i.e., the larger the number of first light reflections, the higher the image quality. For another example, if the user prioritizes smoothness, a smaller number of first light reflections may be configured, i.e., the smaller the number of first light reflections, the smoother the image.

[0130] For example, for a game scene, the terminal can first obtain the number of light reflections pre-configured by the user on the game configuration page, and use the difference between the first light reflection number and the number of vertices of the first object as the number of vertex searches. Therefore, when searching for lighting information, the lighting information of the second object vertex whose difference meets the lighting rendering conditions in the rendered game video frame is obtained according to the number of vertex searches. This can control the number of reflected lights extracted during the game, avoid excessive calculations, and help speed up network training to better meet the real-time requirements of the game.

[0131] In one embodiment, considering that the lighting path may include an initial path and an extended path, the initial path being generated by first-level intersections, second-level intersections, and third-level intersections, and the extended path being a path extended from the initial path, when searching for lighting information, the terminal can obtain lighting information for a second object vertex in a rendered image that meets lighting rendering conditions along the extended path. By distinguishing object nodes that meet the lighting rendering conditions from those that do not, lighting information for object nodes that meet the lighting rendering conditions can be accurately and quickly found.

[0132] In the above embodiment, path tracing is used to find the first object vertex that does not meet the lighting rendering conditions, and the lighting information of the first object vertex that meets the lighting rendering conditions is obtained on the lighting path where the first object vertex is located. This can more accurately simulate complex lighting effects, such as accurately simulating soft shadows, indirect lighting, diffuse reflection and transparency, etc., thereby obtaining accurate lighting information that meets the lighting rendering conditions.

[0133] In one embodiment, before performing lighting rendering, relevant lighting rendering parameters can be configured. During lighting rendering, lighting information extraction and lighting rendering can be performed based on the configured lighting rendering parameters, which is beneficial for improving rendering effects and meeting the image quality or smoothness requirements of different users. Specifically, the terminal can display a configuration page for an interactive application; in response to a first configuration operation triggered on the configuration page, configure the number of light reflections on the configuration page, which includes the number of first light reflections and other light reflections; in response to a second configuration operation triggered on the configuration page, configure the radiation cache on / off state on the configuration page.

[0134] Among them, interactive applications can be game applications, video applications (such as video applications that can broadcast live games), trial installation applications and other applications involving lighting rendering. The number of light reflections can be used to control when the light emission is cut off, such as during neural network training or real-time inference, for the limit value of light reflection between object surfaces, this value is used to control when the light reflection is cut off, and can include the first number of light reflections and other light emission numbers. The first number of light reflections can be the limit value for light reflection between object surfaces during real-time training of the neural network, which can be used to control when the light reflection is cut off. The other number of light emission can be the second number of light reflections, which can be used to control when the light reflection is cut off during real-time inference of the neural network.

[0135] The radiation cache on / off state may be the on / off state of a neural radiation cache (NRC), which is a new method for caching radiation based on a neural network, using path tracing data to train a neural network so as to predict radiation at any position in a three-dimensional (3D) scene. When the radiation cache is on, lighting rendering is performed using the radiation cache method, that is, lighting rendering is performed using the method of the present application, such as first performing a lighting effect test on the rendered image, and then determining whether the neural network needs to be trained based on the test results, so as to use the neural network to generate lighting information for each vertex of an object in the image to be rendered, and finally using the lighting information to perform lighting rendering on the image to be rendered.

[0136] In addition, you can also configure other lighting rendering parameters on the configuration page, such as whether to display the light information of the current pixel (such as Ray stats), whether to use the radiation cache to synthesize the current pixel (such as Visualize NRC), reset the parameters in the neural network (which can be set through the reset network button), and the learning rate of the neural network, etc., as shown in Figure 11. Here, the parameters involved in Figure 11 are explained as follows:

[0137] Enable NRC: Indicates whether the neural radiation cache is turned on or off, which is used to control whether the neural radiation cache lighting rendering method is enabled;

[0138] Max inference bounces: The maximum number of times light is reflected, refracted, and scattered between surfaces in the scene when the neural network is doing real-time inference.

[0139] Max training suffix bounces: The maximum number of times light is reflected, refracted, and scattered between the surfaces of objects in the scene while the neural network is training.

[0140] Max RR suffix bounces: The maximum number of times light is allowed to bounce between the surfaces of objects in the scene when the RR suffix technique is applied during path tracing.

[0141] Terminate threshold inference: The limit value for ray bouncing during real-time neural network inference. This value is used to control when ray bouncing is terminated.

[0142] Terminate threshold suffix: This is the limit for raycasting during neural network training. This value is used to control when raycasting is terminated.

[0143] Ray stats: Indicates whether to display the current pixel's ray information. If checked, the current pixel's ray information is displayed, otherwise it is off.

[0144] NRC Visualization (Visualize NRC): Indicates whether to use NRC rendering when rendering the current pixel;

[0145] Visualize mode checkbox: provides a bias of radiance mode that displays only the raw path tracing results, and a composite radiance mode that displays the final NRC results.

[0146] The reset network button in the Network Params: resets the parameters of the neural network.

[0147] The learning rate in Network Params is the learning rate of the neural network.

[0148] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0149] Based on the same inventive concept, embodiments of the present application also provide a lighting rendering device for implementing the aforementioned lighting rendering method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more lighting rendering device embodiments provided below can be found in the above-mentioned limitations of the lighting rendering method and will not be further elaborated here.

[0150] In one embodiment, as shown in FIG12 , a lighting rendering apparatus is provided, comprising: a detection module 1202 , a search module 1204 , a training module 1206 , an extraction module 1208 , and a rendering module 1210 , wherein:

[0151] The detection module 1202 is used to perform a light effect detection on the rendered image before performing light rendering on the image to be rendered, and obtain a detection result; the rendered image is a previous frame image of the image to be rendered;

[0152] A search module 1204 is configured to, when the detection result indicates that a first object vertex that does not meet the lighting rendering condition exists in the rendered image, obtain lighting information of a second object vertex that meets the lighting rendering condition in the rendered image along the lighting path where the first object vertex is located;

[0153] A training module 1206 is configured to train a neural network based on illumination information of vertices of the second object to obtain a trained neural network;

[0154] An extraction module 1208 is configured to extract illumination information from each vertex of an object, including a vertex of the first object, in the rendered image using a trained neural network to obtain illumination information of each vertex of the object;

[0155] The rendering module 1210 is configured to perform lighting rendering on the image to be rendered according to lighting information of each object vertex.

[0156] In the above embodiment, before performing lighting rendering on the image to be rendered, the rendered image is first subjected to lighting effect detection. When it is detected that there is a first object vertex in the rendered image that does not meet the lighting rendering conditions, lighting information of a second object vertex on the same lighting path as the first object vertex and that meets the lighting rendering conditions is obtained. The obtained lighting information is used to train the neural network, so that the neural network can learn to extract lighting information that meets the lighting rendering conditions for the first object vertex. Therefore, even if the objects or perspectives in the scene change dynamically, the trained neural network can still extract lighting information that meets the lighting rendering conditions. Utilizing this lighting information can effectively improve the lighting rendering effect of the image to be rendered. In addition, only the lighting information of the second object vertex on the same lighting path as the first object vertex and that meets the lighting rendering conditions needs to be used to train the neural network, which can effectively speed up the training speed and meet the real-time requirements of lighting rendering.

[0157] In one embodiment, the first object vertices include first-level intersection points, second-level intersection points, and third-level intersection points;

[0158] The search module 1204 is also used to determine a first-level intersection point that intersects with the direct light of the light source in the rendered image; generate a first secondary ray at the first-level intersection point, and determine the point that intersects with the first secondary ray in the rendered image to obtain a second-level intersection point; generate a second secondary ray at the second-level intersection point, and determine the point that intersects with the second secondary ray in the rendered image to obtain a third-level intersection point; generate a lighting path including the first-level intersection point, the second-level intersection point and the third-level intersection point; and obtain lighting information of the second object vertex that meets the lighting rendering conditions in the rendered image along the lighting path.

[0159] In one embodiment, the search module 1204 is further used to generate an initial path based on the first-level intersection points, the second-level intersection points, and the third-level intersection points; extend the initial lighting path to obtain an extended path; and search for a second object vertex in the rendered image that meets the lighting rendering conditions along the extended path.

[0160] In one embodiment, as shown in FIG13 , the device further comprises:

[0161] A first acquisition module 1212 is configured to acquire a preconfigured first light reflection number;

[0162] A determination module 1214 is configured to determine the number of vertices to be searched based on the number of reflections of the first light and the number of vertices of the first object;

[0163] The search module 1204 is further configured to search for a second object vertex that meets the lighting rendering condition in the rendered image along the lighting path according to the vertex search quantity.

[0164] In one embodiment, as shown in FIG13 , the device further comprises:

[0165] Display module 1216, used to display the configuration page of the interactive application;

[0166] The configuration module 1218 is used to configure the number of light reflections in the configuration page in response to a first configuration operation triggered on the configuration page; and to configure the radiation cache on / off state in response to a second configuration operation triggered on the configuration page.

[0167] In one embodiment, the detection module 1202 is further configured to obtain a configured radiation cache on / off state; when the radiation cache on / off state is a radiation cache on state, perform a light effect detection on the rendered image to obtain a detection result;

[0168] The first acquisition module 1212 is further configured to acquire cached illumination information for each vertex of an object in the image to be rendered when the radiation cache on / off state is the radiation cache off state;

[0169] The rendering module 1210 is further configured to perform lighting rendering on the image to be rendered based on the cached lighting information for each vertex of the object in the image to be rendered.

[0170] In one embodiment, the extraction module 1208 is further configured to extract illumination information of each vertex of the object in the to-be-rendered image through a neural network when the detection result indicates that each vertex of the object in the rendered image satisfies the illumination rendering condition;

[0171] The rendering module 1210 is further configured to perform lighting rendering on the image to be rendered according to lighting information of vertices of each object in the image to be rendered.

[0172] In one embodiment, the rendering module 1210 is further used to train at least two sub-networks based on the lighting information of the vertices of the second object to obtain at least two trained sub-networks; wherein the at least two sub-networks are obtained by structurally splitting the neural network.

[0173] In one embodiment, the number of vertices of the second object is at least two; as shown in FIG13 , the apparatus further includes:

[0174] A second acquisition module 1220 is configured to acquire geometric information of at least two vertices of a second object;

[0175] The training module 1206 is also used to input the lighting information and geometric information of each second object vertex into each sub-network, so that each sub-network generates the predicted lighting information of each second object vertex based on the input geometric information; based on the loss value between the predicted lighting information and the lighting information of each second object vertex, the parameters of each sub-network are optimized respectively.

[0176] In one embodiment, the extraction module 1208 is further used to obtain a preconfigured number of second light reflections, light refractions, and light scatterings; for each object vertex including the first object vertex in the rendered image, illumination information is extracted according to the number of second light reflections, light refractions, and light scatterings; wherein the illumination information includes direct illumination information, reflected light information, refracted light information, and scattered light information.

[0177] In the above embodiment, path tracing is used to find the first object vertex that does not meet the lighting rendering conditions, and the lighting information of the first object vertex that meets the lighting rendering conditions is obtained on the lighting path where the first object vertex is located. This can more accurately simulate complex lighting effects, such as accurately simulating soft shadows, indirect lighting, diffuse reflection and transparency.

[0178] Each module in the aforementioned lighting rendering device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0179] In one embodiment, a computer device is provided. The computer device may be a terminal or a server. For illustration purposes, a terminal is used as an example. Its internal structure diagram may be shown in FIG14 . The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and computer program stored in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. The wireless communication may be implemented via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a lighting rendering method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0180] Those skilled in the art will understand that the structure shown in FIG14 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned lighting rendering method when executing the computer program.

[0182] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned lighting rendering method are implemented.

[0183] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned lighting rendering method when executed by a processor.

[0184] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0185] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0186] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0187] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A lighting rendering method, executed by a computer device, wherein, The method includes: Before performing light rendering on the image to be rendered, performing light effect detection on the already rendered image to obtain a detection result; the already rendered image is the previous frame image of the image to be rendered; When the detection result indicates that there is a first object vertex in the already rendered image that does not meet the light rendering condition, along the light path where the first object vertex is located, obtaining the light information of the second object vertex in the already rendered image that meets the light rendering condition; Training a neural network based on the light information of the second object vertex to obtain a trained neural network; Extracting the light information of each object vertex including the first object vertex in the image to be rendered through the trained neural network to obtain the light information of each object vertex; Performing light rendering on the image to be rendered according to the light information of each object vertex.

2. The method according to claim 1, wherein The first object vertex includes a first-level intersection point, a second-level intersection point, and a third-level intersection point; the obtaining the light information of the second object vertex in the already rendered image that meets the light rendering condition along the light path where the first object vertex is located includes: Determining a first-level intersection point in the already rendered image that intersects with the direct light of the light source; Generating a first secondary light ray at the first-level intersection point and determining the point that intersects with the first secondary light ray in the already rendered image to obtain a second-level intersection point; Generating a second secondary light ray at the second-level intersection point and determining the point that intersects with the second secondary light ray in the already rendered image to obtain a third-level intersection point; Generating a light path including the first-level intersection point, the second-level intersection point, and the third-level intersection point; Along the light path, obtaining the light information of the second object vertex in the already rendered image that meets the light rendering condition.

3. The method according to claim 2, wherein The light path includes an initial segment path and an extended path; the generating a light path including the first-level intersection point, the second-level intersection point, and the third-level intersection point includes: Generating an initial segment path according to the first-level intersection point, the second-level intersection point, and the third-level intersection point; Extending the initial segment light path to obtain an extended path; The finding the second object vertex in the already rendered image that meets the light rendering condition along the light path includes: Finding the second object vertex in the already rendered image that meets the light rendering condition along the extended path.

4. The method according to claim 2 or 3, wherein, The method further includes: Obtaining a pre-configured first light reflection number; Determining the vertex search number according to the first light reflection number and the number of the first object vertices; The finding the second object vertex in the already rendered image that meets the light rendering condition along the light path includes: Finding the second object vertex in the already rendered image that meets the light rendering condition along the light path according to the vertex search number.

5. The method according to claim 4, wherein The method further includes: Displaying a configuration page of an interactive application; Responding to a first configuration operation triggered on the configuration page, configuring the light reflection number on the configuration page, where the light reflection number includes a first light reflection number and other light reflection numbers; In response to a second configuration operation triggered on the configuration page, configure the radiation cache on / off state in the configuration page.

6. The method according to any one of claims 1 to 5, wherein The optical effect detection of the rendered image to obtain the detection result includes: Obtain the configured radiation cache on / off state; When the radiation cache on / off state is the radiation cache on state, perform optical effect detection on the rendered image to obtain the detection result; The method further includes: when the radiation cache on / off state is the radiation cache off state, obtain the cached lighting information for each object vertex in the to-be-rendered image; Based on the cached lighting information for each object vertex in the to-be-rendered image, perform lighting rendering on the to-be-rendered image.

7. The method according to any one of claims 1 to 5, wherein The method further includes: When the detection result indicates that each object vertex in the rendered image satisfies the lighting rendering condition, extract the lighting information for each object vertex in the to-be-rendered image through the neural network; Based on the lighting information for each object vertex in the to-be-rendered image, perform lighting rendering on the to-be-rendered image.

8. The method according to any one of claims 1 to 5, wherein Training the neural network based on the lighting information of the second object vertex to obtain the trained neural network includes: Training at least two sub-networks based on the lighting information of the second object vertex to obtain at least two trained sub-networks; Among them, at least two of the sub-networks are obtained by splitting the structure of the neural network, and the trained neural network includes at least two of the trained sub-networks.

9. The method according to claim 8, wherein, The number of the second object vertices is at least two; the method further includes: Obtain the geometric information of at least two of the second object vertices; The training of at least two sub-networks based on the lighting information of the second object vertex includes: Input the lighting information and geometric information of each second object vertex into each sub-network, so that each sub-network generates the predicted lighting information of each second object vertex based on the input geometric information; Based on the loss value between the predicted lighting information of each second object vertex and the lighting information, optimize the parameters of each sub-network respectively.

10. The method according to any one of claims 1 to 9, wherein The extraction of the lighting information for each object vertex including the first object vertex in the to-be-rendered image includes: Obtain the pre-configured second number of light reflection times, light refraction times, and light scattering times; For each object vertex including the first object vertex in the to-be-rendered image, extract the lighting information according to the second number of light reflection times, the number of light refraction times, and the number of light scattering times; Among them, the lighting information includes direct lighting information, reflected light information, refracted light information, and scattered light information.

11. A light rendering device, wherein, The device includes: A detection module, configured to perform optical effect detection on a rendered image to obtain a detection result before performing lighting rendering on a to-be-rendered image; the rendered image is a previous frame image of the to-be-rendered image; A search module, configured to, when the detection result indicates that there is a first object vertex in the rendered image that does not satisfy the lighting rendering condition, along the lighting path where the first object vertex is located, obtain the lighting information of the second object vertex in the rendered image that satisfies the lighting rendering condition; A training module, configured to train a neural network based on the lighting information of the second object vertices to obtain a trained neural network; An extraction module, configured to extract the lighting information of each object vertex including the first object vertex in the to-be-rendered image through the trained neural network to obtain the lighting information of each object vertex; A rendering module, configured to perform lighting rendering on the to-be-rendered image according to the lighting information of each object vertex.

12. A computer device, comprising a memory and a processor, the memory storing a computer program, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images