Data processing method, device and equipment and readable storage medium

Abstract

The invention discloses a data processing method, device and equipment and a readable storage medium, and the method comprises the steps: determining surface elements hit by M screen probes in a virtual scene space as to-be-propagated surface elements, and obtaining sampling light rays emitted by the to-be-propagated surface elements in P light ray directions in the virtual scene space; and determining illumination information stored in the surface element hit by the sampling light rays in the P light ray directions in the virtual scene space as indirect illumination information collected by the sampling light rays in the P light ray directions in the virtual scene space, thereby generating surface element mixed illumination information of the surface element to be propagated, and probe mixed illumination information of the M screen probes is generated based on the surface element mixed illumination information and the to-be-propagated surface element, and illumination rendering is performed on the screen image based on the probe mixed illumination information. By adopting the method and the device, indirect illumination information generated by multiple reflections of surface elements in a virtual scene space can be collected, and the reality sense of screen picture illumination rendering and the user experience are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and readable storage medium. Background Art

[0002] In recent years, with the development of computer technology, industries such as games, film and television, and virtual reality have shown a booming development trend. Indirect lighting algorithms have been widely used in high-realism rendering of the above scenes. Indirect lighting can simulate the lighting effect produced by light being reflected and refracted multiple times in the scene and finally reaching the surface of the object.

[0003] The current indirect lighting algorithm is the surface finite element algorithm (Surfel Global Illumination), which generates surface finite elements by using the spatial information provided by the geometry buffer corresponding to the screen image, and collects the indirect lighting stored in the geometry buffer within the screen image based on the screen probe. Specifically, when there are surface elements around the hit point of the ray emitted by the screen probe within the screen image, the indirect lighting of the surrounding surface elements is mixed and the result is returned as the indirect lighting of the screen probe.

[0004] However, when there are no facets around the ray hit point, no valid results can be returned, resulting in inaccurate indirect lighting collected by the screen probe, and the facets are unable to collect indirect lighting from other facets. At the same time, only indirect lighting is collected within the screen image, ignoring the lighting effects of other virtual content in the virtual scene space. As a result, the lighting effects of light after multiple reflections and refractions in the virtual scene space cannot be restored, resulting in a lack of realism, a dark rendered simulated screen image, and a poor user experience. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device and readable storage medium, and the lighting information stored in the surface element to be collected includes at least one of the lighting information at the location of the surface element to be collected or the lighting information stored in the surface element that is continuously hit by the surface element to be collected. Therefore, the indirect lighting information generated by multiple reflections of the surface element in the virtual scene space can be collected, thereby improving the lighting rendering effect of the screen image and the user experience.

[0006] An embodiment of the present application provides a data processing method, including:

[0007] Obtain M screen probes set on the screen image, determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated, and obtain sampling rays emitted by the surface elements to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented in the screen image and virtual content outside the screen image; M and P are positive integers;

[0008] Determine the surface element hit by the sampling rays of the P light directions in the virtual scene space as the surface element to be collected, and determine the illumination information stored in the surface element to be collected as the indirect illumination information collected in the virtual scene space by the sampling rays of the P light directions; the illumination information stored in the surface element to be collected includes at least one of illumination information at the location of the surface element to be collected and illumination information stored in the surface element subsequently hit by the surface element to be collected;

[0009] Based on the indirect lighting information collected by the sampling light in P light directions in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and based on the surface element mixed lighting information and the surface element to be propagated, the probe mixed lighting information of M screen probes is generated, and the screen image is rendered based on the probe mixed lighting information.

[0010] The P light directions include the target light direction; the surface elements hit by the sampling light rays of the P light directions in the virtual scene space are determined as the surface elements to be collected, including:

[0011] Determine a position in the virtual scene space hit by the sampling light in the target light direction based on the sampling light in the target light direction and the world coordinates of the virtual content in the virtual scene space;

[0012] If a surface element exists at the position hit by the sampling light in the target light direction in the virtual scene space, the surface element hit by the sampling light in the target light direction is determined as the surface element to be collected;

[0013] If there is no face element at the position hit by the sampling light in the target light direction in the virtual scene space, then the position information of the position hit by the sampling light in the target light direction is obtained, and based on the position information, a new face element is generated in the virtual scene space, and the new face element is determined as the face element to be collected; the new face element is used to be determined as the new face element to be propagated in the next frame of the screen image.

[0014] Among them, the new surface element to be propagated is used to continue to emit sampling light along P light directions in the next frame, and is used to generate the surface element mixed lighting information of the new surface element to be propagated through the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0015] The indirect illumination information collected by the sampling light rays in the P light directions in the virtual scene space is used to generate the surface element mixed illumination information of the surface element to be propagated, including:

[0016] Obtain the illumination weights corresponding to the P light directions, and determine the illumination values ​​corresponding to the P light directions based on the indirect illumination information collected by the sampled light of the P light directions in the virtual scene space;

[0017] The surface element mixed illumination information of the surface element to be propagated is determined based on the P illumination values ​​and the P illumination weights.

[0018] Among them, the P light directions include the target light direction; obtaining the illumination weights corresponding to the P light directions respectively includes:

[0019] Obtaining a normal direction of a position hit by the target light direction in the virtual scene space, and determining a first parameter value of the target light direction based on the normal direction and the target light direction;

[0020] Based on the number P of light directions and the sampling probability density corresponding to each light direction, the second parameter value of the target light direction is determined, and the illumination weight corresponding to the target light direction is generated through the first parameter value and the second parameter value; the sampling probability density is the probability that the surface element to be propagated is emitted along P light directions.

[0021] The M screen probes include a target screen probe, and the P light directions include a target light direction. The probe mixed lighting information of the M screen probes is generated based on the surface element mixed lighting information and the surface element to be propagated, including:

[0022] Determine the surface element to be propagated that is hit by the probe light emitted by the target screen probe in the virtual scene space along the target light direction as the surface element to be selected, obtain historical illumination information of the surface element to be selected, and determine the mixed sampling illumination information of the surface element to be selected in the target light direction based on the surface element mixed illumination information and the historical illumination information of the surface element to be selected;

[0023] Based on the mixed sampling illumination information collected by the probe rays of P light directions in the virtual scene space, the probe mixed illumination information of the target screen probe is generated.

[0024] The screen image is divided into M pixel images, each pixel image is provided with a screen probe, the M pixel images include a target pixel image, and the M screen probes include a target screen probe; the screen image is illuminated and rendered based on the probe mixed illumination information, including:

[0025] Acquire Q neighboring pixel images of the target pixel image, and determine the screen probes in the Q neighboring pixel images as neighboring screen probes; Q is a positive integer;

[0026] Generate probe weights corresponding to the Q neighboring screen probes based on the pixel distances between the Q neighboring screen probes and the target pixel image and the pixel depths corresponding to the Q neighboring screen probes; the pixel depth is the pixel distance between the neighboring screen probe and the camera position corresponding to the screen image;

[0027] Based on the probe mixed lighting information and probe weights corresponding to the Q neighboring screen probes, the neighboring mixed lighting information of the target screen probe is generated. Based on the probe mixed lighting information and the neighboring mixed lighting information of the target screen probe, the target pixel image in the screen image is illuminated and rendered.

[0028] The screen image is presented by a target rendering frame, and the surface element stores the last used frame of the surface element through the surface element cache; the method further includes:

[0029] A bin whose interval between the last used frame and the target rendering frame in the bin cache is greater than or equal to the recycling threshold is determined as a bin to be recycled; the last used frame refers to the picture frame in which the probe ray emitted by the screen probe or the sampling ray emitted by the bin to be propagated collects illumination information for the last time;

[0030] The position information of the surface element to be recycled in the virtual scene space is obtained, and based on the position information, the surface element to be recycled is recycled in a post-rendering frame; the post-rendering frame is the next frame of the target rendering frame.

[0031] The virtual scene space is divided into T world grids, each world grid is used to store S facets within the grid space; T and S are positive integers; and the method further includes:

[0032] When a new facet is detected to be generated in the virtual scene space, the newly generated facet is determined as a facet to be stored, and position information of the facet to be stored in the virtual scene space is obtained;

[0033] Based on the position information of the face element to be stored, the face element to be stored is stored in the world grid corresponding to T world grids to obtain the stored face element; the stored face element is used as the face element to be propagated when it is hit by M screen probes in the virtual scene space.

[0034] Among them, also include:

[0035] Obtain the stored surfels stored in each of the T world grids, and store the location information, material information, and historical lighting information of the last used frame and the target rendering frame corresponding to the stored surfels in the surfel caches corresponding to the T world grids; the historical lighting information of the target rendering frame is determined based on the surfel mixed lighting information of the target rendering frame and the historical lighting information of the previous rendering frame, where the previous rendering frame is the frame previous to the target rendering frame;

[0036] The illumination information stored in the facet to be collected is determined as the indirect illumination information collected by the sampling light rays in P light directions in the virtual scene space, including:

[0037] Based on the world grid and position information corresponding to the facet to be collected, the lighting information stored in the facet to be collected is obtained in the facet cache, and the lighting information stored in the facet to be collected is determined as the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0038] The screen image is divided into M pixel images, M screen probes set on the screen image are obtained, and the surface elements hit by the M screen probes in the virtual scene space are determined as the surface elements to be propagated, including:

[0039] Generate screen probes at random positions in each pixel image, and emit probe rays along P ray directions in the virtual scene space through M screen probes;

[0040] At the position hit by the probe light in the virtual scene space, a surface element is generated, and the generated surface element is determined as the surface element to be propagated.

[0041] In one aspect, an embodiment of the present application provides a data processing device, including:

[0042] A facet determination module is configured to obtain M screen probes set on the screen image, determine the facets hit by the M screen probes in the virtual scene space as facets to be propagated, and obtain sampling rays emitted by the facets to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented on the screen image as well as virtual content outside the screen image; M and P are positive integers;

[0043] An illumination collection module is configured to determine the facets hit by the sampling rays of P light directions in the virtual scene space as facets to be collected, and to determine the illumination information stored in the facets to be collected as the indirect illumination information collected in the virtual scene space by the sampling rays of the P light directions; the illumination information stored in the facets to be collected includes at least one of illumination information at the location of the facet to be collected and illumination information stored in facets subsequently hit by the facet to be collected;

[0044] The lighting processing module is used to generate the surface element mixed lighting information of the surface element to be propagated based on the indirect lighting information collected by the sampling light in P light directions in the virtual scene space, generate the probe mixed lighting information of M screen probes based on the surface element mixed lighting information and the surface element to be propagated, and perform lighting rendering on the screen image based on the probe mixed lighting information.

[0045] In one possible implementation, the P light directions include a target light direction; and the illumination collection module is configured to determine, when the sampled light rays of the P light directions hit a surface element in the virtual scene space as a surface element to be collected, specifically perform the following operations:

[0046] Determine a position in the virtual scene space hit by the sampling light in the target light direction based on the sampling light in the target light direction and the world coordinates of the virtual content in the virtual scene space;

[0047] If a surface element exists at the position hit by the sampling light in the target light direction in the virtual scene space, the surface element hit by the sampling light in the target light direction is determined as the surface element to be collected;

[0048] If there is no face element at the position hit by the sampling light in the target light direction in the virtual scene space, then the position information of the position hit by the sampling light in the target light direction is obtained, and based on the position information, a new face element is generated in the virtual scene space, and the new face element is determined as the face element to be collected; the new face element is used to be determined as the new face element to be propagated in the next frame of the screen image.

[0049] Among them, the new surface element to be propagated is used to continue to emit sampling light along P light directions in the next frame, and is used to generate the surface element mixed lighting information of the new surface element to be propagated through the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0050] In one possible implementation, the illumination processing module is configured to generate the surface element mixed illumination information of the surface element to be propagated based on the indirect illumination information collected by the sampling light rays in the P light directions in the virtual scene space, and specifically to perform the following operations:

[0051] Obtain the illumination weights corresponding to the P light directions, and determine the illumination values ​​corresponding to the P light directions based on the indirect illumination information collected by the sampled light of the P light directions in the virtual scene space;

[0052] The surface element mixed illumination information of the surface element to be propagated is determined based on the P illumination values ​​and the P illumination weights.

[0053] In a possible implementation, the P light directions include a target light direction; and when the illumination processing module is configured to obtain illumination weights corresponding to the P light directions, the illumination processing module is configured to perform the following operations:

[0054] Obtaining a normal direction of a position hit by the target light direction in the virtual scene space, and determining a first parameter value of the target light direction based on the normal direction and the target light direction;

[0055] Based on the number P of light directions and the sampling probability density corresponding to each light direction, the second parameter value of the target light direction is determined, and the illumination weight corresponding to the target light direction is generated through the first parameter value and the second parameter value; the sampling probability density is the probability that the surface element to be propagated is emitted along P light directions.

[0056] In one possible implementation, the M screen probes include a target screen probe, and the P light directions include a target light direction. When the illumination processing module is configured to generate probe mixed illumination information of the M screen probes based on the surface element mixed illumination information and the surface element to be propagated, the illumination processing module is specifically configured to perform the following operations:

[0057] Determine the surface element to be propagated that is hit by the probe light emitted by the target screen probe in the virtual scene space along the target light direction as the surface element to be selected, obtain historical illumination information of the surface element to be selected, and determine the mixed sampling illumination information of the surface element to be selected in the target light direction based on the surface element mixed illumination information and the historical illumination information of the surface element to be selected;

[0058] Based on the mixed sampling illumination information collected by the probe rays of P light directions in the virtual scene space, the probe mixed illumination information of the target screen probe is generated.

[0059] In one possible implementation, a screen image is divided into M pixel images, each pixel image is provided with a screen probe, the M pixel images include a target pixel image, and the M screen probes include a target screen probe; when the lighting processing module is used to perform lighting rendering on the screen image based on the probe mixed lighting information, it is specifically used to perform the following operations:

[0060] Acquire Q neighboring pixel images of the target pixel image, and determine the screen probes in the Q neighboring pixel images as neighboring screen probes; Q is a positive integer;

[0061] Generate probe weights corresponding to the Q neighboring screen probes based on the pixel distances between the Q neighboring screen probes and the target pixel image and the pixel depths corresponding to the Q neighboring screen probes; the pixel depth is the pixel distance between the neighboring screen probe and the camera position corresponding to the screen image;

[0062] Based on the probe mixed lighting information and probe weights corresponding to the Q neighboring screen probes, the neighboring mixed lighting information of the target screen probe is generated. Based on the probe mixed lighting information and the neighboring mixed lighting information of the target screen probe, the target pixel image in the screen image is illuminated and rendered.

[0063] In a possible implementation, a screen image is presented by a target rendering frame, and a surface element stores a last used frame of the surface element via a surface element cache; the lighting processing module is further configured to perform the following operations:

[0064] A bin whose interval between the last used frame and the target rendering frame in the bin cache is greater than or equal to the recycling threshold is determined as a bin to be recycled; the last used frame refers to the picture frame in which the probe ray emitted by the screen probe or the sampling ray emitted by the bin to be propagated collects illumination information for the last time;

[0065] The position information of the surface element to be recycled in the virtual scene space is obtained, and based on the position information, the surface element to be recycled is recycled in a post-rendering frame; the post-rendering frame is the next frame of the target rendering frame.

[0066] In one possible implementation, the virtual scene space is divided into T world grids, each world grid is used to store S surfels within the grid space; T and S are positive integers; and the lighting processing module is further used to perform the following operations:

[0067] When a new facet is detected to be generated in the virtual scene space, the newly generated facet is determined as a facet to be stored, and position information of the facet to be stored in the virtual scene space is obtained;

[0068] Based on the position information of the face element to be stored, the face element to be stored is stored in the world grid corresponding to T world grids to obtain the stored face element; the stored face element is used as the face element to be propagated when it is hit by M screen probes in the virtual scene space.

[0069] In a possible implementation, the lighting processing module is further configured to perform the following operations:

[0070] Obtain the stored surfels stored in each of the T world grids, and store the location information, material information, and historical lighting information of the last used frame and the target rendering frame corresponding to the stored surfels in the surfel caches corresponding to the T world grids; the historical lighting information of the target rendering frame is determined based on the surfel mixed lighting information of the target rendering frame and the historical lighting information of the previous rendering frame, where the previous rendering frame is the frame previous to the target rendering frame;

[0071] The illumination processing module is used to determine the illumination information stored in the facet to be collected as the indirect illumination information collected by the sampling light rays of P light directions in the virtual scene space, and specifically to perform the following operations:

[0072] Based on the world grid and position information corresponding to the facet to be collected, the lighting information stored in the facet to be collected is obtained in the facet cache, and the lighting information stored in the facet to be collected is determined as the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0073] In one possible implementation, the screen is divided into M pixel screens. The surfel determination module is configured to obtain M screen probes set on the screen and determine the surfels hit by the M screen probes in the virtual scene space as the surfels to be propagated. Specifically, the module is configured to perform the following operations:

[0074] Generate screen probes at random positions in each pixel image, and emit probe rays along P ray directions in the virtual scene space through M screen probes;

[0075] At the position hit by the probe light in the virtual scene space, a surface element is generated, and the generated surface element is determined as the surface element to be propagated.

[0076] An embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;

[0077] The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, and the memory is used to store computer programs. When the computer program is executed by the processor, the computer device executes the method provided in the embodiment of the present application.

[0078] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

[0079] In one aspect, an embodiment of the present application provides a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in the embodiment of the present application.

[0080] The embodiment of the present application determines the surface element hit by the screen probe in the virtual scene space as the surface element to be propagated, determines the surface element hit by the sampling light emitted by the surface element to be propagated along each light direction in the virtual scene space as the surface element to be collected, and determines the lighting information stored in the surface element to be collected as the indirect lighting information collected by the sampling light hit in the virtual scene space. The lighting information stored in the surface element to be collected includes at least one of the lighting information at the position of the surface element to be collected or the lighting information stored in the surface element hit by the surface element to be collected. Therefore, multiple reflections of the surface elements in the virtual scene space can be collected. The indirect lighting information generated by the radiation avoids the lack of realism in the lighting effect caused by only collecting lighting information within the screen. The indirect lighting information collected by the sampling light in the virtual scene space is used to generate the surface element mixed lighting information of the surface element to be propagated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0082] Figure 1 This is a schematic diagram of a network architecture provided by an embodiment of the present application;

[0083] Figure 2 This is a data processing scenario provided by the embodiment of the present application. Figure 1 ;

[0084] Figure 3 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 1 ;

[0085] Figure 4 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 2 ;

[0086] Figure 5 This is a data processing scenario provided by the embodiment of the present application. Figure 2 ;

[0087] Figure 6 This is a data processing scenario provided by the embodiment of the present application. Figure 3 ;

[0088] Figure 7 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 3 ;

[0089] Figure 8 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0090] Figure 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0092] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying and measuring targets, and then further performing image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection.

[0093] As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, and simultaneous localization and mapping.

[0094] The solutions provided in the embodiments of this application involve technologies such as computer vision, and are specifically illustrated by the following embodiments.

[0095] See Figure 1 , Figure 1 This is a network architecture diagram provided by an embodiment of the present application. Figure 1As shown, the network architecture may include a business server 100 and a terminal device cluster, and the terminal device cluster may include terminal device 10a, terminal device 10b, ..., terminal device 10n, wherein any terminal device in the terminal device cluster may have a communication connection with the business server 100, for example, there is a communication connection between terminal device 10a and the business server 100, and there is a communication connection between terminal device 10b and the business server 100, wherein the above-mentioned communication connection does not limit the connection method, and may be directly or indirectly connected through wired communication, or directly or indirectly connected through wireless communication, or through other methods, and this application does not impose any restrictions here.

[0096] It should be understood that Figure 1 Each terminal device in the terminal cluster shown can be installed with an application client. When the application client runs in each terminal device, it can be respectively connected to the above Figure 1 The business servers 100 shown in the figure interact with each other so that the business server 100 can receive business data from each terminal device. Among them, the application client can be a game application, video editing application, social application, instant messaging application, live broadcast application, short video application, video application, music application, shopping application, novel application, payment application, browser, etc., which has image, video and other data information functions. Among them, the application client can be an independent client or an embedded sub-client integrated in a client (such as an instant messaging client, a social client, a video client, etc.), which is not limited here.

[0097] Specifically, for ease of understanding, taking terminal device 10a as an example, terminal device 10a can run an application client, which can be a game application. When the application client runs in terminal device 10a, terminal device 10a can display the screen image in the virtual scene space, and terminal device 10a sends a rendering request for the screen image to the business server 100. Terminal device 10a and business server 100 can also be integrated into one server, which is not limited in this embodiment of the present application.

[0098] It is understood that the virtual scene space can be a three-dimensional virtual scene that can include various virtual content and spatial relationship information within the virtual world. Virtual content can include terrain, houses, trees, people, etc. The screen image can be an image that requires lighting rendering. The virtual scene space can include virtual content displayed on the screen image as well as virtual content outside the screen image.

[0099] The service server 100 may obtain M screen probes (Screen Probe) set on the screen image based on the rendering request, and determine the surfels (surfels, also called surface finite elements) hit by the M screen probes in the virtual scene space as the surfels to be propagated.

[0100] Screen probes can be understood as individual three-dimensional sensing points that can sense the lighting information passing through them. Screen probes are generated at a certain density based on the geometry cache and placed in screen space. Screen probes can emit rays in all directions to collect lighting information from the surface cache they hit. During rendering, the lighting results can be calculated based on the weighted calculation of the screen probes based on the rendering point. A facet is a type of surface cache. It is a unit of surface information stored on the surface of an object with a certain area. It stores parameters such as the position, normal, and lighting information of the surface point within a certain range.

[0101] The service server 100 can obtain sampling rays emitted by the surface element to be propagated in the virtual scene space along P ray directions. The sampling rays can be rays projected in all directions by the surface element to be propagated in the virtual scene space to simulate light reflection or refraction. The sampling rays can have a starting point, a direction, and a maximum emission distance. There can be multiple sampling rays, and the sampling rays can hit multiple surface elements and multiple locations.

[0102] The business server 100 may determine the surface elements hit by the sampling rays of P light directions in the virtual scene space as surface elements to be collected. The surface elements hit by the sampling rays in the virtual scene space may be surface elements where the sampling rays intersect with the normals corresponding to the surface elements.

[0103] The service server 100 may determine the illumination information stored for the to-be-collected surface element as the indirect illumination information collected in the virtual scene space by sampling light rays in P light directions. The illumination information stored for the to-be-collected surface element may include at least one of illumination information at the location of the to-be-collected surface element or illumination information stored for surface elements subsequently hit by the to-be-collected surface element. The illumination information may include material information, albedo, illumination value, and other information.

[0104] It can be understood that the surface element to be collected can continue to emit sampling light along P light directions to collect the lighting information stored by other surface elements, and determine the lighting information at the position of the surface element to be collected and the lighting information stored by the surface elements that the surface element to be collected continues to hit as the lighting information stored by the surface element to be collected.

[0105] The service server 100 can generate the surface element mixed illumination information of the surface element to be propagated based on the indirect illumination information collected by the sampled light rays in the P light directions in the virtual scene space. The surface element mixed illumination information can be generated by weighted summation of the indirect illumination information.

[0106] The service server 100 performs a weighted calculation based on the surface element mixed illumination information and the historical illumination information stored for the surface element to be propagated to obtain probe mixed illumination information corresponding to each of the M screen probes. Based on the probe mixed illumination information, the service server 100 performs illumination rendering on the screen image to obtain a rendered image for the screen image. The rendered image is then transmitted to the terminal device 10a. The terminal device 10a can display the image content of the rendered image on the screen as a two-dimensional image.

[0107] Rendering is the process of projecting a 3D scene object model into a 2D digital image based on the specified environment, materials, lighting, and rendering parameters. Lighting rendering is the process of calculating and processing the lighting of each virtual content on the screen, which allows the pixels in the rendered image to have a lighting effect.

[0108] The embodiment of the present application determines the surface element hit by the screen probe in the virtual scene space as the surface element to be propagated, determines the surface element hit by the sampling light emitted by the surface element to be propagated along each light direction in the virtual scene space as the surface element to be collected, and determines the lighting information stored in the surface element to be collected as the indirect lighting information collected by the sampling light hit in the virtual scene space. The lighting information stored in the surface element to be collected includes at least one of the lighting information at the position of the surface element to be collected or the lighting information stored in the surface element hit by the surface element to be collected. Therefore, the surface elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0109] See Figure 2 , Figure 2 This is a data processing scenario provided by the embodiment of the present application. Figure 1 .

[0110] like Figure 2As shown, the terminal device 10a can run an application client, which can be a game application. When the application client runs in the terminal device 10a, the terminal device 10a can display the screen image S in the virtual scene space K, and the terminal device 10a can send a rendering request for the screen image S to the business server 100.

[0111] It is understood that the virtual scene space K can be a three-dimensional virtual scene that can include various virtual content and spatial relationship information within the virtual world. Virtual content can include terrain, houses, trees, people, etc. The screen image S can be a two-dimensional image projected from the camera's viewpoint within the virtual scene space K. The screen image S can be displayed on a screen. The virtual scene space K can include both virtual content presented within the screen image S and virtual content outside of the screen image S.

[0112] The screen image S may refer to the two-dimensional range presented by the area S in the virtual scene space. The lighting rendering result of the screen image S may change in the time frame rate. In the first rendering frame, the business server 100 may generate M screen probes at random positions on the screen image S based on the rendering request. The M screen probes may include screen probe T1 and screen probe T2. The business server 100 may emit probe rays along P ray directions in the virtual scene space K through the M screen probes.

[0113] Among them, screen probes can be understood as three-dimensional sensing points that can sense the lighting information passing through them. Screen probes can be generated at a certain density based on the geometric cache and placed in the screen space. Screen probes can emit rays in all directions to collect lighting information from the cached surface that is hit. When rendering, the lighting results can be calculated from the screen probes based on the rendering points. The face element can also be called a surface finite element. It is a type of surface cache. It is a unit that stores surface information with a certain area attached to the surface of an object. It stores parameters such as the position, normal, and lighting information of the point on the surface within a certain range. The sampling ray can be a ray that the face element projects in all directions in the virtual scene space to simulate light reflection or refraction. The probe ray can have a starting point, a light direction, and a maximum emission distance. There can be multiple probe rays, multiple face elements hit by the sampling ray, and multiple hit positions.

[0114] It can be understood that the position of each screen probe in the pixel image may shift as the rendering frame changes in time, that is, the relative position of each screen probe in the same rendering frame is the same, but the relative position in different rendering frames may be different. The offset method of the screen probe can be to set an initial position, generate a random position offset as the rendering frame changes in time, and determine the position of the screen probe in the current frame by the position of the previous frame and the position offset. The embodiment of the present application does not limit the offset of the screen probe.

[0115] For example, if the probe rays emitted by screen probe T1 along P light directions hit positions F1, F2, and F3, positions F1, F2, and F3 may return a first illumination value to screen probe T1. If the probe rays emitted by screen probe T2 along P light directions hit positions F4, F5, and F6, positions F4, F5, and F6 may return a second illumination value to screen probe T2. The first illumination value and the second illumination value may be preset values ​​or 0.

[0116] The service server 100 may record the position information corresponding to positions F1, F2, F3, F4, F5, and F6 in the virtual scene space K. Surf 1 is generated based on the position information of position F1, surfel 2 is generated based on the position information of position F2, surfel 3 is generated based on the position information of position F3, surfel 4 is generated based on the position information of position F4, surfel 5 is generated based on the position information of position F5, and surfel 6 is generated based on the position information of position F6.

[0117] It can be understood that in the first rendering frame, the probe rays emitted by screen probes T1 and T2 along P ray directions do not hit any existing facets in the virtual scene space K, and screen probes T1 and T2 do not collect illumination information for the existing facets. The service server 100 can perform illumination rendering on the screen image S using the first illumination value collected by screen probe T1 and the second illumination value collected by screen probe T2, obtain a rendered image S1 for the screen image S, and send the rendered image S1 to the terminal device 10a. The terminal device 10a can display the image content of the rendered image S1 on the screen in the form of a two-dimensional image as the first frame of the application image displayed by the application client.

[0118] Rendering is the process of projecting a 3D scene object model into a 2D digital image based on the specified environment, materials, lighting, and rendering parameters. Lighting rendering is the process of calculating and processing the lighting of each virtual content on the screen, which allows the pixels in the rendered image to have a lighting effect.

[0119] In the second rendering frame, the service server 100 can retrieve the M screen probes set on the screen image and determine the surfels hit by the M screen probes in the virtual scene space as surfels to be propagated. Surfels to be propagated are surfels required for the screen probes to collect surfel mixed illumination information. For example, surfels 1, 2, and 3 hit by screen probe T1, and surfels 4, 5, and F6 hit by screen probe T2 can be determined as surfels to be propagated.

[0120] The service server 100 can emit sampling rays along P ray directions in the virtual scene space K through bins 1, 2, 3, 4, 5, and 6. The sampling ray emitted by bin 1 hits bin 5 and position F7, the sampling ray emitted by bin 2 hits bin 4, the sampling ray emitted by bin 4 hits position F8, and the sampling ray emitted by bin 6 hits position F9. Bins 3 and 5 can also emit sampling rays to hit bins or positions, which will not be detailed in this embodiment of the present application.

[0121] The business server 100 can determine the direct lighting information of the face element through the lighting value of the virtual light source and the occlusion coefficient of the ray emitted from the virtual light source to the face element in the virtual scene space (which can be obtained from the shadow map in the virtual scene space). The direct lighting information collected by the face element can also be called the lighting information of the location of the face element. The embodiment of the present application does not limit the method of determining the direct lighting information of the face element.

[0122] It can be understood that the sampling light emitted by surface element 1 can determine the lighting information stored in surface element 5 and the third lighting value returned by position F7 as the indirect lighting information 1 collected by the sampling light emitted by surface element 1 in the virtual scene space. The service server 100 can generate surface element mixed lighting information 1 for surface element 1 based on the indirect lighting information 1. The sampling light emitted by surface element 2 can determine the lighting information stored in surface element 4 as the indirect lighting information 2 collected by the sampling light emitted by surface element 2 in the virtual scene space. The service server 100 can generate surface element mixed lighting information 2 for surface element 2 based on the indirect lighting information 2. The sampling light emitted by surface element 4 can determine the fourth lighting value returned by position F8 as the indirect lighting information 4 collected by the sampling light emitted by surface element 4 in the virtual scene space. The service server 100 can generate surface element mixed lighting information 4 for surface element 4 based on the indirect lighting information 4. The sampling light emitted by bin 6 can determine the fifth illumination value returned by position F9 as the indirect illumination information 6 collected by the sampling light emitted by bin 6 in the virtual scene space. The service server 100 can generate bin mixed illumination information 6 for bin 6 based on the indirect illumination information 6. The service server 100 can also generate bin mixed illumination information 3 corresponding to bin 3 and bin mixed illumination information 5 corresponding to bin 5 through the above process. This embodiment of the present application will not be described in detail here.

[0123] It can be understood that if the surface element collects direct lighting information of a virtual light source, then the direct lighting information can be used together with the indirect lighting information collected by the surface element to determine the lighting information stored by the surface element. For example, the indirect lighting value generated by the indirect lighting information in each direction can be added to the direct lighting value corresponding to the direct lighting information to obtain the lighting information stored by the surface element.

[0124] The service server 100 may record the position information corresponding to positions F7, F8, and F9 in the virtual scene space K. Surf 7 is generated based on the position information of position F7, Surf 8 is generated based on the position information of position F8, and Surf 9 is generated based on the position information of position F9.

[0125] In the second rendering frame, the screen probe may be offset from its initial position (the surface element to be propagated hit after the offset may be different, which is not limited in this embodiment of the present application). Taking the probe rays emitted by screen probe T1 along P light directions, hitting surface element 1, surface element 2, and surface element 3, and the probe rays emitted by screen probe T2 along P light directions, hitting surface element 4, surface element 5, and surface element 6 as an example, the business server 100 may generate probe mixed lighting information 1 of screen probe T1 based on surface element mixed lighting information 1, surface element mixed lighting information 2, surface element mixed lighting information 3, surface element 1, surface element 2, and surface element 3. The business server 100 may generate probe mixed lighting information 2 of screen probe T2 based on surface element mixed lighting information 4, surface element mixed lighting information 5, surface element mixed lighting information 6, surface element 4, surface element 5, and surface element 6.

[0126] The service server 100 can perform illumination rendering on the screen image S using the probe mixed illumination information 1 collected by the screen probe T1 and the probe mixed illumination information 2 collected by the screen probe T2, thereby obtaining a rendered image S2 for the screen image S, and then sending the rendered image S2 to the terminal device 10a. The terminal device 10a can display the content of the rendered image S2 on the screen in the form of a two-dimensional image as the second frame of the application image displayed by the application client.

[0127] It can be understood that in the third rendering frame, surface elements 1 to 9 can continue to emit sampling rays along P light directions. If the sampling rays hit a new position, a new surface element can be generated at the new position hit by the sampling rays. If a surface element exists at the position hit by the sampling rays, the surface element hit by the sampling rays can be determined as the surface element to be collected, and the lighting information stored in the surface element to be collected can be determined as the indirect lighting information collected by the sampling rays. Screen probes T1 and screen probes 2 can also continue to emit probe rays along P light directions. If the probe rays hit a new position, a new surface element can be generated at the new position hit by the probe rays. If a surface element exists at the position hit by the probe rays, the surface element hit by the probe rays can be determined as the surface element to be propagated, and the probe mixed lighting information collected by the probe rays can be determined based on the surface element mixed lighting information stored in the surface element to be propagated and the probe mixed lighting information collected by the probe rays. For example, in the third rendering frame, the sampling light emitted by the newly generated surface element 8 hits surface element 2. Since surface element 1 has been reflected multiple times in the third rendering frame, surface element 8 can directly collect the indirect lighting information that surface element 1 further collects from other surface elements. Therefore, surface element 8 can further collect richer and more complete lighting information.

[0128] For ease of understanding, the embodiments of the present application are illustrated using the first rendering frame, the second rendering frame, and the third rendering frame in the screen image as examples. It can be understood that after several rendering frames, the sampling light emitted by the surface element in the virtual scene space has been reflected multiple times, and the surface elements that can be generated for the screen image have become stable. The probe mixed lighting information that can be collected by the screen probe has also become stable, which can provide a stable and efficient rendering solution for the lighting rendering of the screen image.

[0129] The embodiment of the present application can be implemented based on the Unreal Engine. The lighting algorithm implemented in the embodiment of the present application can be turned off by commanding r.SurfelLighting in the console of the Unreal Engine, and the lighting algorithm implemented in the embodiment of the present application can be turned on by commanding r.SurfelLighting 1.

[0130] The embodiment of the present application determines the surface element hit by the screen probe in the virtual scene space as the surface element to be propagated, determines the surface element hit by the sampling light emitted by the surface element to be propagated along each light direction in the virtual scene space as the surface element to be collected, and determines the lighting information stored in the surface element to be collected as the indirect lighting information collected by the sampling light hit in the virtual scene space. The lighting information stored in the surface element to be collected includes at least one of the lighting information at the position of the surface element to be collected or the lighting information stored in the surface element hit by the surface element to be collected. Therefore, the surface elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0131] See Figure 3 , Figure 3 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 1 The data processing method can be executed by a computer device, which can be Figure 1 The business server 100 shown in FIG. 1 is a diagram illustrating a data processing method executed by a computer device. The data processing method may include at least the following steps S101 to S103:

[0132] Step S101: Acquire M screen probes set on the screen, determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated, and obtain sampling rays emitted by the surface elements to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented on the screen and virtual content outside the screen; M and P are positive integers;

[0133] Specifically, a computer device can display a screen image in a virtual scene space. A virtual scene space can be a three-dimensional virtual scene that can include various virtual content within the virtual world, as well as information such as spatial relationships. Virtual content can include terrain, houses, trees, people, and so on. A screen image can be a two-dimensional image projected from a camera's viewpoint within the virtual scene space. The screen image can be displayed on a screen and can include both virtual content presented within the screen image and virtual content outside the screen image.

[0134] The screen image can refer to the two-dimensional range presented by the area in the virtual scene space. The lighting rendering result of the screen image can change in the time frame rate. In the target rendering frame, the computer device can obtain the M screen probes set on the screen image, and determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated.

[0135] Screen probes can be understood as individual three-dimensional sensing points that can sense the lighting information passing through them. Screen probes are generated at a certain density based on the geometry cache and placed within the screen space. They emit rays in all directions to collect lighting information from the surface cache they hit. During rendering, the lighting results are calculated based on the weighted calculations from the screen probes based on the rendering point. A facet, also known as a surface finite element, is a type of surface cache. It is a unit of surface information stored on the surface of an object with a certain area. It stores parameters such as the position, normal, and lighting information of the surface point within a certain range.

[0136] It can be understood that the screen probe can be set or generated in the screen image starting from the first frame of the screen image. The screen probe can emit probe rays along P light directions in the virtual scene space. If the probe ray hits a new position, a new face element can be generated at the new position hit by the probe ray. If a face element exists at the position hit by the probe ray, the face element hit by the probe ray can be determined as the face element to be propagated.

[0137] The surface element to be propagated may emit sampling rays along P ray directions in the virtual scene space, and the computer device may obtain the sampling rays emitted by the surface element to be propagated along the P ray directions in the virtual scene space.

[0138] Among them, the sampling light can be a ray projected by the surface element in the virtual scene space to simulate the reflection or refraction of light. The sampling light can have a starting point, a light direction and a maximum emission distance. There can be multiple sampling rays, and there can be multiple surface elements hit by the sampling light, and there can also be multiple locations hit.

[0139] The computer device can determine the direct lighting information of the face element through the lighting value of the virtual light source and the occlusion coefficient of the virtual light source to the face element in the virtual scene space (which can be obtained from the shadow map in the virtual scene space). The direct lighting information collected by the face element can also be referred to as the lighting information of the location of the face element. The embodiment of the present application does not limit the method for determining the direct lighting information of the face element. It can be understood that if the face element collects direct lighting information of the virtual light source, then the direct lighting information can be used together with the indirect lighting information collected by the face element to determine the lighting information stored by the face element. For example, the indirect lighting value generated by the indirect lighting information in each direction can be added to the direct lighting value corresponding to the direct lighting information to obtain the lighting information stored by the face element.

[0140] Step S102: Surface elements hit by the sampling rays of the P light directions in the virtual scene space are determined as surface elements to be collected, and the illumination information stored in the surface elements to be collected is determined to be the indirect illumination information collected by the sampling rays of the P light directions in the virtual scene space; the illumination information stored in the surface elements to be collected includes at least one of illumination information at the location of the surface element to be collected and illumination information stored in surface elements subsequently hit by the surface element to be collected;

[0141] Specifically, the computer device can determine the position hit by the sampling light in the virtual scene space based on the sampling light and the world coordinates of the virtual content in the virtual scene space. The computer device can determine the surface elements to be propagated along P light directions and hit by the sampling light in the virtual scene space as the surface elements to be collected.

[0142] It can be understood that the computer device can divide the virtual scene space into a 64*64*32 grid centered on the camera. The virtual content in each grid has a world coordinate corresponding to the virtual scene space. The computer device can query the hierarchical bounding volume tree corresponding to the virtual content based on the world coordinates of the virtual content and the light information of the probe light or the sampling light (which may include the starting point, light direction and maximum emission distance). If the leaf node of the hierarchical bounding volume tree is queried, the virtual content determined by the hierarchical bounding volume that hits the leaf node is determined, thereby obtaining the position information of the position hit by the probe light or the sampling light. If the leaf node of the hierarchical bounding volume tree is not queried within the maximum emission distance of the probe light or the sampling light, it can be determined that the probe light or the sampling light did not hit the virtual content corresponding to the hierarchical bounding volume tree. By determining the position hit by the probe light and the sampling light in the virtual scene space through the hierarchical bounding volume tree, the light detection can be accelerated, and the time complexity can be reduced to the logarithmic level with respect to the number of objects, thereby improving the rendering efficiency of the lighting rendering.

[0143] In an embodiment of the present application, a hardware accelerator may be used to perform ray detection technology, calculate the sampling ray of the face element to be propagated and the world coordinates of the virtual content in the virtual scene space, thereby determining the primitive expression (for example, a triangle) hit by the sampling ray and the virtual content in the virtual scene space and the position information (world coordinates) of each face element. The hardware accelerator may include a dedicated space segmentation acceleration structure to accelerate the intersection of light and scene, a dedicated shader unit to process materials and lighting, and a high-speed cache mechanism to access large amounts of scene texture data.

[0144] The computer device may determine the illumination information stored for the to-be-collected surface element as the indirect illumination information collected in the virtual scene space by sampling light rays in P light directions. The illumination information stored for the to-be-collected surface element may include at least one of illumination information at the location of the to-be-collected surface element or illumination information stored for surfaces subsequently hit by the to-be-collected surface element. The illumination information may include material information, albedo, illumination value, and other information.

[0145] It can be understood that the surface element to be collected can continue to emit sampling light along P light directions to collect the lighting information stored by other surface elements, and determine the lighting information at the position of the surface element to be collected and the lighting information stored by the surface elements that the surface element to be collected continues to hit as the lighting information stored by the surface element to be collected.

[0146] For example, the sampling rays of the face element 1 to be propagated (face element 1) along P light directions hit face elements 2 and 3 in the virtual scene space, and the sampling rays of the face element 4 to be propagated (face element 4) along P light directions hit face elements 5 and 6 in the virtual scene space, then face elements 2 and face elements 3 can be determined as the face elements to be collected of face element 1.

[0147] The computer device can determine the lighting information stored in surface element 2 as the indirect lighting information 2 collected by the sampling light of surface element 1 to be propagated, and determine the lighting information stored in surface element 3 as the indirect lighting information 3 collected by the sampling light of surface element 1 to be propagated. The computer device can determine the lighting information stored in surface element 5 as the indirect lighting information 5 collected by the sampling light of surface element 4 to be propagated, and determine the lighting information stored in surface element 6 as the indirect lighting information 6 collected by the sampling light of surface element 4 to be propagated.

[0148] It can be understood that the lighting information stored by the facet 5 may not only include the lighting information of its own location, but also include the lighting information stored by the facets that the facet 5 continues to hit. After several rendering frames, the sampling light emitted by the facets in the virtual scene space has been reflected multiple times. The facets that can be generated for the screen image have become stable, and the probe mixed lighting information that can be collected by the screen probe has also become stable, which can provide a stable and efficient rendering solution for the lighting rendering of the screen image.

[0149] Optionally, the embodiment of the present application may also set a propagation threshold for the probe light emitted by each screen probe. When the number of reflections of the probe light recorded by the sampling light between the facets reaches the propagation threshold, the reflection of the probe light is stopped. The setting of the propagation threshold can be determined based on the contribution of the illumination value to the illumination rendering of the screen image. An appropriate propagation threshold can be selected so that even if the probe light continues to propagate, there is no obvious difference in the illumination rendering of the screen image, thereby reducing the computational cost of light detection and improving the rendering efficiency of illumination rendering.

[0150] Step S103, based on the indirect lighting information collected by the sampling light in P light directions in the virtual scene space, generate the surface element mixed lighting information of the surface element to be propagated, generate the probe mixed lighting information of M screen probes based on the surface element mixed lighting information and the surface element to be propagated, and perform lighting rendering on the screen image based on the probe mixed lighting information.

[0151] Specifically, the computer device can generate surface element mixed illumination information for the surface element to be propagated based on the indirect illumination information collected by sampling light rays in P light directions in the virtual scene space. The surface element mixed illumination information can be calculated by determining illumination weights for different light directions based on the sampling probability density (Sample Direction Probability Density Function) of the light directions of different sampling light rays, and performing a weighted summation of the indirect illumination information. The sampling probability density can be the probability that light is emitted along a certain direction on a given surface.

[0152] For example, the indirect illumination information collected by the surface element to be propagated 1 includes indirect illumination information 2 and indirect illumination information 3, and the indirect illumination information collected by the surface element to be propagated 4 includes indirect illumination information 5 and indirect illumination information 6. The computer device can determine the surface element mixed illumination information of the surface element to be propagated 1 based on the indirect illumination information 2, the indirect illumination information 3, and the direct illumination information of the surface element to be propagated 1, and determine the surface element mixed illumination information of the surface element to be propagated 4 based on the indirect illumination information 5, the indirect illumination information 6, and the direct illumination information of the surface element to be propagated 4. The surface element mixed illumination information of the surface element to be propagated 1 can be obtained by accumulating a first operation value (the operation result of multiplying the indirect illumination information 2 by the illumination weight of the light direction), a second operation value (the operation result of multiplying the indirect illumination information 3 by the illumination weight of the light direction), and the direct illumination value (direct illumination information) of the surface element to be propagated 1.

[0153] The computer device may generate probe mixed lighting information for M screen probes based on the surface element mixed lighting information and the surface element to be propagated. The probe mixed lighting information may be weighted summed by weighted summing of the surface element mixed lighting information of each surface element to be propagated and historical lighting information. The probe mixed lighting information is obtained by weighted summing the weighted lighting information of each surface element to be propagated using the lighting weights determined by the light directions of different probe rays. The historical lighting information may be determined by the surface element mixed lighting information of a frame previous to the target rendering frame.

[0154] The computer device performs illumination rendering on the screen image based on the probe mixed illumination information to obtain a rendered image. The computer device can display the image content of the rendered image on the screen in the form of a two-dimensional image.

[0155] Rendering is the process of projecting a 3D scene object model into a 2D digital image based on the specified environment, materials, lighting, and rendering parameters. Lighting rendering is the process of calculating and processing the lighting of each virtual content on the screen, which allows the pixels in the rendered image to have a lighting effect.

[0156] It can be understood that in the next rendering frame, the face element to be propagated and the newly generated face element can continue to emit sampling rays along P ray directions. If the sampling ray hits a new position, a new face element can be generated at the new position hit by the sampling ray. If a face element exists at the position hit by the sampling ray, the face element hit by the sampling ray can be determined as the face element to be collected, and the lighting information stored in the face element to be collected can be determined as the indirect lighting information collected by the sampling ray. The screen probe can also continue to emit probe rays along P ray directions. If the probe ray hits a new position, a new face element can be generated at the new position hit by the probe ray. If a face element exists at the position hit by the probe ray, the face element hit by the probe ray can be determined as the face element to be propagated, and the probe mixed lighting information collected by the probe ray is determined based on the face element mixed lighting information stored in the face element to be propagated and the probe mixed lighting information collected by the probe ray. For example, in the next rendering frame, the sampling light emitted by the newly generated surface element hits the already generated surface element. Since the already generated surface element has been reflected multiple times, the newly generated surface element can directly collect the indirect lighting information collected by the already generated surface element on other surface elements. Therefore, the newly generated surface element can further collect richer and more complete lighting information.

[0157] The embodiment of the present application determines the surface element hit by the screen probe in the virtual scene space as the surface element to be propagated, determines the surface element hit by the sampling light emitted by the surface element to be propagated along each light direction in the virtual scene space as the surface element to be collected, and determines the lighting information stored in the surface element to be collected as the indirect lighting information collected by the sampling light hit in the virtual scene space. The lighting information stored in the surface element to be collected includes at least one of the lighting information at the position of the surface element to be collected or the lighting information stored in the surface element hit by the surface element to be collected. Therefore, the surface elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0158] See Figure 4 , Figure 4 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 2 The data processing method can be executed by a computer device, which can be Figure 1The business server 100 shown in FIG. 1 is shown. The following description will be made by taking the data processing method executed by a computer device as an example. The data processing method may include at least the following steps S201 to S207:

[0159] In step S201, a screen probe is generated at a random position in each pixel image, and probe rays are emitted along P ray directions in the virtual scene space through M screen probes; a face element is generated at the position hit by the probe ray in the virtual scene space, and the generated face element is determined as the face element to be propagated.

[0160] Specifically, a computer device can display a screen image in a virtual scene space. The virtual scene space can be a three-dimensional virtual scene that can include various virtual content and spatial relationship information within the virtual world. Virtual content can include terrain, houses, trees, people, etc. The screen image can be a two-dimensional image projected from a camera's viewpoint within the virtual scene space. The screen image can be displayed on a screen. The virtual scene space can include both virtual content presented within the screen image and virtual content outside the screen image.

[0161] The screen image may refer to the two-dimensional range presented by the area in the virtual scene space. The screen image may be divided into M pixel images. The division method may be to determine every 16*16 screen pixels as a pixel image. The computer device may generate a screen probe at a random position in each pixel image.

[0162] Screen probes can be understood as individual three-dimensional sensing points that can sense the lighting information passing through them. Screen probes are generated at a certain density based on the geometry cache and placed within the screen space. They emit rays in all directions to collect lighting information from the surface cache they hit. During rendering, the lighting results are calculated based on the weighted calculations from the screen probes based on the rendering point. A facet, also known as a surface finite element, is a type of surface cache. It is a unit of surface information stored on the surface of an object with a certain area. It stores parameters such as the position, normal, and lighting information of the surface point within a certain range.

[0163] A computer device can emit probe rays along P light directions in a virtual scene space through M screen probes, wherein the probe rays can be rays projected by the screen probe in all directions in the virtual scene space to simulate light reflection or refraction. The probe rays can have a starting point, a light direction, and a maximum emission distance. There can be multiple probe rays, and there can be multiple face elements hit by the probe rays, and there can also be multiple hit locations.

[0164] The computer device can determine the primitive expression (for example, a triangle) hit by the probe light in the virtual scene space based on the world coordinates of the probe light and the virtual content in the virtual scene space. The computer device generates a face element at the position hit by the probe light in the virtual scene space, and determines the generated face element as the face element to be propagated. The face element to be propagated is also the face element required for the screen probe to collect face elements and mix the lighting information. The embodiment of the present application can generate new face elements through the probe light emitted by the screen probe. The generated face elements are not limited to the screen image, and off-screen face elements can be generated, which greatly improves the accuracy of face element collection of lighting information. There is no need to rely on the geometric cache of the screen image to generate face elements, which can improve the fault tolerance of face element generation and improve the feasibility and practicality of lighting rendering.

[0165] The computer device can store the newly generated face element, and the process can be: when it is detected that a new face element is generated in the virtual scene space, the newly generated face element is determined as the face element to be stored, and the position information of the face element to be stored in the virtual scene space is obtained; based on the position information of the face element to be stored, the face element to be stored is stored in the world grid corresponding to T world grids to obtain the stored face element; the stored face element is used as the face element to be propagated when it is hit by M screen probes in the virtual scene space.

[0166] Specifically, the computer device may divide the virtual scene space into T world grids, each of which is used to store S facets within the grid space. For example, the virtual scene space may be divided into a 64*64*32 grid centered on the camera, with each facet center point having corresponding world coordinates in the virtual scene space. The computer device may generate a hash index for the facet based on the world grid to which the facet belongs, the grid coordinates of the world grid, and the world coordinates of the facet. Each world grid stores facets within a certain spatial range, with each grid corresponding to a grid coordinate, and each facet corresponding to a hash index.

[0167] Please also refer to Figure 5 , Figure 5 This is a data processing scenario provided by the embodiment of the present application. Figure 2 ,like Figure 5 As shown, the virtual scene space can be divided into world grid 1, world grid 2, ..., world grid N. The world grid can be stored in a grid cache. Each position in the grid cache corresponds to a grid coordinate. The grid cache can record the offset position and offset of each world grid. Among them, the offset position can refer to the hash index of the face element stored in the world grid, that is, the face elements stored in the world grid are continuous in the face element cache, and the offset position can extend the first position (the first hash index) of this continuous space. The offset refers to the number of face elements stored in the world grid.

[0168] For example, for world grid 2, the stored bins include bin 1, bin 2, ..., bin N. Then the offset position of world grid 2 may be the hash index of bin 1, and the offset of world grid 2 may be N.

[0169] When a new surfel is detected in the virtual scene space, the computer device may determine the newly generated surfel as a surfel to be stored, obtain the location information of the surfel to be stored in the virtual scene space, determine the world grid corresponding to the surfel to be stored based on the world coordinates in the location information of the surfel to be stored, store the surfel to be stored in the corresponding world grid among T world grids, and generate a hash index of the surfel based on the grid coordinates of the stored world grid and the world coordinates of the surfel to obtain the stored surfel. The stored surfel is used as the surfel to be propagated when it is hit by M screen probes in the virtual scene space.

[0170] Step S202: Acquire M screen probes set on the screen, determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated, and obtain sampling rays emitted by the surface elements to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented on the screen and virtual content outside the screen; M and P are positive integers;

[0171] Specifically, the lighting rendering result of the screen image can change in the time frame rate. In the target rendering frame, the computer device can obtain the M screen probes set on the screen image, and determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated.

[0172] The surface element to be propagated may emit sampling rays along P ray directions in the virtual scene space, and the computer device may obtain the sampling rays emitted by the surface element to be propagated along the P ray directions in the virtual scene space.

[0173] Among them, the sampling light can be a ray projected by the surface element in the virtual scene space to simulate the reflection or refraction of light. The sampling light can have a starting point, a light direction and a maximum emission distance. There can be multiple sampling rays, and there can be multiple surface elements hit by the sampling light, and there can also be multiple locations hit.

[0174] The computer device can determine the direct lighting information of the face element through the lighting value of the virtual light source and the occlusion coefficient of the ray emitted from the virtual light source to the face element in the virtual scene space (which can be obtained from the shadow map in the virtual scene space). The direct lighting information collected by the face element can also be referred to as the lighting information of the location of the face element. The embodiment of the present application does not limit the method for determining the direct lighting information of the face element.

[0175] It can be understood that if the surface element collects direct lighting information of a virtual light source, then the direct lighting information can be used together with the indirect lighting information collected by the surface element to determine the lighting information stored by the surface element. For example, the indirect lighting value generated by the indirect lighting information in each direction can be added to the direct lighting value corresponding to the direct lighting information to obtain the lighting information stored by the surface element.

[0176] Step S203, based on the sampling light in the target light direction and the world coordinates of the virtual content in the virtual scene space, determine the position hit by the sampling light in the target light direction in the scene space; if there is a face element at the position hit by the sampling light in the target light direction in the scene space, then determine the face element hit by the sampling light in the target light direction as the face element to be collected; if there is no face element at the position hit by the sampling light in the target light direction in the scene space, obtain the position information of the position hit by the sampling light in the target light direction, generate a new face element in the virtual scene space based on the position information, and determine the new face element as the face element to be collected; the new face element is used to be determined as the new face element to be propagated in the next frame of the screen image.

[0177] Specifically, the computer device may determine the position hit by the sampling light in the direction of the light in the scene space based on the sampling light and the world coordinates of the virtual content in the virtual scene space.

[0178] Probe rays and sampling rays can be set with starting points, light directions, and maximum emission distances. Each virtual content in the virtual scene space can be represented by primitives, such as triangles. Each virtual content can have its own world coordinates and hierarchical bounding volumes. The computer device can construct a bounding volume hierarchy (BVH) tree based on the world coordinates and bounding boxes of the virtual content. In the hierarchical bounding volume tree, all virtual content will be enclosed in the leaf nodes of the hierarchical bounding volume tree. Each leaf node can represent a hierarchical bounding volume, and the hierarchical bounding volume can continue to be enclosed by a larger hierarchical bounding volume, recursively enclosed, and finally form the hierarchical bounding volume of the root node. The hierarchical bounding volume of the root node will enclose the entire scene. The computer device can also construct a separate hierarchical bounding volume tree for each virtual content.

[0179] The computer device can query in the hierarchical bounding volume tree based on the world coordinates of the virtual content and the light information of the probe ray or the sampling ray (which may include the starting point, the light direction and the maximum emission distance). If the leaf node of the hierarchical bounding volume tree is queried, the virtual content determined by the hierarchical bounding volume that hits the leaf node is determined, thereby obtaining the position information of the position hit by the probe ray or the sampling ray. If the leaf node of the hierarchical bounding volume tree is not queried within the maximum emission distance of the probe ray or the sampling ray, it can be determined that the probe ray or the sampling ray did not hit the virtual content corresponding to the hierarchical bounding volume tree. By determining the positions hit by the probe ray and the sampling ray in the virtual scene space through the hierarchical bounding volume tree, the ray detection can be accelerated and the time complexity can be reduced to the logarithmic level with respect to the number of objects, thereby improving the rendering efficiency of the lighting rendering. It can be understood that the embodiments of the present application can use a hardware accelerator to perform light detection technology, calculate the world coordinates of probe rays or sampling rays and virtual content in the virtual scene space, and thus determine the primitive expression (for example, a triangle) hit by the sampling rays and the virtual content in the virtual scene space and the position information (world coordinates) of each facet. The hardware accelerator can include a special space segmentation acceleration structure to accelerate the intersection of light and scene, a special shader unit to process materials and lighting, and a high-speed cache mechanism to access a large amount of scene texture data.

[0180] If a surface element exists at a position hit by the sampling light in the target light direction in the scene space, the surface element hit by the sampling light in the target light direction is determined as the surface element to be collected.

[0181] If no surface exists at the location hit by the sampling ray in the target light direction in the scene space, then position information of the location hit by the sampling ray in the target light direction is obtained. The position information may include the spatial normal, world coordinates, and material information of the virtual content at the location hit. Based on the position information, the computer device may generate a new surface in the virtual scene space and determine the new surface as the surface to be collected. The new surface is then determined as the new surface to be propagated in the next frame of the screen image.

[0182] Among them, the new surface element to be propagated is used to continue to emit sampling rays along P light directions in the next frame, and is used to generate surface element mixed lighting information of the new surface element to be propagated based on the indirect lighting information collected in the virtual scene space by the sampling rays of P light directions. It can be understood that the computer device can generate new surface elements through the position information of the positions hit by the sampling rays and probe rays, rather than relying on the geometric cache of the screen image to generate a fixed number of surface elements. The new surface elements will also continue to emit sampling rays to continue to generate surface elements, realizing unlimited light propagation, which is more in line with the actual situation of light propagation. The generated surface elements are not limited to the screen image, and off-screen surface elements can be generated, which greatly improves the accuracy of surface element collection of lighting information, improves the realism of screen image lighting rendering and user experience.

[0183] Step S204: determining the illumination information stored in the facet to be collected as the indirect illumination information collected by the sampling light rays in P light directions in the virtual scene space;

[0184] Specifically, the computer device may store relevant data of the surfels through a surfel cache, and the process may be: obtaining stored surfels respectively stored in T world grids, and storing position information, material information, and historical lighting information of the last used frame and the target rendering frame corresponding to the stored surfels in the surfel caches corresponding to the T world grids; the historical lighting information of the target rendering frame is determined based on the surfel mixed lighting information of the target rendering frame and the historical lighting information of the previous rendering frame, where the previous rendering frame is the previous frame of the target rendering frame;

[0185] Specifically, the computer device obtains the stored face elements stored in T world grids respectively, and stores the relevant data corresponding to the stored face elements in the face element cache corresponding to the T world grids. The storage unit corresponding to each hash index in the face element cache can store the relevant data of the face element corresponding to the hash index. The relevant data of the face element may include position information, material information, the last used frame, and the historical lighting information of the target rendering frame.

[0186] The last used frame is the frame in which the last probe ray emitted by the screen probe or the sampling ray emitted by the target surface collected lighting information. The historical lighting information of the target rendering frame is determined based on the target rendering frame's surface mixed lighting information and the historical lighting information of the previous rendering frame, where the previous rendering frame is the frame before the target rendering frame.

[0187] The computer device can obtain the lighting information stored in the facet to be collected. The process can be: based on the world grid and position information corresponding to the facet to be collected, obtain the lighting information stored in the facet to be collected in the facet cache, and determine the lighting information stored in the facet to be collected as the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0188] Specifically, the computer device may determine the grid coordinates and hash index of the facet to be collected based on the world grid and position information corresponding to the facet to be collected. Using the grid coordinates and hash index of the facet to be collected, the computer device may obtain the lighting information stored in the facet cache for the facet to be collected. The computer device may determine the lighting information stored for the facet to be collected as the indirect lighting information collected in the virtual scene space by sampling light rays in P light directions. The lighting information stored for the facet to be collected may include at least one of the lighting information at the location of the facet to be collected and the lighting information stored for facets subsequently hit by the facet to be collected. The lighting information may include material information, albedo, lighting value, and other information.

[0189] It can be understood that the surface element to be collected can continue to emit sampling light along P light directions to collect the lighting information stored by other surface elements, and determine the lighting information at the position of the surface element to be collected and the lighting information stored by the surface elements that the surface element to be collected continues to hit as the lighting information stored by the surface element to be collected.

[0190] After several rendering frames, the sampling light emitted by the surface elements in the virtual scene space has been reflected multiple times. The surface elements that can be generated for the screen image have become stable, and the probe mixed lighting information that can be collected by the screen probe has also become stable, which can provide a stable and efficient rendering solution for the lighting rendering of the screen image.

[0191] Optionally, the computer device can recycle facets that have not been used for a long time. The process may be: facets whose interval number of frames between the last used frame and the target rendering frame in the facet cache is greater than or equal to the recycling threshold are determined as facets to be recycled; the last used frame refers to the picture frame in which the illumination information of the facet is collected for the last time by the probe light emitted by the screen probe or the sampling light emitted by the facet to be propagated; the position information of the facet to be recycled in the virtual scene space is obtained, and based on the position information, the facet to be recycled is recycled in the post-rendering frame; the post-rendering frame is the next frame of the target rendering frame.

[0192] Specifically, the computer device may determine, as a facet to be recycled, a facet whose interval number of frames between the last used frame and the target rendering frame in the facet cache is greater than or equal to a recycling threshold. The computer device may obtain the position information of the facet to be recycled in the virtual scene space, and based on the position information, recycle the facet to be recycled in the next frame of the target rendering frame. The recycling process may be to set the radius of the facet to be recycled to 0, and recycle the hash index corresponding to the facet so that it can be used when a new facet is subsequently generated, thereby avoiding repeated generation of hash indexes, reducing the amount of calculation, avoiding occupying the facet cache, and improving the rendering efficiency of the lighting rendering of the screen image in the virtual scene space.

[0193] Step S205: Obtain the illumination weights corresponding to the P light directions respectively, and determine the illumination values ​​corresponding to the P light directions respectively based on the indirect illumination information collected by the sampled light of the P light directions in the virtual scene space; determine the surface element mixed illumination information of the surface element to be propagated based on the P illumination values ​​and the P illumination weights.

[0194] Specifically, the sampling light rays emitted by the surface element to be propagated in P light directions have different specific illumination weights. The computer device can obtain the illumination weights corresponding to the P light directions. The specific process of determining the weights for different light directions can be: obtaining the normal direction of the position hit by the target light direction in the virtual scene space, and determining the first parameter value of the target light direction based on the normal direction and the target light direction; determining the second parameter value of the target light direction based on the number P of light directions and the sampling probability density corresponding to each light direction, and generating the illumination weight corresponding to the target light direction through the first parameter value and the second parameter value; the sampling probability density is the probability that the surface element to be propagated is emitted along the P light directions.

[0195] Specifically, for ease of understanding, the target light direction among P light directions is taken as an example for explanation. The computer device can obtain the normal direction of the spatial normal of the position hit by the target light direction in the virtual scene space, and determine the first parameter value of the target light direction based on the normal direction and the target light direction. The computer device can determine the second parameter value of the target light direction based on the number of light directions P and the sampling probability density corresponding to each light direction, and generate the illumination weight corresponding to the target light direction through the first parameter value and the second parameter value. The calculation method of the illumination weight can refer to the formula (1):

[0196]

[0197] Where weight is the illumination weight corresponding to the target light direction, Normal is the direction of the space normal hit by the target light direction, RayDirection is the target light direction, P is the number of light directions, and SampleDirectionPDF is the sampling probability density. The sampling probability density refers to the probability that a light ray will be emitted along a certain direction on a given surface. In ray detection, light rays originate from the viewpoint and propagate along different directions, interacting with virtual content in the virtual scene space. When a light ray intersects a surface of virtual content, the direction of the light reflection needs to be determined. This direction can be determined by calculating the sampling probability density.

[0198] The computer device can determine the illumination values ​​corresponding to the P light directions based on the indirect illumination information collected by the sampled light rays in the virtual scene space in the P light directions, and determine the surface element mixed illumination information of the surface element to be propagated based on the P illumination values ​​and the P illumination weights. For example, the indirect illumination information collected by the surface element 1 to be propagated includes indirect illumination information 2 (the illumination weight of the indirect illumination information 2 is 10%) and indirect illumination information 3 (the illumination weight of the indirect illumination information 2 is 15%). The computer device can determine the sum of 10% of the illumination value corresponding to the indirect illumination information 2, 15% of the illumination value corresponding to the indirect illumination information 3, and the direct illumination value of the surface element to be propagated as the surface element mixed illumination information of the surface element to be propagated.

[0199] In step S206, the surface element to be propagated, which is hit by the probe light emitted by the target screen probe along the target light direction in the virtual scene space, is determined as the surface element to be selected, and the historical illumination information of the surface element to be selected is obtained. The mixed sampling illumination information of the surface element to be selected in the target light direction is determined based on the surface element mixed illumination information and the historical illumination information of the surface element to be selected; based on the mixed sampling illumination information collected by the probe light in P light directions in the virtual scene space, the probe mixed illumination information of the target screen probe is generated.

[0200] Specifically, the M screen probes include a target screen probe, and the P light directions include a target light direction. For ease of understanding, the target screen probe among the M screen probes is used as an example for explanation. The computer device can determine the surface element to be propagated that is hit by the probe light emitted by the target screen probe along the target light direction in the virtual scene space as the surface element to be selected. The computer device can obtain historical lighting information of the surface element to be selected, and determine the mixed sampling lighting information of the surface element to be selected in the target light direction based on the surface element mixed lighting information and historical lighting information of the surface element to be selected. For example, the computer device can assign a 30% lighting weight to the historical lighting information of the previous frame, and then the computer device can determine the sum of 30% of the lighting value corresponding to the historical lighting information and 70% of the lighting value corresponding to the surface element mixed lighting information as the mixed sampling lighting information of the surface element to be selected in the target light direction.

[0201] It can be understood that in each frame of the screen image, the surface element to be propagated will emit sampling rays in all directions, collect the lighting information stored in the surface elements hit by the sampling rays in all directions, and calculate the lighting weight of each direction based on the direction of the light. After the lighting value in each direction is multiplied and accumulated by the lighting weight, it will be weighted mixed with the historical lighting information stored in the surface element to be propagated in the previous frame and stored as the historical lighting information of the next frame. It is also the lighting information collected by the sampling light or probe light that is returned to the surface element that detects the next frame. By weighted mixing with the historical lighting information, the lighting information stored in the surface element can be denoised in time, thereby making the indirect lighting information stored in the surface element more stable and improving the practicality of lighting rendering.

[0202] The probe rays emitted by the target screen probe in P light directions have different specific illumination weights. The method for determining the illumination weights of the probe rays can refer to the method for determining the illumination weights of the sampling rays in step S205, and this embodiment of the present application does not limit this.

[0203] The computer device can generate probe mixed lighting information of the target screen probe by weighted summing the mixed sampled lighting information collected by probe lights of P light directions in the virtual scene space based on the lighting weights corresponding to each lighting direction.

[0204] Step S207, obtaining Q neighborhood pixel images of the target pixel image, and determining the screen probes in the Q neighborhood pixel images as neighborhood screen probes; Q is a positive integer; based on the pixel distances between the Q neighborhood screen probes and the target pixel image and the pixel depths corresponding to the Q neighborhood screen probes, generating probe weights corresponding to the Q neighborhood screen probes; the pixel depth is the pixel distance between the neighborhood screen probe and the camera position corresponding to the screen image; based on the probe mixed lighting information corresponding to the Q neighborhood screen probes and the probe weights, generating the neighborhood mixed lighting information of the target screen probe, and based on the probe mixed lighting information of the target screen probe and the neighborhood mixed lighting information, performing lighting rendering on the target pixel image in the screen image.

[0205] Specifically, the screen image can be divided into M pixel images, each pixel image is provided with a screen probe, please refer to Figure 6 , Figure 6 This is a data processing scenario provided by the embodiment of the present application. Figure 3 ,like Figure 6 As shown, the screen S can be divided into pixel screen H1, pixel screen H2, pixel screen H3, pixel screen H4, pixel screen H5, pixel screen H6, pixel screen H7, pixel screen H8 and pixel screen H9. Among them, pixel screen H1 is provided with a screen probe T1, pixel screen H2 is provided with a screen probe T2, pixel screen H3 is provided with a screen probe T3, pixel screen H4 is provided with a screen probe T4, pixel screen H5 is provided with a screen probe T5, pixel screen H6 is provided with a screen probe T6, pixel screen H7 is provided with a screen probe T7, pixel screen H8 is provided with a screen probe T8, and pixel screen H9 is provided with a screen probe T9.

[0206] It can be understood that the position of each screen probe in the pixel image may shift as the rendering frame time changes, that is, the relative position of each screen probe in the same rendering frame is the same, but the relative position in different rendering frames may be different. The offset method of the screen probe can be to set an initial position, generate a random position offset as the rendering frame time changes, and determine the position of the screen probe in the current frame by the position of the previous frame and the position offset.

[0207] For ease of understanding, a computer device may be rendering unit pixel A in pixel picture H1. Pixel picture H1 may also be referred to as a target pixel picture. The computer device may obtain Q neighboring pixel pictures of unit pixel A. A neighboring pixel picture may be a pixel picture adjacent to the target pixel picture. For example, if Q may be 3, the neighboring pixel pictures of pixel picture H1 may include neighboring pixel pictures H3, H5, and H9.

[0208] The computer device can determine screen probe T3 as neighboring screen probe T3, screen probe T5 as neighboring screen probe T5, and screen probe T9 as neighboring screen probe T9. The computer device can generate probe weights corresponding to the neighboring screen probes based on the pixel distance between the neighboring screen probes and the target pixel image and the pixel depth corresponding to the neighboring screen probes. The pixel depth is the pixel distance between the neighboring screen probes and the camera position corresponding to the screen image. The probe weight can be obtained by interpolation, and the embodiment of the present application does not limit the method for determining the probe weight.

[0209] The computer device may generate neighborhood mixed lighting information of the target screen probe based on the probe mixed lighting information and probe weights corresponding to the neighboring screen probes. The neighborhood mixed lighting information may be obtained by weighted summing the probe weights of each neighboring screen probe and the lighting values ​​corresponding to the probe mixed lighting information. The computer device may perform lighting rendering on unit pixel A in the target pixel image based on the probe mixed lighting information of the target screen probe and the neighborhood mixed lighting information, thereby performing lighting rendering on all unit pixels in the screen image to obtain a rendered image. The computer device may display the image content of the rendered image on the screen in the form of a two-dimensional image.

[0210] Rendering is the process of projecting a 3D scene object model into a 2D digital image based on the specified environment, materials, lighting, and rendering parameters. Lighting rendering is the process of calculating and processing the lighting of each virtual content on the screen, which allows the pixels in the rendered image to have a lighting effect.

[0211] It can be understood that in the next rendering frame, the face element to be propagated and the newly generated face element can continue to emit sampling rays along P ray directions. If the sampling ray hits a new position, a new face element can be generated at the new position hit by the sampling ray. If a face element exists at the position hit by the sampling ray, the face element hit by the sampling ray can be determined as the face element to be collected, and the lighting information stored in the face element to be collected can be determined as the indirect lighting information collected by the sampling ray. The screen probe can also continue to emit probe rays along P ray directions. If the probe ray hits a new position, a new face element can be generated at the new position hit by the probe ray. If a face element exists at the position hit by the probe ray, the face element hit by the probe ray can be determined as the face element to be propagated, and the probe mixed lighting information collected by the probe ray is determined based on the face element mixed lighting information stored in the face element to be propagated and the probe mixed lighting information collected by the probe ray. For example, in the next rendering frame, the sampling light emitted by the newly generated surface element hits the already generated surface element. Since the already generated surface element has been reflected multiple times, the newly generated surface element can directly collect the indirect lighting information collected by the already generated surface element on other surface elements. Therefore, the newly generated surface element can further collect richer and more complete lighting information.

[0212] The embodiment of the present application can generate new face elements through the probe light emitted by the screen probe. The generated face elements are not limited to the screen image, and off-screen face elements can be generated, which greatly improves the accuracy of the face element in collecting lighting information. There is no need to rely on the geometric cache of the screen image to generate face elements, which can improve the fault tolerance of face element generation and improve the feasibility and practicality of lighting rendering. The computer device can determine the face elements hit by the screen probe in the virtual scene space as the face elements to be propagated, and the face elements hit by the sampling light emitted by the face elements to be propagated in the virtual scene space along various light directions as the face elements to be collected, and the lighting information stored in the face elements to be collected as the indirect lighting information collected by the hit sampling light in the virtual scene space. The lighting information stored in the face elements to be collected includes at least one of the lighting information at the location of the face element to be collected or the lighting information stored by the face elements continued to be hit by the face element to be collected. Therefore, the face elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0213] On the other hand, the computer can generate new facets based on the positional information of the locations hit by sampling rays and probe rays, rather than relying on the screen's geometric cache to generate a fixed number of facets. New facets will continue to emit sampling rays to generate facets, achieving infinite light propagation, which is more consistent with the actual light propagation. Facets and their corresponding hash indexes that have not been used for a long time can also be recycled for subsequent use when generating new facets, avoiding the repeated generation of hash indexes, reducing the amount of computation, and avoiding occupying the facet cache, thereby improving the rendering efficiency of the screen lighting in the virtual scene space.

[0214] See Figure 7 , Figure 7 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 3 The data processing method can be executed by a computer device, which can be Figure 1 The service server 100 is shown.

[0215] Specifically, the computer device can display a screen image in a virtual scene space. The lighting rendering result of the screen image can change at a temporal frame rate. In each frame of the screen image, the computer device executes the surfel update portion, the surfel to be propagated portion, and the screen probe portion.

[0216] For the surface elements to be propagated, the computer device may determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated. In each frame of the screen image, the surface elements to be propagated determined by the computer device may be different.

[0217] It can be understood that the screen probe can be set or generated in the screen image starting from the first frame of the screen image. The screen probe can emit probe rays along P light directions in the virtual scene space. If the probe ray hits a new position, a new face element can be generated at the new position hit by the probe ray. If a face element exists at the position hit by the probe ray, the face element hit by the probe ray can be determined as the face element to be propagated, and the probe mixed lighting information collected by the probe ray is determined based on the face element mixed lighting information stored in the face element to be propagated and the face element to be propagated.

[0218] The surface element to be propagated can emit sampling rays along P ray directions in the virtual scene space. The sampling rays collect indirect lighting information of other surfaces. The computer device can obtain the sampling rays emitted by the surface element to be propagated along the P ray directions in the virtual scene space. The computer device can determine the location where the sampling rays intersect the spatial normal in the virtual scene space as the location in the scene space hit by the sampling rays in that ray direction.

[0219] The computer device can determine the direct lighting information of the face element through the lighting value of the virtual light source and the occlusion coefficient of the virtual light source to the face element in the virtual scene space (which can be obtained from the shadow map in the virtual scene space). The direct lighting information collected by the face element can also be referred to as the lighting information of the location of the face element. The embodiment of the present application does not limit the method for determining the direct lighting information of the face element. It can be understood that if the face element collects direct lighting information of the virtual light source, then the direct lighting information can be used together with the indirect lighting information collected by the face element to determine the lighting information stored by the face element. For example, the indirect lighting value generated by the indirect lighting information in each direction can be added to the direct lighting value corresponding to the direct lighting information to obtain the lighting information stored by the face element.

[0220] If the sampling ray hits a face element, the computer device can determine the face element to be propagated along the P light directions and hit the face element in the virtual scene space as the face element to be collected. It can be understood that the face element to be collected can continue to emit sampling rays along the P light directions to collect the lighting information stored by other face elements, and determine the lighting information at the position of the face element to be collected and the lighting information stored by the face element to be collected that continues to hit as the lighting information stored by the face element to be collected. If the sampling ray does not hit a face element (i.e., it hits a new position), the computer device can record the position information hit by the sampling ray, and the computer device can generate a new face element in the virtual scene space based on the position information, and determine the new face element as the face element to be collected. The new face element is used to be determined as a new face element to be propagated in the next frame of the screen image. The new to-be-propagated surface element is used to continue emitting sampling rays along P ray directions in the next frame, and is used to generate surface element mixed lighting information for the new to-be-propagated surface element based on the indirect lighting information collected in the virtual scene space by the sampling rays from the P ray directions. For example, in the next rendering frame, the sampling rays emitted by the newly generated surface element hit the already generated surface element. Since the already generated surface element has been reflected multiple times, the newly generated surface element can directly collect the indirect lighting information further collected by the already generated surface element on other surfaces, thereby further collecting richer and more complete lighting information for the newly generated surface element.

[0221] For the surface source to be propagated, the computer device can determine the illumination values ​​corresponding to the P light directions based on the indirect illumination information collected by the sampled light in the virtual scene space in the P light directions, and determine the surface element mixed illumination information of the surface element to be propagated based on the P illumination values ​​and the P illumination weights. For example, the indirect illumination information collected by the surface element 1 to be propagated includes indirect illumination information 2 (the illumination weight of the indirect illumination information 2 is 10%) and indirect illumination information 3 (the illumination weight of the indirect illumination information 2 is 15%). The computer device can determine the sum of 10% of the illumination value corresponding to the indirect illumination information 2, 15% of the illumination value corresponding to the indirect illumination information 3, and the direct illumination value of the surface element to be propagated as the surface element mixed illumination information of the surface element to be propagated.

[0222] For the screen probe part, the screen probe may also continue to emit probe rays along the P light directions, and collect the surface element mixed lighting information of the hit surface elements through the probe rays.

[0223] If the probe ray misses a facet (ie, hits a new position), the computer device may record the position information where the probe ray hits, and generate a new facet at the new position where the probe ray hits based on the position information.

[0224] If the probe ray hits a surface element, the computer device can determine the surface element hit by the probe ray as the surface element to be propagated, and determine the probe mixed lighting information collected by the probe ray based on the surface element mixed lighting information stored in the surface element to be propagated and the surface element to be propagated, thereby mixing the surface element mixed lighting information in various directions for each screen probe.

[0225] The computer device can perform lighting rendering on the pixel interpolation in the screen image according to the probe mixed lighting information of the screen probe to obtain a rendered image. The computer device can display the image content of the rendered image in the form of a two-dimensional image on the screen.

[0226] For the facet update part, when the indirect lighting information collected by each facet in each direction lock changes, the updated lighting information can be stored in the facet cache through the hash index of the facet for facet update, and the cache information related to the facet is updated.

[0227] The computer device may identify bins in the bin cache for which the number of frames between the last used frame and the target rendering frame is greater than or equal to a recycling threshold as bins to be recycled (i.e., mark the buffer as to be recycled). The computer device may obtain position information of the bins to be recycled in the virtual scene space and, based on the position information, recycle the bins to be recycled in a frame following the target rendering frame.

[0228] The embodiment of the present application can generate new face elements through the probe light emitted by the screen probe. The generated face elements are not limited to the screen image, and off-screen face elements can be generated, which greatly improves the accuracy of the face element in collecting lighting information. There is no need to rely on the geometric cache of the screen image to generate face elements, which can improve the fault tolerance of face element generation and improve the feasibility and practicality of lighting rendering. The computer device can determine the face elements hit by the screen probe in the virtual scene space as the face elements to be propagated, and the face elements hit by the sampling light emitted by the face elements to be propagated in the virtual scene space along various light directions as the face elements to be collected, and the lighting information stored in the face elements to be collected as the indirect lighting information collected by the hit sampling light in the virtual scene space. The lighting information stored in the face elements to be collected includes at least one of the lighting information at the location of the face element to be collected or the lighting information stored by the face elements continued to be hit by the face element to be collected. Therefore, the face elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0229] On the other hand, the computer can generate new facets based on the positional information of the locations hit by sampling rays and probe rays, rather than relying on the screen's geometric cache to generate a fixed number of facets. New facets will continue to emit sampling rays to generate facets, achieving infinite light propagation, which is more consistent with the actual light propagation. Facets and their corresponding hash indexes that have not been used for a long time can also be recycled for subsequent use when generating new facets, avoiding the repeated generation of hash indexes, reducing the amount of computation, and avoiding occupying the facet cache, thereby improving the rendering efficiency of the screen lighting in the virtual scene space.

[0230] See Figure 8 , Figure 8 This is a structural diagram of a data processing device provided in an embodiment of the present application. Figure 8 As shown, the data processing device 1 includes a surface element determination module 810 , an illumination collection module 820 and an illumination processing module 830 .

[0231] Surfactant determination module 810 is configured to obtain M screen probes set on the screen, determine the surfels hit by the M screen probes in the virtual scene space as surfels to be propagated, and obtain sampling rays emitted by the surfels to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented on the screen and virtual content outside the screen; M and P are positive integers;

[0232] The illumination collection module 820 is configured to determine the surface elements hit by the sampling rays in the P light directions in the virtual scene space as surface elements to be collected, and to determine the illumination information stored in the surface elements to be collected as the indirect illumination information collected by the sampling rays in the P light directions in the virtual scene space;

[0233] The lighting processing module 830 is used to generate the surface element mixed lighting information of the surface element to be propagated based on the indirect lighting information collected by the sampling light in P light directions in the virtual scene space, generate the probe mixed lighting information of M screen probes based on the surface element mixed lighting information and the surface element to be propagated, and perform lighting rendering on the screen image based on the probe mixed lighting information.

[0234] In one possible implementation, the P light directions include a target light direction; and the illumination collection module 820 is configured to determine, when the sampled light rays of the P light directions hit the surface element in the virtual scene space as the surface element to be collected, specifically perform the following operations:

[0235] Determine a position in the scene space hit by the sampling light in the target light direction based on the sampling light in the target light direction and the world coordinates of the virtual content in the virtual scene space;

[0236] If a surface element exists at the position hit by the sampling light in the target light direction in the scene space, the surface element hit by the sampling light in the target light direction is determined as the surface element to be collected;

[0237] If there is no face element at the position hit by the sampling light in the target light direction in the scene space, then the position information of the position hit by the sampling light in the target light direction is obtained, and based on the position information, a new face element is generated in the virtual scene space, and the new face element is determined as the face element to be collected; the new face element is used to be determined as the new face element to be propagated in the next frame of the screen image.

[0238] Among them, the new surface element to be propagated is used to continue to emit sampling light along P light directions in the next frame, and is used to generate the surface element mixed lighting information of the new surface element to be propagated through the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0239] In one possible implementation, the illumination processing module 830 is configured to generate the surface element mixed illumination information of the surface element to be propagated based on the indirect illumination information collected by the sampling light rays in the P light directions in the virtual scene space, and specifically to perform the following operations:

[0240] Obtain the illumination weights corresponding to the P light directions, and determine the illumination values ​​corresponding to the P light directions based on the indirect illumination information collected by the sampled light of the P light directions in the virtual scene space;

[0241] The surface element mixed illumination information of the surface element to be propagated is determined based on the P illumination values ​​and the P illumination weights.

[0242] In one possible implementation, the P light directions include a target light direction; and when the illumination processing module 830 is configured to obtain illumination weights corresponding to the P light directions, the illumination processing module 830 is configured to perform the following operations:

[0243] Obtaining a normal direction of a position hit by the target light direction in the virtual scene space, and determining a first parameter value of the target light direction based on the normal direction and the target light direction;

[0244] Based on the number P of light directions and the sampling probability density corresponding to each light direction, the second parameter value of the target light direction is determined, and the illumination weight corresponding to the target light direction is generated through the first parameter value and the second parameter value; the sampling probability density is the probability that the surface element to be propagated is emitted along P light directions.

[0245] In one possible implementation, the M screen probes include a target screen probe, and the P light directions include a target light direction. When the illumination processing module 830 is configured to generate probe mixed illumination information of the M screen probes based on the bin mixed illumination information and the bin to be propagated, the illumination processing module 830 is specifically configured to perform the following operations:

[0246] Determine the surface element to be propagated that is hit by the probe light emitted by the target screen probe in the virtual scene space along the target light direction as the surface element to be selected, obtain historical illumination information of the surface element to be selected, and determine the mixed sampling illumination information of the surface element to be selected in the target light direction based on the surface element mixed illumination information and the historical illumination information of the surface element to be selected;

[0247] Based on the mixed sampling illumination information collected by the probe rays of P light directions in the virtual scene space, the probe mixed illumination information of the target screen probe is generated.

[0248] In one possible implementation, the screen is divided into M pixel screens, each pixel screen is provided with a screen probe, the M pixel screens include a target pixel screen, and the M screen probes include a target screen probe; the lighting processing module 830 is configured to perform lighting rendering on the screen based on the probe mixed lighting information, specifically performing the following operations:

[0249] Acquire Q neighboring pixel images of the target pixel image, and determine the screen probes in the Q neighboring pixel images as neighboring screen probes; Q is a positive integer;

[0250] Generate probe weights corresponding to the Q neighboring screen probes based on the pixel distances between the Q neighboring screen probes and the target pixel image and the pixel depths corresponding to the Q neighboring screen probes; the pixel depth is the pixel distance between the neighboring screen probe and the camera position corresponding to the screen image;

[0251] Based on the probe mixed lighting information and probe weights corresponding to the Q neighboring screen probes, the neighboring mixed lighting information of the target screen probe is generated. Based on the probe mixed lighting information and the neighboring mixed lighting information of the target screen probe, the target pixel image in the screen image is illuminated and rendered.

[0252] In a possible implementation, the screen image is presented by a target rendering frame, and the bin stores the last used frame of the bin via a bin cache. The lighting processing module 830 is further configured to perform the following operations:

[0253] A bin whose interval between the last used frame and the target rendering frame in the bin cache is greater than or equal to the recycling threshold is determined as a bin to be recycled; the last used frame refers to the picture frame in which the probe ray emitted by the screen probe or the sampling ray emitted by the bin to be propagated collects illumination information for the last time;

[0254] The position information of the surface element to be recycled in the virtual scene space is obtained, and based on the position information, the surface element to be recycled is recycled in a post-rendering frame; the post-rendering frame is the next frame of the target rendering frame.

[0255] In one possible implementation, the virtual scene space is divided into T world grids, each world grid is used to store S surfels within the grid space; T and S are positive integers; and the lighting processing module 830 is further used to perform the following operations:

[0256] When a new facet is detected to be generated in the virtual scene space, the newly generated facet is determined as a facet to be stored, and position information of the facet to be stored in the virtual scene space is obtained;

[0257] Based on the position information of the face element to be stored, the face element to be stored is stored in the world grid corresponding to T world grids to obtain the stored face element; the stored face element is used as the face element to be propagated when it is hit by M screen probes in the virtual scene space.

[0258] In a possible implementation, the illumination processing module 830 is further configured to perform the following operations:

[0259] Obtain the stored surfels stored in each of the T world grids, and store the location information, material information, and historical lighting information of the last used frame and the target rendering frame corresponding to the stored surfels in the surfel caches corresponding to the T world grids; the historical lighting information of the target rendering frame is determined based on the surfel mixed lighting information of the target rendering frame and the historical lighting information of the previous rendering frame, where the previous rendering frame is the frame previous to the target rendering frame;

[0260] The illumination processing module 830 is used to determine the illumination information stored in the facet to be collected as the indirect illumination information collected by the sampling light rays of P light directions in the virtual scene space, and specifically to perform the following operations:

[0261] Based on the world grid and position information corresponding to the facet to be collected, the lighting information stored in the facet to be collected is obtained in the facet cache, and the lighting information stored in the facet to be collected is determined as the indirect lighting information collected by the sampling light in P light directions in the virtual scene space.

[0262] In one possible implementation, the screen is divided into M pixel screens. The surfel determination module 810 is configured to obtain M screen probes set on the screen and determine the surfels hit by the M screen probes in the virtual scene space as the surfels to be propagated. Specifically, the module performs the following operations:

[0263] Generate screen probes at random positions in each pixel image, and emit probe rays along P ray directions in the virtual scene space through M screen probes;

[0264] At the position hit by the probe light in the virtual scene space, a surface element is generated, and the generated surface element is determined as the surface element to be propagated.

[0265] The embodiment of the present application can generate new face elements through the probe light emitted by the screen probe. The generated face elements are not limited to the screen image, and off-screen face elements can be generated, which greatly improves the accuracy of the face element in collecting lighting information. There is no need to rely on the geometric cache of the screen image to generate face elements, which can improve the fault tolerance of face element generation and improve the feasibility and practicality of lighting rendering. The computer device can determine the face elements hit by the screen probe in the virtual scene space as the face elements to be propagated, and the face elements hit by the sampling light emitted by the face elements to be propagated in the virtual scene space along various light directions as the face elements to be collected, and the lighting information stored in the face elements to be collected as the indirect lighting information collected by the hit sampling light in the virtual scene space. The lighting information stored in the face elements to be collected includes at least one of the lighting information at the location of the face element to be collected or the lighting information stored by the face elements continued to be hit by the face element to be collected. Therefore, the face elements in the virtual scene space can be collected multiple times. The indirect lighting information generated by reflection avoids the lack of realism in lighting effects caused by only collecting lighting information within the screen. By using the indirect lighting information collected by the sampling light in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and the probe mixed lighting information of the screen probe is generated based on the surface element mixed lighting information and the surface element to be propagated. Finally, the screen is rendered by the probe mixed lighting information. This can collect lighting information more comprehensively and accurately, thereby improving the realism of lighting rendering of virtual content on the screen, providing users with more realistic lighting rendering effects, and improving user experience.

[0266] On the other hand, the computer can generate new facets based on the positional information of the locations hit by sampling rays and probe rays, rather than relying on the screen's geometric cache to generate a fixed number of facets. New facets will continue to emit sampling rays to generate facets, achieving infinite light propagation, which is more consistent with the actual light propagation. Facets and their corresponding hash indexes that have not been used for a long time can also be recycled for subsequent use when generating new facets, avoiding the repeated generation of hash indexes, reducing the amount of computation, and avoiding occupying the facet cache, thereby improving the rendering efficiency of the screen lighting in the virtual scene space.

[0267] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0268] See Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 9 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Figure 9 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0269] In such Figure 9 In the computer device 1000 shown, the network interface 1004 can provide a network communication element; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0270] Obtain M screen probes set on the screen image, determine the surface elements hit by the M screen probes in the virtual scene space as the surface elements to be propagated, and obtain sampling rays emitted by the surface elements to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented in the screen image and virtual content outside the screen image; M and P are positive integers;

[0271] The surface element hit by the sampling rays of the P light directions in the virtual scene space is determined as the surface element to be collected, and the lighting information stored in the surface element to be collected is determined as the indirect lighting information collected by the sampling rays of the P light directions in the virtual scene space;

[0272] Based on the indirect lighting information collected by the sampling light in P light directions in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, and based on the surface element mixed lighting information and the surface element to be propagated, the probe mixed lighting information of M screen probes is generated, and the screen image is rendered based on the probe mixed lighting information.

[0273] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 、 Figure 4 and Figure 7 The description of the data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0274] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the processor executes the computer program, it can execute the above Figure 3 、 Figure 4 and Figure 7 The description of the above-mentioned data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0275] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been displayed or is about to be displayed.

[0276] In addition, it should be noted that the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned Figure 3 、 Figure 4 and Figure 7 The method provided by any corresponding embodiment.

[0277] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0278] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example in terms of network elements. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described network elements for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0279] The methods and related devices provided in the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the steps in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0280] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0281] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0282] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Acquire M screen probes set on a screen image, determine the surface elements hit by the M screen probes in a virtual scene space as surface elements to be propagated, and obtain sampling rays emitted by the surface elements to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented in the screen image and virtual content outside the screen image; M and P are positive integers; Determine the surface element hit by the sampling light rays of the P light directions in the virtual scene space as the surface element to be collected, and determine the illumination information stored in the surface element to be collected as the indirect illumination information collected by the sampling light rays of the P light directions in the virtual scene space; The illumination information stored in the surface element to be collected includes at least one of illumination information at the location of the surface element to be collected or illumination information stored in the surface element subsequently hit by the surface element to be collected; Based on the indirect lighting information collected by the sampling light rays of the P light directions in the virtual scene space, the surface element mixed lighting information of the surface element to be propagated is generated, based on the surface element mixed lighting information and the surface element to be propagated, the probe mixed lighting information of the M screen probes is generated, and the screen image is illuminated and rendered based on the probe mixed lighting information.

2. The method according to claim 1, characterized in that The P light directions include a target light direction; and determining, in the virtual scene space, the surface elements hit by the sampling light rays of the P light directions as the surface elements to be collected, including: Determining a position in the virtual scene space hit by the sampling light in the target light direction based on the sampling light in the target light direction and the world coordinates of the virtual content in the virtual scene space; If a surface element exists at the position hit by the sampling light in the target light direction in the virtual scene space, the surface element hit by the sampling light in the target light direction is determined as the surface element to be collected; If no face element exists at the position hit by the sampling light in the target light direction in the virtual scene space, then the position information of the position hit by the sampling light in the target light direction is obtained, and based on the position information, a new face element is generated in the virtual scene space, and the new face element is determined as the face element to be collected; the new face element is used to be determined as a new face element to be propagated in the next frame of the screen image.

3. The method according to claim 2, characterized in that The new surface element to be propagated is used to continue to emit sampling light along P light directions in the next frame, and is used to generate surface element mixed lighting information of the new surface element to be propagated through the indirect lighting information collected by the sampling light in the P light directions in the virtual scene space.

4. The method according to claim 1, wherein The generating of the surface element mixed illumination information of the surface element to be propagated based on the indirect illumination information collected by the sampling light rays in the P light directions in the virtual scene space includes: Obtaining illumination weights corresponding to the P light directions, respectively, and determining illumination values ​​corresponding to the P light directions, respectively, based on indirect illumination information collected by sampled light rays of the P light directions in the virtual scene space; The surface element mixed illumination information of the surface element to be propagated is determined based on the P illumination values ​​and the P illumination weights.

5. The method according to claim 4, characterized in that The P light directions include a target light direction; and obtaining illumination weights corresponding to the P light directions respectively includes: Obtaining a normal direction of a position hit by the target light direction in the virtual scene space, and determining a first parameter value of the target light direction based on the normal direction and the target light direction; Based on the number P of light directions and the sampling probability density corresponding to each light direction, the second parameter value of the target light direction is determined, and the illumination weight corresponding to the target light direction is generated by the first parameter value and the second parameter value; the sampling probability density is the probability that the surface element to be propagated is emitted along the P light directions.

6. The method according to claim 1, characterized in that The M screen probes include a target screen probe, and the P light directions include a target light direction; and generating the probe mixed illumination information of the M screen probes based on the surface element mixed illumination information and the surface element to be propagated includes: Determine the surface element to be propagated that is hit by the probe light emitted by the target screen probe in the virtual scene space along the target light direction as the surface element to be selected, obtain historical illumination information of the surface element to be selected, and determine the mixed sampling illumination information of the surface element to be selected in the target light direction based on the surface element mixed illumination information and historical illumination information of the surface element to be selected; Based on the mixed sampling illumination information collected by the probe rays of the P light directions in the virtual scene space, the probe mixed illumination information of the target screen probe is generated.

7. The method according to claim 1, characterized in that The screen picture is divided into M pixel pictures, each pixel picture is provided with a screen probe, the M pixel pictures include a target pixel picture, and the M screen probes include a target screen probe; The performing illumination rendering on the screen image based on the probe mixed illumination information includes: Acquire Q neighboring pixel pictures of the target pixel picture, and determine screen probes in the Q neighboring pixel pictures as neighboring screen probes; Q is a positive integer; Generate probe weights corresponding to the Q neighboring screen probes based on pixel distances between the Q neighboring screen probes and the target pixel image and pixel depths corresponding to the Q neighboring screen probes; the pixel depth is the pixel distance between the neighboring screen probe and the camera position corresponding to the screen image; Based on the probe mixed lighting information corresponding to the Q neighboring screen probes and the probe weights, the neighboring mixed lighting information of the target screen probe is generated, and based on the probe mixed lighting information of the target screen probe and the neighboring mixed lighting information, the target pixel image in the screen image is illuminated and rendered.

8. The method according to claim 1, characterized in that The screen image is presented by a target rendering frame, and the surface element stores a last used frame of the surface element via a surface element cache; the method further includes: Determine a bin for which the interval between the last used frame and the target rendering frame in the bin cache is greater than or equal to a recycling threshold as a bin to be recycled; the last used frame refers to a picture frame in which illumination information of the bin is last collected by a probe ray emitted by a screen probe or a sampling ray emitted by the bin to be propagated; Obtaining position information of the to-be-reclaimed surface element in the virtual scene space, and recycling the to-be-reclaimed surface element in a post-rendering frame based on the position information; the post-rendering frame is a next frame of the target rendering frame.

9. The method according to claim 1, characterized in that The virtual scene space is divided into T world grids, each world grid is used to store S facets within the grid space; T and S are positive integers; the method further comprises: When it is detected that a new facet is generated in the virtual scene space, the newly generated facet is determined as a facet to be stored, and position information of the facet to be stored in the virtual scene space is obtained; Based on the position information of the face element to be stored, the face element to be stored is stored in the world grid corresponding to the T world grids to obtain a stored face element; the stored face element is used as the face element to be propagated when it is hit by the M screen probes in the virtual scene space.

10. The method according to claim 9, characterized in that Also includes: Obtaining stored surfels respectively stored in the T world grids, and storing position information, material information, and historical lighting information of the last used frame and the target rendering frame corresponding to the stored surfels in surfel caches corresponding to the T world grids; The historical lighting information of the target rendering frame is determined based on the surface element mixed lighting information of the target rendering frame and the historical lighting information of a previous rendering frame, where the previous rendering frame is a frame previous to the target rendering frame; The step of determining the illumination information stored in the to-be-collected facet as the indirect illumination information collected by the sampling light rays in the P light directions in the virtual scene space includes: Based on the world grid and position information corresponding to the facet to be collected, the lighting information stored in the facet to be collected is obtained in the facet cache, and the lighting information stored in the facet to be collected is determined to be the indirect lighting information collected by the sampling light in the P light directions in the virtual scene space.

11. The method according to claim 1, characterized in that The screen image is divided into M pixel images, and the acquiring of M screen probes set on the screen image and determining the surface elements hit by the M screen probes in the virtual scene space as surface elements to be propagated include: Generate a screen probe at a random position in each pixel image, and emit probe rays along P ray directions in the virtual scene space through the M screen probes; At the position hit by the probe light in the virtual scene space, a surface element is generated, and the generated surface element is determined as the surface element to be propagated.

12. A data processing device, characterized in that: include: a facet determination module configured to obtain M screen probes set on a screen image, determine facets hit by the M screen probes in a virtual scene space as facets to be propagated, and obtain sampling rays emitted by the facets to be propagated along P ray directions in the virtual scene space; the virtual scene space includes virtual content presented in the screen image and virtual content outside the screen image; M and P are positive integers; an illumination collection module, configured to determine the surface elements hit by the sampling rays of the P light directions in the virtual scene space as surface elements to be collected, and determine the illumination information stored in the surface elements to be collected as the indirect illumination information collected by the sampling rays of the P light directions in the virtual scene space; The illumination information stored in the surface element to be collected includes at least one of illumination information at the location of the surface element to be collected or illumination information stored in the surface element subsequently hit by the surface element to be collected; A lighting processing module is used to generate the surface element mixed lighting information of the surface element to be propagated based on the indirect lighting information collected by the sampling light rays of the P light directions in the virtual scene space, generate the probe mixed lighting information of the M screen probes based on the surface element mixed lighting information and the surface element to be propagated, and perform lighting rendering on the screen image based on the probe mixed lighting information.

13. A computer device, characterized in that: include: processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium and is suitable for being read and executed by a processor, so as to enable a computer device having the processor to perform the method according to any one of claims 1 to 11.

Images