Reflection data processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a reflection data processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The reflection data processing method and device are applied to various data reflection scenes such as cloud technology, artificial intelligence, intelligent transportation, games, videos and vehicle-mounted scenes. The reflection data processing method comprises the following steps: performing three-dimensional scene reconstruction on a to-be-reflected screen scene to obtain a to-be-processed three-dimensional scene; performing voxelization on the to-be-reflected screen scene to obtain a to-be-processed voxel scene; in the to-be-processed voxel scene, performing reflection detection on each to-be-processed reflection point in the to-be-processed three-dimensional scene to obtain a voxel detection result; determining a reflection color of the to-be-processed reflection point based on the voxel detection result; and obtaining a reflection color image corresponding to the to-be-processed three-dimensional scene according to the reflection color of the to-be-processed reflection point, and rendering the to-be-reflected screen scene based on the reflection color image to obtain a target virtual scene. According to the invention, the reflection effect can be improved.

Description

Technical Field

[0001] This application relates to reflection data processing technology in the field of rendering, and more particularly to a reflection data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] When rendering virtual scenes based on screen scenes, the rendering effect can be improved by reflecting the screen scene. The process of reflecting the screen scene is called screen space reflection (SSR). However, in order to achieve screen space reflection, the reflection color is often determined based on the pixel color of the screen space. This makes the reflected information screen space information, which affects the integrity of the reflection and thus affects the reflection effect. Summary of the Invention

[0003] This application provides a reflection data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve reflection effects.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a reflection data processing method, the method comprising:

[0006] The 3D scene of the screen to be reflected is reconstructed to obtain the 3D scene to be processed.

[0007] The screen scene to be reflected is voxelized to obtain the voxel scene to be processed;

[0008] In the voxel scene to be processed, reflection detection is performed on each reflection point in the three-dimensional scene to be processed to obtain voxel detection results. The voxel detection results indicate whether a voxel used to reflect to the reflection point to be processed is detected.

[0009] Based on the voxel detection results, the reflection color of the reflection point to be processed is determined;

[0010] The reflection color image corresponding to the three-dimensional scene to be processed is obtained from the reflection color of the reflection point to be processed. The screen scene to be reflected is rendered based on the reflection color image to obtain the target virtual scene.

[0011] This application provides a reflection data processing apparatus, the reflection data processing apparatus comprising:

[0012] The scene reconstruction module is used to reconstruct the 3D scene of the screen to be reflected, and obtain the 3D scene to be processed.

[0013] A voxel representation module is used to voxelize the screen scene to be reflected to obtain a voxel scene to be processed.

[0014] The reflection detection module is used to perform reflection detection on each reflection point in the three-dimensional scene to be processed in the scene to be processed, and obtain a voxel detection result, wherein the voxel detection result indicates whether a voxel to be reflected toward the reflection point to be processed is detected.

[0015] The color determination module is used to determine the reflection color of the reflection point to be processed based on the voxel detection results.

[0016] The scene rendering module is used to obtain a reflection color image corresponding to the three-dimensional scene to be processed from the reflection color of the reflection point to be processed, and to render the screen scene to be reflected based on the reflection color image to obtain the target virtual scene.

[0017] In this embodiment of the application, the voxel representation module is further configured to voxelize the screen scene to be reflected to obtain an initial voxel scene; determine the target scene object to which each initial voxel in the initial voxel scene belongs; determine the voxel distance between the initial voxel and the target scene object; and combine the voxel distance and the voxel color of the initial voxel to determine the voxel to be processed, thereby obtaining the voxel scene to be processed including multiple voxels to be processed.

[0018] In this embodiment of the application, the reflection detection module is further configured to perform the following processing on each reflection point in the three-dimensional scene to be processed: determine the reflection direction by combining the reflection point position and reflection point normal of the reflection point and the viewing direction; and perform reflection detection in the voxel scene to be processed along the reflection direction with the reflection point position as the reflection detection starting point to obtain the voxel detection result.

[0019] In this embodiment of the application, the reflection detection module is further configured to perform reflection detection on each reflection point in the three-dimensional scene to be processed in the scene to be processed, and determine the current voxel to be processed corresponding to the current detection position; when the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object, the current voxel to be processed is determined as the reflected voxel; and the detected reflected voxel is determined as the voxel detection result.

[0020] In this embodiment, the reflection detection module is further configured to acquire reflection detection parameters when the current voxel distance indicates that the current voxel to be processed is located outside the target scene object to which it belongs. The reflection detection parameters include one or both of reflection detection distance and reflection detection count. When it is determined based on the reflection detection parameters that the reflection detection termination condition is met, the voxel that was not detected is determined as the voxel detection result. The reflection detection termination condition includes one or both of the following: the reflection detection distance is greater than the reflection detection distance threshold, and the reflection detection count is greater than the reflection detection count threshold.

[0021] In this embodiment of the application, the reflection detection module is further configured to advance a specified step length along the reflection direction to obtain the next voxel to be processed when it is determined based on the reflection detection parameters that the reflection detection continuation condition is met. The reflection detection continuation condition includes one or two of the following: the reflection detection distance is less than or equal to the reflection detection distance threshold, and the reflection detection count is less than or equal to the reflection detection count threshold; and the voxel detection result is determined based on the next voxel to be processed.

[0022] In this embodiment of the application, the reflection detection module is further configured to: obtain the current voxel detection count when the current voxel distance indicates that the current voxel is located on the surface or inside the target scene object to which it belongs; and determine the current voxel as the reflected voxel when the current voxel detection count indicates that the current voxel is the voxel after the first voxel.

[0023] In this embodiment of the application, the reflection detection module is further configured to determine the undetected voxel as the voxel detection result when the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed.

[0024] In this embodiment, the reflection detection module is further configured to: determine the reflection level based on the reflection parameters of the reflection point to be processed when the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed; the reflection parameters include metallicity and roughness; when the reflection level is a first reflection level, determine the voxel that was not detected as the voxel detection result, wherein the first reflection level refers to the metallicity being greater than a metallicity threshold and the roughness being greater than a roughness threshold; when the reflection level is a second reflection level, determine the first voxel to be processed as the voxel to be reflected, and obtain the voxel detection result that detected the voxel to be reflected, wherein the second reflection level refers to the metallicity being greater than the metallicity threshold and the roughness being less than or equal to the roughness threshold.

[0025] In this embodiment of the application, the color determination module is further configured to determine the specified color as the reflection color of the reflection point to be processed when the voxel detection result indicates that the reflected voxel is not detected; and to determine the reflection color of the reflection point to be processed based on the reflected voxel when the voxel detection result indicates that the reflected voxel is detected.

[0026] In this embodiment of the application, the color determination module is further configured to determine the reflection level based on the reflection parameters of the reflection point to be processed; when the reflection level is the first reflection level, the voxel color of the reflected voxel is determined as the reflection color of the reflection point to be processed.

[0027] In this embodiment of the application, the color determination module is further configured to: determine the voxel start point and voxel end point where the reflected light enters the reflected voxel when the reflection level is the second reflection level; determine the reflection detection range of the voxel start point and the voxel end point in the screen space; perform reflection detection within the reflection detection range to obtain a pixel detection result, wherein the pixel detection result indicates whether a reflected pixel for reflecting to the reflection point to be processed is detected; and determine the reflection color of the reflection point to be processed based on the pixel detection result.

[0028] In this embodiment of the application, the color determination module is further configured to determine the voxel color of the reflected voxel as the reflection color of the reflection point to be processed when the pixel detection result indicates that the reflected pixel is not detected; and to determine the pixel color of the reflected pixel as the reflection color of the reflection point to be processed when the voxel detection result indicates that the reflected pixel is detected.

[0029] In this embodiment of the application, the scene reconstruction module is further configured to perform the following processing on each screen pixel to be reflected in the screen scene to be reflected: combining the scene depth image and spatial transformation parameters corresponding to the screen scene to be reflected, determining the reflection point position and reflection point normal of the screen pixel to be reflected in world space; combining the reflection point position and the reflection point normal, determining the reflection point to be processed, and obtaining the three-dimensional scene to be processed including multiple reflection points to be processed.

[0030] This application provides an electronic device for reflection data processing, the electronic device comprising:

[0031] Memory is used to store executable instructions or computer programs.

[0032] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the reflection data processing method provided in the embodiments of this application.

[0033] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the reflection data processing method provided in this application.

[0034] This application provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the reflection data processing method provided in this application.

[0035] The embodiments of this application have at least the following beneficial effects: When performing reflection processing based on a screen scene to be reflected, the screen scene to be reflected is first voxelized into a voxel scene to be processed. Then, reflection detection is performed on each reflection point corresponding to the screen scene to be reflected in the voxel scene to be processed, so as to obtain the voxel detection result of whether the reflected voxel is detected. Then, the screen scene to be reflected is rendered based on the reflection color determined by the voxel detection result. That is to say, the reflection detection is based on voxels, the reflection color is determined based on the detection result of the reflected voxel, and the voxel is information of world space. Therefore, the obtained reflection information includes not only information of screen space, but also information of world space, which improves the integrity of reflection and thus improves the reflection effect. Attached Figure Description

[0036] Figure 1 This is an exemplary schematic diagram of the step-by-step result;

[0037] Figure 2 This is an exemplary diagram illustrating the storage of depth information;

[0038] Figure 3 This is a schematic diagram of the architecture of the reflection data processing system provided in the embodiments of this application;

[0039] Figure 4 This is one of the embodiments provided in this application. Figure 3 A schematic diagram of the terminal structure in the diagram;

[0040] Figure 5 This is a flowchart illustrating the reflection data processing method provided in the embodiments of this application. Figure 1 ;

[0041] Figure 6 This is a flowchart illustrating the reflection data processing method provided in the embodiments of this application. Figure 2 ;

[0042] Figure 7 This is a schematic diagram of the process for obtaining voxel detection results provided in an embodiment of this application;

[0043] Figure 8This is a schematic diagram of the process for determining the reflected color provided in an embodiment of this application;

[0044] Figure 9 This is an exemplary screen space reflection flowchart provided in an embodiment of this application;

[0045] Figure 10 This is a schematic diagram of an exemplary voxel field provided in an embodiment of this application;

[0046] Figure 11 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 1 ;

[0047] Figure 12 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 2 ;

[0048] Figure 13 This is another exemplary screen space reflection flowchart provided in the embodiments of this application;

[0049] Figure 14 This is an exemplary schematic diagram of ray tracing within a voxel provided in an embodiment of this application;

[0050] Figure 15 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 3 ;

[0051] Figure 16 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 4 ;

[0052] Figure 17 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 5 ;

[0053] Figure 18 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 6 . Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] In the following description, the terms "first" and "second" are used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0057] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0058] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0059] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0060] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0061] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science used to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0062] It should be noted that artificial intelligence (AI) technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large models or foundational models, can be widely applied to downstream tasks in various areas of AI after fine-tuning. AI software technologies include computer vision, speech processing, natural language processing, and machine learning / deep learning. In this embodiment, the rendering of the target virtual scene can be achieved using AI technology.

[0063] 2) A virtual scene is a virtual scene displayed through audio and video information sent by a cloud server to an application running on it. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application embodiment does not limit the dimension of the virtual scene. For example, a virtual scene may include a virtual sky, virtual land, virtual ocean, etc., and the virtual land may include environmental elements such as virtual deserts and virtual cities. Users can control virtual objects to move within this virtual scene. In this application embodiment, a game scene is an example of a virtual scene.

[0064] 3) Virtual objects are interactive images of various people and things within a virtual scene, or movable objects within a virtual scene. These movable objects can be virtual characters, virtual animals, anime characters, and virtual props, such as people and animals displayed in a virtual scene; a virtual object can also be a virtual avatar representing the user within the virtual scene. A virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space within the virtual scene. In this embodiment, the target scene object is a virtual object.

[0065] 4) Ray tracing is a rendering technique in computer graphics used to simulate the propagation and interaction of light rays to generate realistic and lifelike images. For example, ray tracing can be applied to screen-space reflections.

[0066] 5) Screen space reflection refers to a specular reflection technology based on screen space; where screen space refers to the pixel space of the image, corresponding to a two-dimensional image coordinate system.

[0067] It should be noted that in order to achieve screen space reflection, hardware-based ray tracing, 3D ray tracing based on depth reconstruction of object position, pixel-based linear 2D ray tracing, or high Z-value ray tracing (Hi-Z Trace) can be used.

[0068] When using hardware-based ray tracing to achieve screen-space reflections, the hardware-based implementation prevents its applicability to devices without ray tracing hardware. Furthermore, since the reflected color is determined by the pixel colors in screen space, the reflected information is screen-space information, affecting the integrity of the reflection and consequently impacting the reflection effect.

[0069] When screen-space reflection is achieved using 3D ray tracing based on depth-reconstructed object positions, rays are emitted from screen pixels and move in the virtual scene at certain steps in 3D space to detect information. The position of the virtual object is inferred from the depth of point A hit by the ray, and further detection is performed based on the calculated reflection direction. The pixel color of the point detected in the reflection direction is assigned to point A, thus completing the reflection processing. However, moving in the scene at certain steps to detect information increases performance overhead; furthermore, the precision that a screen pixel can represent is limited. For example, if virtual object A is far from the camera (greater than a specified camera distance), the information traced by a single pixel includes not only virtual object B but also virtual objects around virtual object B (within a specified range), affecting the precision of screen-space reflection; and although ray tracing is performed in 3D space, the obtained reflection information is only screen-space information, and information outside the screen is missing, affecting the integrity of the reflection and thus the reflection effect.

[0070] Furthermore, due to perspective effects, stepping in a virtual scene with a certain step size in 3D space cannot produce uniform 2D stepping within screen space, causing reflection processing to fail to capture virtual objects surrounding the object to be reflected. For example, see... Figure 1 , Figure 1 This is an exemplary schematic diagram of the stepping result; such as Figure 1 As shown, the stepping result 1-1 obtained by stepping in the scene with a certain step size in three-dimensional space is shown in the screen space; it is easy to see that the stepping result 1-1 is not uniform.

[0071] When using Linear 2D to implement screen-space reflections, Linear 2D introduces perspective correction interpolation based on Digital Differential Analyzer (DDA), which can improve the uniformity of the stepping results; see further... Figure 1The stepping results 1-2 for screen-space reflection using Linear2D are uniform. However, when performing screen-space reflection, the reflected color is determined based on the pixel color in screen space, resulting in the reflected information being screen-space information. This affects the integrity of the reflection and thus the reflection effect; furthermore, it affects the reflection efficiency.

[0072] When using high Z-value ray tracing to implement screen-space reflections, since high Z-values ​​represent a screen-space quadtree, depth values ​​are stored using multi-level asymptotic texture (MIP) channels. During storage, the depth value of each layer is the minimum or maximum of the four adjacent values ​​of the layer above. This allows for stepping based on the stored depth values. For example, see [link to example]. Figure 2 , Figure 2 This is an exemplary diagram illustrating the storage of depth information; such as... Figure 2 As shown, each depth value in layer 2-4 is determined by the four depth values ​​corresponding to layer 2-3, each depth value in layer 2-3 is determined by the four depth values ​​corresponding to layer 2-2, each depth value in layer 2-2 is determined by the four depth values ​​corresponding to layer 2-1, and each depth value in layer 2-1 is determined by the four depth values ​​corresponding to layer 2-0. However, when using high Z-value ray tracing to achieve screen-space reflection, the reflected color is determined based on the pixel color in screen space, making the reflected information screen-space information, affecting the integrity of the reflection and thus affecting the reflection effect. Furthermore, when using high Z-value ray tracing to achieve screen-space reflection, different graphics processing interfaces (such as DX and OpenGL) have different depth value storage levels, requiring determination of the image processing interface used, increasing the resource consumption of reflection processing and affecting reflection efficiency.

[0073] Based on this, embodiments of this application provide a reflection data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve reflection effects and reflection efficiency. The following describes exemplary applications of the electronic device for reflection data processing (hereinafter referred to as the reflection data processing device) provided in embodiments of this application. The reflection data processing device provided in embodiments of this application can be implemented as various types of terminals such as robots, smartphones, smartwatches, laptops, tablets, desktop computers, smart home appliances, set-top boxes, smart vehicle devices, portable music players, personal digital assistants, dedicated messaging devices, intelligent voice interaction devices, portable gaming devices, and smart speakers, or it can be implemented as a server. The following will describe exemplary applications when the reflection data processing device is implemented as a terminal.

[0074] See Figure 3 , Figure 3 This is a schematic diagram of the architecture of the reflection data processing system provided in the embodiments of this application; as shown Figure 3As shown, to support a reflection data processing application, in the reflection data processing system 100, a terminal 400 (terminals 400-1 and 400-2 are shown as examples) connects to a server 200 via a network 300; the network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both; and the server 200 provides computing services to the terminal 400 via the network 300. Additionally, the reflection data processing system 100 also includes a database 500 for providing data support to the server 200; and... Figure 3 The example shown illustrates a scenario where the database 500 is independent of the server 200. However, the database 500 can also be integrated into the server 200, and this embodiment does not limit this to any particular case.

[0075] Terminal 400 is used to reconstruct a 3D scene of the screen to be reflected to obtain a 3D scene to be processed; to voxelize the screen to be reflected to obtain a voxel scene to be processed; to perform reflection detection on each reflection point in the 3D scene to be processed in the voxel scene to be processed to obtain a voxel detection result, the voxel detection result indicating whether a reflected voxel used to reflect to the reflection point to be processed is detected; to determine the reflection color of the reflection point to be processed based on the voxel detection result; to obtain a reflection color image corresponding to the 3D scene to be processed from the reflection color of the reflection point to be processed; and to render the screen to be reflected based on the reflection color image to obtain a target virtual scene (graphical interfaces 410-1 and 410-2 are shown as examples).

[0076] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0077] See Figure 4 , Figure 4 This is one of the embodiments provided in this application. Figure 3 A schematic diagram of the terminal structure in the diagram; such as Figure 4As shown, terminal 400 includes at least one processor 410, memory 450, at least one network interface 420, and user interface 430. The various components in terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general labeled all buses as Bus System 440.

[0078] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0079] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0080] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0081] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0082] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0083] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0084] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0085] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0086] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0087] In some embodiments, the reflection data processing apparatus provided in this application can be implemented in software. Figure 4 A reflection data processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a scene reconstruction module 4551, a voxel representation module 4552, a reflection detection module 4553, a color determination module 4554, and a scene rendering module 4555. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0088] In some embodiments, the reflection data processing apparatus provided in this application can be implemented in hardware. As an example, the reflection data processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the reflection data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0089] In some embodiments, the terminal or server can implement the reflection data processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as live streaming APPs or game APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0090] The following will describe the reflection data processing method provided in this application embodiment, with reference to exemplary applications and implementations of the reflection data processing device provided in the embodiments of this application. Furthermore, the reflection data processing method provided in the embodiments of this application is applicable to various data reflection scenarios such as cloud technology, artificial intelligence, smart transportation, games, video, and vehicle-mounted systems.

[0091] See Figure 5 , Figure 5 This is a flowchart illustrating the reflection data processing method provided in the embodiments of this application. Figure 1 ,in, Figure 5 The execution entity for each step is the reflection data processing device; the following will combine... Figure 5 The steps shown are explained.

[0092] Step 101: Reconstruct the three-dimensional scene of the screen to be reflected to obtain the three-dimensional scene to be processed.

[0093] In this embodiment, when the reflection data processing device receives a scene rendering request, it responds to the request by obtaining the screen scene to be reflected based on the identification information in the scene rendering request, or by obtaining the screen scene to be reflected from the scene rendering request. The scene rendering request refers to a request to render a virtual scene, such as a game scene rendering request during gameplay, a video frame rendering request in a video playback scene, or an image rendering request in an information browsing scene. Since the screen scene to be reflected is scene data in screen space, in order to perform reflection processing on the screen scene to be reflected, the reflection data processing device reconstructs the screen scene to be reflected into scene data in three-dimensional world space, thus obtaining the three-dimensional scene to be processed.

[0094] It should be noted that the screen scene to be reflected refers to the scene data in the screen space to be rendered, where screen space is pixel space; the screen scene to be reflected includes multiple screen pixels to be reflected, where each screen pixel represents a pixel in the screen scene to be reflected. 3D scene reconstruction refers to the process of converting scene data from screen space to world space, where world space refers to the space corresponding to the world coordinate system. The 3D scene to be processed includes multiple reflection points to be processed, where each reflection point represents the corresponding information of the screen pixels to be reflected in world space.

[0095] In this embodiment of the application, the reflection data processing device performs three-dimensional scene reconstruction on the screen scene to be reflected to obtain a three-dimensional scene to be processed. The process includes: the reflection data processing device performs the following processing on each pixel of the screen scene to be reflected: combining the scene depth image and spatial transformation parameters corresponding to the screen scene to be reflected, determining the reflection point position and reflection point normal of the pixel in world space; and combining the reflection point position and reflection point normal to determine the reflection point to be processed; thereby obtaining a three-dimensional scene to be processed including multiple reflection points to be processed.

[0096] It should be noted that since the scene to be reflected includes multiple pixels, the reflection data processing device processes each pixel in the scene to achieve 3D scene reconstruction. Here, when reconstructing the 3D scene, the reflection data processing device can also obtain the depth information corresponding to the scene, i.e., the scene depth image; and the spatial transformation parameters corresponding to the scene, such as the camera matrix, used to achieve data conversion between screen space and world space. Therefore, by combining the scene depth image and the spatial transformation parameters, the reflection data processing device can determine the position and normal of each pixel in world space, called the reflection point position and reflection point normal. It is easy to see that the reflection point position represents the location used for reflection information.

[0097] Step 102: Voxelize the scene to be reflected on the screen to obtain the voxel scene to be processed.

[0098] In this embodiment of the application, the reflection data processing device performs ray tracing in the voxelized representation of the screen scene to be reflected; thus, the reflection data processing device voxels the screen scene to be reflected, and the obtained voxelized representation is the voxel scene to be processed.

[0099] It should be noted that voxelization refers to the process of decomposing each scene object in the scene into at least one cube (voxel). The voxel scene to be processed includes multiple voxels to be processed, and each voxel to be processed is a voxel in the voxel scene to be processed; wherein, the size of the voxel to be processed can be determined based on the actual implementation, and this application embodiment does not limit it.

[0100] It should also be noted that, Figure 5 The execution order of steps 101 and 102 shown is an exemplary execution order; steps 101 and 102 are not in any particular order of execution, they can be executed simultaneously, sequentially, or a combination of the two, etc., and this application embodiment does not limit this.

[0101] See Figure 6 , Figure 6 This is a flowchart illustrating the reflection data processing method provided in the embodiments of this application. Figure 2 ,in, Figure 6 The execution entity for each step is the reflection data processing device; such as Figure 6 As shown in the embodiment of this application, step 102 can be implemented by steps 1021 to 1024; that is, the reflection data processing device performs voxelization on the scene to be reflected on the screen to obtain the voxel scene to be processed, including steps 1021 to 1024. Each step will be described below.

[0102] Step 1021: Voxelize the scene to be reflected on the screen to obtain the initial voxel scene.

[0103] It should be noted that the reflection data processing device voxels the scene to be reflected on the screen, and the result obtained is the initial voxel scene. Each voxel in the initial voxel scene is called an initial voxel. The size of the initial voxel is the same as the size of the voxel to be processed. The initial voxel includes a corresponding color, which is called the voxel color, that is, the color of the voxel in the initial voxel scene.

[0104] Step 1022: Determine the target scene object to which each initial voxel in the initial voxel scene belongs.

[0105] It should be noted that since the initial voxel scene is obtained by voxelizing each scene object in the scene to be reflected on the screen, each initial voxel corresponds to a scene object; here, the scene object to which the initial voxel belongs is called the target scene object; thus, the reflection data processing device can determine a corresponding target scene object for each initial voxel.

[0106] Step 1023: Determine the voxel distance between the initial voxel and the target scene object.

[0107] In this embodiment, the reflection data processing device determines the positional relationship between the initial voxel and the target scene object, thus obtaining the voxel distance. The voxel distance is a distance representing the positional relationship between the initial voxel and the target scene object, such as the sign distance function (SDF) value.

[0108] For example, when the voxel distance is positive, it means that the initial voxel is outside the target scene object; when the voxel distance is 0, it means that the initial voxel is on the surface of the target scene object; when the voxel distance is negative, it means that the initial voxel is on the surface or inside the target scene object.

[0109] Step 1024: Combine the voxel distance and the voxel color of the initial voxel to determine the voxel to be processed, and obtain a voxel scene including multiple voxels to be processed.

[0110] In this embodiment, the reflection data processing device combines voxel distance and voxel color to form a voxel to be processed; wherein, the voxel to be processed corresponds to the initial voxel, and the voxel to be processed includes the voxel distance and voxel color of the initial voxel. Here, after the reflection data processing device obtains the voxel to be processed corresponding to each initial voxel in the initial voxel scene, it also obtains multiple voxels to be processed corresponding to multiple initial voxels, and the multiple voxels to be processed are used to form the voxel scene to be processed.

[0111] Understandably, by representing the screen scene to be reflected as a voxel scene to be processed, ray tracing efficiency can be improved based on voxel distance, since each voxel in the voxel scene includes not only voxel color but also voxel distance. Furthermore, the voxel scene to be processed provides the feasibility for obtaining complete reflection information.

[0112] Step 103: In the voxel scene to be processed, perform reflection detection on each reflection point in the 3D scene to be processed to obtain the voxel detection result.

[0113] In this embodiment, the reflection data processing device performs the following processing on each reflection point in the three-dimensional scene to be processed: reflection detection is performed on the reflection point in the voxel scene to be processed, and the obtained reflection detection result is the voxel detection result. The voxel detection result indicates whether a voxel used to reflect information to the reflection point to be processed is detected. Therefore, the voxel detection result can indicate whether a voxel is detected or not; and the reflected voxel refers to the voxel in the voxel scene to be processed used to reflect information to the reflection point to be processed.

[0114] It should be noted that reflection detection is the same as ray tracing, so step 103 describes the process of determining the reflected voxel through ray tracing.

[0115] In step 103 of this embodiment, the reflection data processing device performs reflection detection on each reflection point in the three-dimensional scene to be processed in the voxel scene to be processed, and obtains voxel detection results. This includes: the reflection data processing device performs the following processing on each reflection point in the three-dimensional scene to be processed: first, it determines the reflection direction by combining the reflection point position, reflection point normal, and viewing direction of the reflection point; then, it performs reflection detection along the reflection direction in the voxel scene to be processed, taking the reflection point position as the reflection detection starting point, and obtains voxel detection results.

[0116] In the embodiments of this application, see Figure 7 , Figure 7 This is a schematic diagram of the process for obtaining voxel detection results provided in an embodiment of this application, wherein, Figure 7 The execution entity for each step is the reflection data processing device; such as Figure 7 As shown in the embodiment of this application, step 103 can be implemented by steps 1031 to 1033; that is, the reflection data processing device performs reflection detection on each reflection point in the three-dimensional scene to be processed in the voxel scene to be processed, and obtains the voxel detection result, including: voxelizing the screen scene to be reflected to obtain the voxel scene to be processed, including steps 1031 to 1033. Each step is described below.

[0117] Step 1031: In the voxel scene to be processed, perform reflection detection on each reflection point in the 3D scene to be processed, and determine the current voxel to be processed corresponding to the current detection position.

[0118] In this embodiment of the application, the reflection data processing device performs reflection detection on each reflection point in the three-dimensional scene to be processed in the voxel scene to be processed. This reflection detection can be performed along the reflection direction in the voxel scene to be processed. Based on the current detection position determined by the reflection detection, the voxel to be processed corresponding to the current detection position is determined, thus obtaining the current voxel to be processed.

[0119] Step 1032: When the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object, the current voxel to be processed is determined as the reflected voxel.

[0120] It should be noted that since each voxel to be processed includes a voxel distance, and the current voxel to be processed is the voxel to be processed corresponding to the current detection position, the current voxel to be processed also includes the corresponding voxel distance; here, the voxel distance of the current voxel to be processed is called the current voxel distance. Since the current voxel distance represents the positional relationship between the current voxel to be processed and the target scene object to which the current voxel to be processed belongs, when the current voxel distance indicates that the current voxel to be processed is located on the surface or inside the target scene object to which it belongs, it indicates that the reflection data processing device has detected a candidate reflected voxel; therefore, the reflection data processing device can directly determine the current voxel to be processed as a reflected voxel, or it can determine the voxel detection result after judging other information of the current voxel to be processed. This application embodiment does not limit this.

[0121] In this embodiment of the application, step 1032 can be implemented by steps 10321 and 10322; that is, when the current voxel distance corresponding to the voxel to be processed indicates that the voxel to be processed is located on the surface or inside the target scene object, the reflection data processing device determines the voxel to be processed as the reflected voxel, including steps 10321 and 10322. Each step is described below.

[0122] Step 10321: When the current voxel distance corresponding to the voxel to be processed indicates that the voxel to be processed is located on the surface or inside the target scene object, obtain the current voxel detection count.

[0123] It should be noted that when the current voxel distance indicates that the voxel to be processed is located on or inside the surface of the target scene object, the reflection data processing device further determines the voxel order corresponding to the voxel to be processed to ascertain whether the voxel to be processed is the first voxel to be processed detected by reflection. Here, the reflection data processing device determines the voxel order corresponding to the voxel to be processed as the current voxel detection count; thus, the current voxel detection count represents the order of the currently detected voxels to be processed.

[0124] Step 10322: When the current voxel detection count indicates that the voxel to be processed is the voxel to be processed after the first voxel to be processed, the current voxel to be processed is determined as the reflected voxel.

[0125] In this embodiment, when the current voxel detection count indicates that the voxel to be processed is the voxel to be processed after the first voxel to be processed, it means that the current voxel to be processed is not the first voxel to be processed. At this time, the current voxel to be processed is located on the surface or inside the target scene object, indicating that a voxel used for reflection information has been detected, and the current voxel to be processed is the voxel to be reflected. Therefore, the reflection data processing device determines the current voxel to be processed as the voxel to be reflected.

[0126] Step 1033: The detected reflective voxels are identified as voxel detection results.

[0127] It should be noted that the reflection data processing device identifies the detected voxel as a voxel detection result, which means that the voxel was detected.

[0128] In this embodiment of the application, after the reflection data processing device obtains the current voxel detection count, the reflection data processing method further includes: when the current voxel detection count indicates that the voxel to be processed is the first voxel to be processed, the reflection data processing device determines the undetected reflected voxel as the voxel detection result.

[0129] It should be noted that when the current voxel distance indicates that the voxel to be processed is located on the surface or inside the target scene object, if the current voxel detection count indicates that the voxel to be processed is the first voxel to be processed, then the reflection data processing device can directly determine that no reflected voxel has been detected, and thus the voxel detection result is that no reflected voxel has been detected.

[0130] It is understandable that when the first voxel to be processed is detected to be located on the surface or inside the target scene object, it is directly determined that no reflective voxel has been detected, and the reflection detection for the reflection point to be processed ends, thereby improving the reflection detection efficiency and the reflection processing efficiency.

[0131] In this embodiment of the application, after the reflection data processing device obtains the current voxel detection count, the reflection data processing method further includes: when the current voxel detection count indicates that the voxel to be processed is the first voxel to be processed, the reflection data processing device first determines the reflection level based on the reflection parameters of the reflection point to be processed; when the reflection level is the first reflection level, the undetected reflected voxel is determined as the voxel detection result; and when the reflection level is the second reflection level, the first voxel to be processed is determined as the reflected voxel, thus obtaining the voxel detection result of the detected reflected voxel.

[0132] It should be noted that when the current voxel distance indicates that the voxel to be processed is located on or inside the surface of the target scene object, if the current voxel detection count indicates that the voxel to be processed is the first voxel to be processed, the reflection data processing device can also determine the reflection level of the reflection point; where the reflection level represents the degree of material reflection of the reflection point. Here, the reflection data processing device determines the reflection level based on the reflection parameters of the reflection point to be processed, including metallicity and roughness. When the reflection level is the first reflection level, it indicates that the material reflection degree of the reflection point to be processed is low, thus determining that no reflected voxel has been detected, and ending the reflection detection for that reflection point, identifying the undetected reflected voxel as the voxel detection result. When the reflection level is the second reflection level, it indicates that the material reflection degree of the reflection point to be processed is high, thus identifying the first voxel to be processed as the reflected voxel, obtaining the voxel detection result of the detected reflected voxel, and performing pixel-level reflection detection on the first voxel to be processed.

[0133] It should also be noted that the first reflection level refers to a metallicity greater than the metallicity threshold and a roughness greater than the roughness threshold, while the second reflection level refers to a metallicity greater than the metallicity threshold and a roughness less than or equal to the roughness threshold. Materials of the first reflection level have a lower degree of reflection than those of the second reflection level.

[0134] Understandably, for the first voxel to be processed, determining whether the voxel to be reflected is detected by judging the degree of material reflection of the reflection point can improve the targeting of reflection processing and thus improve the reflection processing effect.

[0135] See also Figure 7 In step 1031 of this application embodiment, after the reflection data processing device determines the current voxel to be processed corresponding to the current detection position, the reflection data processing method further includes steps 1034 and 1035, which will be described below.

[0136] Step 1034: When the current voxel distance indicates that the voxel to be processed is located outside the target scene object, obtain the reflection detection parameters.

[0137] It should be noted that when the current voxel distance indicates that the voxel to be processed is located outside the target scene object, it means that the voxel to be processed is not a reflected voxel. Therefore, the reflection data processing device determines whether to continue reflection detection. Here, the reflection data processing device determines whether to continue reflection detection for the reflection point to be processed based on reflection detection parameters. Reflection detection parameters include one or both of the following: reflection detection distance and reflection detection count. The reflection detection distance refers to the distance at which reflection detection has been performed on the reflection point to be processed, and the reflection detection count refers to the number of times reflection detection has been performed on the reflection point to be processed. Each time reflection detection is completed for a voxel to be processed, the reflection detection count increases by a specified number, such as 1.

[0138] Step 1035: When the reflection detection end condition is met based on the reflection detection parameters, the undetected voxels are identified as voxel detection results.

[0139] It should be noted that when the reflection data processing device determines that the reflection detection termination condition is met based on the reflection detection parameters, it determines that the reflection point to be processed has completed reflection detection and no reflected voxel has been detected, thus ending the reflection detection of the reflection point to be processed and obtaining a voxel detection result indicating that no reflected voxel was detected. The reflection detection termination condition includes one or both of the following: the reflection detection distance is greater than the reflection detection distance threshold, and the number of reflection detections is greater than the reflection detection count threshold.

[0140] In this embodiment of the application, after the reflection data processing device acquires the reflection detection parameters, the reflection data method further includes steps 1036 and 1037, which will be described below.

[0141] Step 1036: When the reflection detection parameters determine that the reflection detection continuation condition is met, advance a specified step length along the reflection direction to obtain the next voxel to be processed.

[0142] It should be noted that when the reflection data processing device determines that the reflection detection continuation condition is met based on the reflection detection parameters, it determines that the reflection point to be processed has not completed reflection detection and no reflected voxel has been detected, and thus continues the reflection detection of the reflection point to be processed. Here, the reflection data processing device advances a specified step length along the reflection direction to continue the reflection detection of the reflection point to be processed. The next voxel to be processed is the voxel to be processed detected after the current voxel to be processed; the reflection direction is the direction determined based on the line of sight and the normal of the reflection point to be processed; the specified step length refers to the forward distance of the reflection detection, which can be positively correlated with the size of the voxel to be processed. Furthermore, meeting the reflection detection continuation condition includes one or two of the following: the reflection detection distance is less than or equal to the reflection detection distance threshold, and the number of reflection detections is less than or equal to the reflection detection count threshold; moreover, the reflection detection termination condition and the reflection detection continuation condition are relative.

[0143] For example, when the reflection detection termination condition is met (i.e., the reflection detection distance is greater than the reflection detection distance threshold), the reflection detection continuation condition is met (i.e., the reflection detection distance is less than or equal to the reflection detection distance threshold); when the reflection detection termination condition is met (i.e., the reflection detection count is greater than the reflection detection count threshold), the reflection detection continuation condition is met (i.e., the reflection detection count is less than or equal to the reflection detection distance threshold); when the reflection detection termination condition is met (i.e., the reflection detection distance is greater than the reflection detection distance threshold and the reflection detection count is greater than the reflection detection count threshold), the reflection detection continuation condition is met (i.e., the reflection detection distance is less than or equal to the reflection detection distance threshold and the reflection detection count is less than or equal to the reflection detection count threshold); when the reflection detection termination condition is met (i.e., the reflection detection distance is greater than the reflection detection distance threshold or the reflection detection count is greater than the reflection detection count threshold), the reflection detection continuation condition is met (i.e., the reflection detection distance is less than or equal to the reflection detection distance threshold and the reflection detection count is less than or equal to the reflection detection count threshold).

[0144] Step 1037: Determine the voxel detection result based on the next voxel to be processed.

[0145] In this embodiment of the application, after the reflection data processing device obtains the next voxel to be processed, it determines whether the next voxel to be processed is a reflected voxel, and then obtains a voxel detection result indicating whether a reflected voxel is detected.

[0146] It should be noted that the process by which the reflection data processing device determines the voxel detection result based on the next voxel to be processed is similar to the process by which it determines the voxel detection result based on the current voxel to be processed, and this embodiment of the application will not repeat the description of this process.

[0147] Step 104: Based on the voxel detection results, determine the reflection color of the reflection point to be processed.

[0148] In this embodiment, the reflection data processing device determines the reflection color of the reflection point to be processed based on the content represented by the voxel detection results. Here, the reflection color of the reflection point to be processed can be the color of the reflected voxel, a specified color (e.g., black), or the color of the pixel corresponding to the reflected voxel, specifically determined based on the voxel detection results.

[0149] See Figure 8 , Figure 8 This is a schematic diagram of the process for determining the reflected color provided in an embodiment of this application; as shown below. Figure 8 As shown in the embodiment of this application, step 104 can be implemented through steps 1041 and 1042; that is, the reflection data processing device determines the reflection color of the reflection point to be processed based on the voxel detection results, including steps 1041 and 1042. Each step will be described below.

[0150] Step 1041: When the voxel detection result indicates that no reflected voxel is detected, the specified color is determined as the reflection color of the reflection point to be processed.

[0151] In this embodiment of the application, when the voxel detection result indicates that no reflective voxel is detected, it indicates that the reflection point to be processed has no reflective information. Therefore, the reflection data processing device determines the specified color as the reflection color of the reflection point to be processed.

[0152] Step 1042: When the voxel detection result indicates that a reflected voxel has been detected, determine the reflection color of the reflection point to be processed based on the reflected voxel.

[0153] In the embodiments of this application, when the voxel detection result indicates that a reflected voxel has been detected, it means that the reflection data processing device can determine the reflection color of the reflection point to be processed based on the reflected voxel. For example, the voxel color of the reflected voxel can be determined as the reflection color of the reflection point to be processed, or the pixel color detected based on the reflected voxel can be determined as the reflection color of the reflection point to be processed.

[0154] See also Figure 8 In step 1042 of this application embodiment, the reflection data processing device determines the reflection color of the reflection point to be processed based on the reflected voxel, including steps 10421 and 10422. Each step will be described below.

[0155] Step 10421: Determine the reflection level based on the reflection parameters of the reflection point to be processed.

[0156] It should be noted that when the voxel detection result indicates that a reflected voxel has been detected, the reflection data processing device can directly determine the voxel color of the reflected voxel as the reflection color of the reflection point to be processed, or it can determine the reflection color of the reflection point to be processed based on the reflected voxel by judging the degree of material reflection of the reflection point to be processed.

[0157] Step 10422: When the reflection level is the first reflection level, the voxel color of the voxel to be reflected is determined as the reflection color of the reflection point to be processed.

[0158] It should be noted that when the reflection data processing device determines the reflection level to be the first reflection level, it indicates that the material of the reflection point to be processed has a low degree of reflection, and thus the voxel color of the voxel to be reflected is directly determined as the reflection color of the reflection point to be processed.

[0159] See also Figure 8Step 10421 is followed by step 10423; that is, after the reflection data processing device determines the reflection level based on the reflection parameters of the reflection point to be processed, the reflection data processing method further includes step 10423. Each step is explained below.

[0160] Step 10423: When the reflection level is the second reflection level, determine the voxel start point and voxel end point where the reflected light enters the voxel being reflected, determine the reflection detection range of the voxel start point and voxel end point in the screen space, perform reflection detection within the reflection detection range, obtain pixel detection results, and determine the reflection color of the reflection point to be processed based on the pixel detection results.

[0161] It should be noted that when the reflection data processing device determines the reflection level to be the second reflection level, it indicates that the material of the reflection point to be processed has a high degree of reflection. Therefore, the reflection data processing device continues to perform pixel-level reflection detection within the screen space corresponding to the reflected voxel to determine the reflection color of the reflection point to be processed based on the pixel detection results. The pixel detection result indicates whether a reflected pixel used to reflect information to the reflection point to be processed is detected. Therefore, the pixel detection result can indicate that a reflected pixel was detected or not. The reflection detection range is the pixel range obtained by converting the voxel's starting and ending points into screen space. Within the reflection detection range, the reflection data processing device performs reflection detection based on the pixel detection step size to detect whether there are pixels used to reflect information to the reflection point to be processed. The pixel to be reflected is the pixel used to reflect information to the reflection point to be processed.

[0162] In step 10423 of this embodiment, the reflection data processing device determines the reflection color of the reflection point to be processed based on the pixel detection result, including: when the pixel detection result indicates that no reflected pixel is detected, the reflection data processing device determines the voxel color of the reflected voxel as the reflection color of the reflection point to be processed; and when the voxel detection result indicates that a reflected pixel is detected, the reflection data processing device determines the pixel color of the reflected pixel as the reflection color of the reflection point to be processed.

[0163] Understandably, when the voxel detection result indicates that a reflective voxel has been detected, performing pixel-level reflection detection within the reflective voxel can improve the accuracy of reflection.

[0164] Step 105: Obtain the reflection color image corresponding to the 3D scene to be processed from the reflection color of the reflection point to be processed. Render the screen scene to be reflected based on the reflection color image to obtain the target virtual scene.

[0165] It should be noted that after the reflection data processing device obtains the reflection color of each reflection point in the 3D scene to be processed, it combines the reflection colors of each reflection point in the 3D scene to be processed to obtain a reflection color image corresponding to the 3D scene to be processed. This reflection color image is also the reflection color image of the screen scene to be reflected. Therefore, the reflection data processing device renders the screen scene to be reflected based on the reflection color image, and the obtained rendering result is the target virtual scene.

[0166] In this embodiment, the reflection data processing device can render the screen scene to be reflected using the reflection color image within its own device to present the target virtual scene; or it can send the reflection color image to other devices (such as rendering devices like terminals) to render the screen scene to be reflected using the reflection color image on other devices to present the target virtual scene.

[0167] Understandably, when performing reflection processing on a screen scene to be reflected, the screen scene to be reflected is first voxelized into a voxel scene to be processed. Then, reflection detection is performed on each reflection point corresponding to the screen scene to be reflected in the voxel scene to be processed, so as to obtain the voxel detection result of whether the reflected voxel is detected. Then, the reflection color determined by the voxel detection result is used to render the screen scene to be reflected. In other words, reflection detection is based on voxels, and the reflection color is determined based on the detection result of the reflected voxel. Since voxels are information of world space, the obtained reflection information includes not only screen space information but also world space information, thereby improving the reflection effect.

[0168] The following describes an exemplary application of the embodiments of this application in a real-world application scenario. This exemplary application describes the process of implementing screen-space reflection in the gaming field. It is readily apparent that the reflection data processing method provided by the embodiments of this application can implement screen-space reflection in any scene rendering domain, such as games or videos. Here, the gaming field is used as an example for illustration.

[0169] It should be noted that reflection processing is often performed when rendering game scenes in the gaming field. When the reflection data processing method provided in this application embodiment is applied to the gaming field, it can realize the reflection of both glossy materials and mirror objects.

[0170] See 9. Figure 9 This is an exemplary screen space reflection flowchart provided in an embodiment of this application; as shown... Figure 9 As shown, the exemplary screen space reflection process includes steps 201 to 212, and each step is described below.

[0171] Step 201: Reconstruct the 3D scene (referred to as the 3D scene to be processed) based on scene depth information (referred to as scene depth image) and camera matrix (referred to as spatial transformation parameters).

[0172] It should be noted that reconstructing a 3D scene means reconstructing the game scene in screen space into a 3D scene in order to determine the position and normal of the reflection point corresponding to each pixel in the game scene in world space (called the reflection point position and reflection point normal).

[0173] Step 202: Based on the position and normal of the reflection point, as well as the direction of the line of sight, calculate the direction of the reflected ray (called the reflection direction) and the starting point of the ray tracing (called the reflection detection starting point).

[0174] It should be noted that during forward rendering, the position and normal of the reflection point are obtained by reconstructing the 3D scene based on the scene depth information. However, during deferred rendering, the position and normal of the reflection point can be used directly.

[0175] Step 203: Perform ray tracing in the voxel field (referred to as the voxel scene to be processed) from the ray tracing starting point along the direction of the reflected light rays.

[0176] It should be noted that a voxel field is a voxelized representation of the game scene (referred to as the screen scene to be reflected). In a voxel field, each voxel's attributes include its SDF value (also known as distance value or voxel distance) relative to its surrounding scene object and its color (referred to as voxel color). The SDF value of voxels on the surface of the surrounding scene object is "+ / -0.5", the SDF value of voxels inside the surrounding scene object is negative (e.g., -1.5, -2.5, -3.5, etc.), and the SDF value of voxels outside the surrounding scene object is positive (e.g., 1.5, 2.5, 3.5, etc.).

[0177] For example, see Figure 10 , Figure 10 This is a schematic diagram of an exemplary voxel field provided in an embodiment of this application; as shown... Figure 10 As shown, voxelization is performed on game scene 10-1 to obtain voxel field 10-2.

[0178] Step 204: Determine if the distance value of the voxel corresponding to the current ray tracing point is less than or equal to 0. If yes, proceed to step 205; otherwise, proceed to step 206.

[0179] It should be noted that a distance value less than or equal to 0 indicates that the voxel is located on or inside the surface of the object in the scene; while a distance value greater than 0 indicates that the voxel is located outside the object in the scene, and it is determined that the reflected voxel was not hit.

[0180] Step 205: Determine whether the voxel corresponding to the current ray tracing point (referred to as the current voxel to be processed) is the first voxel (referred to as the first voxel to be processed). If yes, proceed to step 210; otherwise, proceed to step 208.

[0181] It should be noted that when the voxel corresponding to the current ray tracing point is the first voxel, the ray tracing point is within the scene object, thus it is determined that the reflected voxel was not hit. However, when the voxel corresponding to the current ray tracing point is a voxel after the first voxel, the ray tracing point is within the scene object being reflected, thus it is determined that the reflected voxel was hit.

[0182] Step 206: Determine whether the maximum number of ray tracing attempts has been reached or the maximum ray tracing distance has been exceeded (referred to as satisfying the reflection detection termination condition). If yes, proceed to step 210; otherwise, proceed to step 207.

[0183] It should be noted that reaching the maximum number of ray tracing attempts or exceeding the maximum ray tracing distance are the conditions for ending ray tracing. Therefore, when the maximum number of ray tracing attempts or the maximum ray tracing distance is reached or exceeded, the ray tracing process for that reflection point will end; otherwise, ray tracing will continue.

[0184] Step 207: Update the current ray tracing point by stepping a specified distance (called the specified step size) in the direction of the reflected ray. Then execute step 204.

[0185] It should be noted that by continuing to move forward along the direction of the reflected light, a new ray tracing point is determined to update the current ray tracing point and obtain a new current ray tracing point.

[0186] Step 208: Determine if the reflected voxel is hit.

[0187] It should be noted that hitting the reflected voxel indicates that the voxel used to reflect information has been detected.

[0188] Step 209: Use the voxel color of the voxel to be reflected as the reflection color. Then proceed to step 212.

[0189] It should be noted that since each voxel in the voxel field includes a corresponding voxel color, when it is determined that the voxel to be reflected is hit, the voxel color of the voxel to be reflected is directly used as the reflection color.

[0190] Step 210: Determine that the reflected voxel was not hit.

[0191] It should be noted that the failure to hit the reflected voxel indicates that no voxel used for reflecting information was detected.

[0192] Step 211: Set the specified color as the reflection color. Then proceed to step 212.

[0193] It should be noted that no voxels were detected for the purpose of providing reflection information, thus the specified color was used as the reflection color.

[0194] Step 212: Render the game scene based on reflected colors.

[0195] It should be noted that when implementing screen space reflection based on the process described in steps 201 to 212, see [reference needed]. Figure 11 , Figure 11 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 1 ;like Figure 11 As shown, in the rendered game scene 11-1, the virtual floor 11-11 reflects scene objects within a specified range around it, such as information about the virtual curtains 11-12 on the virtual wall. Figure 11 The dashed box in Figure 11-2 is shown.

[0196] See Figure 12 , Figure 12 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 2 ;like Figure 12 As shown, reflection processing is performed on game scene 12-1 to obtain the rendered target game scene 12-2; in the target game scene 12-2, the color of the clothing of the virtual character 12-21 is reflected on the virtual stone surface that is wet with water, and the reflection positions 12-22 to 12-25 are shown as an example.

[0197] It should be noted that the screen space reflection process described in steps 201 to 212 applies to reflective objects such as glossy materials with a roughness higher than (or equal to) a roughness threshold and a metallicity higher than a metallicity threshold. When performing reflection processing on specular materials with a roughness lower than a roughness threshold and a metallicity higher than a metallicity threshold, please refer to [reference needed]. Figure 13 , Figure 13 This is another exemplary screen space reflection flowchart provided in the embodiments of this application; as shown Figure 13 As shown, the exemplary screen space reflection process includes steps 301 to 314, and each step is described below.

[0198] Step 301: Reconstruct the 3D scene based on scene depth information and camera matrix.

[0199] It should be noted that the implementation of the process described in step 301 is similar to that described in step 201, and will not be repeated here in the embodiments of this application.

[0200] Step 302: Based on the position and normal of the reflection point, as well as the direction of the line of sight, calculate the direction of the reflected ray and the starting point of the ray tracing.

[0201] It should be noted that the implementation of the process described in step 302 is similar to that described in step 202, and will not be repeated here in the embodiments of this application.

[0202] Step 303: Perform ray tracing in the voxel field from the starting point of the ray tracing along the direction of the reflected light rays.

[0203] It should be noted that the implementation of the process described in step 303 is similar to that described in step 203, and will not be repeated here in the embodiments of this application.

[0204] Step 304: Determine if the distance value of the voxel corresponding to the current ray tracing point is less than or equal to 0. If yes, proceed to step 307; otherwise, proceed to step 305.

[0205] It should be noted that the implementation of the process described in step 304 is similar to that described in step 204, and will not be repeated here in the embodiments of this application.

[0206] Step 305: Determine whether the maximum number of ray tracing attempts or the maximum ray tracing distance has been reached. If yes, proceed to step 312; otherwise, proceed to step 306.

[0207] It should be noted that the implementation of the process described in step 305 is similar to that described in step 206, and will not be repeated here in the embodiments of this application.

[0208] Step 306: Update the current ray tracing point by stepping a specified distance in the direction of the reflected ray. Then proceed to step 304.

[0209] It should be noted that the implementation of the process described in step 306 is similar to that described in step 207, and will not be repeated here in the embodiments of this application.

[0210] Step 307: Determine if the reflected voxel is hit.

[0211] It should be noted that the implementation of the process described in step 307 is similar to that described in step 208, and will not be repeated here in the embodiments of this application.

[0212] Step 308: Perform pixel-based linear 2D ray tracing within the reflected voxel.

[0213] It should be noted that after hitting the reflected voxel, the position where the ray enters the reflected voxel and the position where it leaves the reflected voxel (called the voxel start point and voxel end point) are recorded. The position where the ray enters the reflected voxel is taken as the starting point of the ray tracing within the voxel, and the direction of the ray is taken as the direction of the ray tracing for Linear2D processing.

[0214] See Figure 14 , Figure 14 This is an exemplary schematic diagram of ray tracing within a voxel provided in an embodiment of this application; as shown... Figure 14 As shown, after determining that the virtual object's voxel 14-1 has been hit by the ray direction 14-3, the starting point for ray tracing within voxel 14-1 is taken as the position 14-2 where the ray enters the voxel, and Linear2D processing is performed with the ray direction 14-3 as the ray tracing direction. Here, rectangle 14-4 represents the screen.

[0215] Understandably, performing pixel-based linear 2D ray tracing within the reflected voxel can improve the uniformity of the tracing results.

[0216] Step 309: Determine whether the reflected pixel has been hit. If yes, proceed to step 310; otherwise, proceed to step 311.

[0217] It should be noted that hitting a reflected pixel indicates that a pixel used for reflection information has been detected, while missing a reflected pixel indicates that no pixel used for reflection information has been detected.

[0218] Step 310: Use the pixel color of the pixel to be reflected as the reflection color. Then proceed to step 314.

[0219] It should be noted that since each pixel includes a corresponding pixel color, when it is determined that a reflected pixel has been hit, the pixel color of the reflected pixel is directly used as the reflection color.

[0220] Step 311: Use the voxel color of the voxel to be reflected as the reflection color. Then proceed to step 314.

[0221] It should be noted that the implementation of the process described in step 311 is similar to that described in step 209, and will not be repeated here in the embodiments of this application.

[0222] Step 312: Determine that the reflected voxel was not hit.

[0223] It should be noted that the implementation of the process described in step 312 is similar to that described in step 210, and will not be repeated here in the embodiments of this application.

[0224] Step 313: Select the specified color as the reflection color.

[0225] It should be noted that the implementation of the process described in step 313 is similar to that described in step 211, and will not be repeated here in the embodiments of this application.

[0226] Step 314: Render the game scene based on reflected colors.

[0227] See Figure 15 , Figure 15 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 3 ;like Figure 15 As shown, in the rendered game scene 15-1, the color of the clothing of the virtual character 15-12 is reflected on the virtual floor 15-11.

[0228] See Figure 16 , Figure 16 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 4 ;like Figure 16 As shown, in the rendered game scene 16-1, the virtual objects reflected on the virtual floor 16-11 (e.g., virtual curtains, virtual vegetation) are blocky; while in the rendered game scene 16-2, the information reflected on the virtual floor 16-21 includes patterns of virtual objects (e.g., patterns of virtual curtains, patterns of virtual vegetation, etc.). The rendering of game scene 16-1 uses... Figure 9 The screen space reflection process shown is implemented; the rendering of game scene 16-2 is achieved using... Figure 13 The screen space reflection process shown is implemented in game scene 16-2, and the reflection accuracy is higher than that of game scene 16-1.

[0229] See Figure 17 , Figure 17 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 5 ;like Figure 17 As shown, in the rendered game scene 17-1, the virtual curtain reflected on the virtual floor 17-11 is blocky; while in the rendered game scene 17-2, the virtual curtain pattern is reflected on the virtual floor 17-21; and in the rendered game scene 17-3, the virtual curtain pattern is reflected on the virtual floor 17-31, and the edge information of the virtual curtain and the virtual floor is accurately reflected. The rendering of game scene 17-1 uses... Figure 9 The screen space reflection process shown is implemented; the rendering of game scene 17-2 is achieved using... Figure 13 The screen-space reflection process shown skips the processing of the first voxel; the rendering of game scene 17-3 uses... Figure 13 The screen space reflection process shown is implemented such that the reflection accuracy of game scene 17-2 is higher than that of game scene 17-1, while the reflection accuracy of game scene 17-3 is higher than that of game scene 17-2.

[0230] It should be noted that the reflection data processing device provided in this application embodiment is applicable to reflection processing scenarios, and can not only improve the accuracy and efficiency of reflection, but also improve the integrity of reflection.

[0231] For example, see Figure 18 , Figure 18 This is an exemplary illustration of screen space reflection results provided in an embodiment of this application. Figure 6 ;like Figure 18 As shown, in the rendered game scene 18-1, the virtual floor 18-11 can not only reflect screen information (such as virtual curtains), but also information outside the screen (such as the walls of virtual buildings, virtual sky, etc.), thus improving the integrity of the reflection.

[0232] It is understood that the embodiments of this application perform ray tracing in a voxel field based on timing and SDF values, which can quickly determine whether a reflected voxel has been hit, thereby improving reflection processing efficiency and thus improving the frame rate and rendering efficiency of the game. This makes it applicable to devices such as terminals, thereby expanding the applicability of reflection processing. Furthermore, during ray tracing in a voxel field, determining the reflection color based on the reflected voxel allows obtaining reflected information outside the screen, thereby improving reflection integrity. Moreover, when the reflective material is a reflective object such as a mirror, determining the reflected voxel first and then performing Linear2D processing on the reflected voxel can improve the accuracy and effect of reflection.

[0233] The following continues to describe the exemplary structure of the reflection data processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software modules stored in the reflection data processing device 455 in the memory 450 may include:

[0234] The scene reconstruction module 4551 is used to perform three-dimensional scene reconstruction on the scene of the screen to be reflected, so as to obtain the three-dimensional scene to be processed.

[0235] Voxel representation module 4552 is used to voxelize the screen scene to be reflected to obtain the voxel scene to be processed.

[0236] The reflection detection module 4553 is used to perform reflection detection on each reflection point in the three-dimensional scene to be processed in the voxel scene to be processed, and obtain voxel detection results. The voxel detection results indicate whether a voxel to be reflected toward the reflection point to be processed is detected.

[0237] Color determination module 4554 is used to determine the reflection color of the reflection point to be processed based on the voxel detection results;

[0238] The scene rendering module 4555 is used to obtain a reflection color image corresponding to the three-dimensional scene to be processed from the reflection color of the reflection point to be processed, and to render the screen scene to be reflected based on the reflection color image to obtain the target virtual scene.

[0239] In this embodiment of the application, the voxel representation module 4552 is further configured to voxelize the screen scene to be reflected to obtain an initial voxel scene; determine the target scene object to which each initial voxel in the initial voxel scene belongs; determine the voxel distance between the initial voxel and the target scene object; and combine the voxel distance and the voxel color of the initial voxel to determine the voxel to be processed, thereby obtaining the voxel scene to be processed including multiple voxels to be processed.

[0240] In this embodiment of the application, the reflection detection module 4553 is further configured to perform the following processing on each reflection point in the three-dimensional scene to be processed: determine the reflection direction by combining the reflection point position and reflection point normal of the reflection point and the viewing direction; and perform reflection detection in the voxel scene to be processed along the reflection direction with the reflection point position as the reflection detection starting point to obtain the voxel detection result.

[0241] In this embodiment of the application, the reflection detection module 4553 is further configured to perform reflection detection on each reflection point in the three-dimensional scene to be processed in the scene to be processed, and determine the current voxel to be processed corresponding to the current detection position; when the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object, the current voxel to be processed is determined as the reflected voxel; and the detected reflected voxel is determined as the voxel detection result.

[0242] In this embodiment, the reflection detection module 4553 is further configured to acquire reflection detection parameters when the current voxel distance indicates that the current voxel to be processed is located outside the target scene object to which it belongs. The reflection detection parameters include one or both of reflection detection distance and reflection detection count. When it is determined based on the reflection detection parameters that the reflection detection termination condition is met, the voxel that was not detected is determined as the voxel detection result. The reflection detection termination condition includes one or both of the following: the reflection detection distance is greater than the reflection detection distance threshold, and the reflection detection count is greater than the reflection detection count threshold.

[0243] In this embodiment of the application, the reflection detection module 4553 is further configured to advance a specified step length along the reflection direction to obtain the next voxel to be processed when it is determined based on the reflection detection parameters that the reflection detection continuation condition is met. The reflection detection continuation condition includes one or two of the following: the reflection detection distance is less than or equal to the reflection detection distance threshold, and the reflection detection count is less than or equal to the reflection detection count threshold; and the voxel detection result is determined based on the next voxel to be processed.

[0244] In this embodiment of the application, the reflection detection module 4553 is further configured to obtain the current voxel detection count when the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object to which it belongs; and to determine the current voxel to be processed as the voxel to be reflected when the current voxel detection count indicates that the current voxel to be processed is the voxel to be processed after the first voxel to be processed.

[0245] In this embodiment of the application, the reflection detection module 4553 is further configured to determine the undetected voxel as the voxel detection result when the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed.

[0246] In this embodiment, the reflection detection module 4553 is further configured to determine the reflection level based on the reflection parameters of the reflection point to be processed when the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed. The reflection parameters include metallicity and roughness. When the reflection level is a first reflection level, the voxel that was not detected is determined as the voxel detection result. The first reflection level refers to the metallicity being greater than a metallicity threshold and the roughness being greater than a roughness threshold. When the reflection level is a second reflection level, the first voxel to be processed is determined as the voxel to be reflected, and the voxel detection result that detected the voxel to be reflected is obtained. The second reflection level refers to the metallicity being greater than the metallicity threshold and the roughness being less than or equal to the roughness threshold.

[0247] In this embodiment of the application, the color determination module 4554 is further configured to determine the specified color as the reflection color of the reflection point to be processed when the voxel detection result indicates that the reflected voxel is not detected; and to determine the reflection color of the reflection point to be processed based on the reflected voxel when the voxel detection result indicates that the reflected voxel is detected.

[0248] In this embodiment of the application, the color determination module 4554 is further configured to determine the reflection level based on the reflection parameters of the reflection point to be processed; when the reflection level is the first reflection level, the voxel color of the reflected voxel is determined as the reflection color of the reflection point to be processed.

[0249] In this embodiment of the application, the color determination module 4554 is further configured to: determine the voxel start point and voxel end point where the reflected light enters the reflected voxel when the reflection level is the second reflection level; determine the reflection detection range of the voxel start point and the voxel end point in the screen space; perform reflection detection within the reflection detection range to obtain a pixel detection result, wherein the pixel detection result indicates whether a reflected pixel for reflecting to the reflection point to be processed is detected; and determine the reflection color of the reflection point to be processed based on the pixel detection result.

[0250] In this embodiment of the application, the color determination module 4554 is further configured to determine the voxel color of the reflected voxel as the reflection color of the reflection point to be processed when the pixel detection result indicates that the reflected pixel is not detected; and to determine the pixel color of the reflected pixel as the reflection color of the reflection point to be processed when the voxel detection result indicates that the reflected pixel is detected.

[0251] In this embodiment of the application, the scene reconstruction module 4551 is further configured to perform the following processing on each screen pixel in the screen scene to be reflected: combining the scene depth image and spatial transformation parameters corresponding to the screen scene to be reflected, determining the reflection point position and reflection point normal of the screen pixel in world space; combining the reflection point position and the reflection point normal, determining the reflection point to be processed, and obtaining the three-dimensional scene to be processed including multiple reflection points to be processed.

[0252] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of a reflection data processing device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the reflection data processing device to perform the reflection data processing method described above in this application.

[0253] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the reflection data processing method provided in this application. For example, ... Figure 5 The reflection data processing method is shown.

[0254] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0255] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0256] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0257] As an example, computer-executable instructions can be deployed to execute on a single electronic device (in which case, this single electronic device is a reflection data processing device), or to execute on multiple electronic devices located at one location (in which case, the multiple electronic devices located at one location are reflection data processing devices), or to execute on multiple electronic devices distributed across multiple locations and interconnected via a communication network (in which case, the multiple electronic devices distributed across multiple locations and interconnected via a communication network are reflection data processing devices).

[0258] It is understood that in the embodiments of this application, data related to the scene of the screen to be reflected is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0259] In summary, when performing reflection processing based on a screen scene to be reflected, the embodiments of this application first convert the screen scene to be reflected into a voxel scene to be processed. Then, reflection detection is performed on each reflection point corresponding to the screen scene to be reflected in the voxel scene to be processed to obtain the voxel detection result of whether the reflected voxel is detected. Finally, the screen scene to be reflected is rendered based on the reflection color determined by the voxel detection result. That is, reflection detection is voxel-based, and the reflection color is determined based on the detection result of the reflected voxel. Since voxels represent world space information, the obtained reflection information includes not only screen space information but also world space information, thereby improving the reflection effect. Furthermore, voxel-based reflection detection, since the voxel to be processed includes voxel distance, can quickly determine whether the reflected voxel is detected, thereby improving reflection efficiency. This makes the reflection data processing method provided by the embodiments of this application applicable to various devices such as terminals, improving the applicability of reflection processing. Also, for the second reflection level, after detecting the voxel to be reflected, pixel-based linear ray tracing within the voxel to be reflected can improve the accuracy of reflection.

[0260] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A reflection data processing method, characterized in that, The method includes: The 3D scene of the screen to be reflected is reconstructed to obtain the 3D scene to be processed. The screen scene to be reflected is voxelized to obtain the voxel scene to be processed; In the voxel scene to be processed, reflection detection is performed on each reflection point in the three-dimensional scene to be processed to obtain voxel detection results. The voxel detection results indicate whether a voxel used to reflect to the reflection point to be processed is detected. Based on the voxel detection results, the reflection color of the reflection point to be processed is determined; The reflection color image corresponding to the three-dimensional scene to be processed is obtained from the reflection color of the reflection point to be processed. The screen scene to be reflected is rendered based on the reflection color image to obtain the target virtual scene.

2. The method according to claim 1, characterized in that, The process of voxelizing the screen scene to be reflected to obtain the voxel scene to be processed includes: The scene to be reflected on the screen is voxelized to obtain an initial voxel scene; Determine the target scene object to which each initial voxel in the initial voxel scene belongs; Determine the voxel distance between the initial voxel and the target scene object; By combining the voxel distance and the voxel color of the initial voxel, the voxel to be processed is determined, resulting in a voxel scene including multiple voxels to be processed.

3. The method according to claim 1 or 2, characterized in that, The process involves performing reflection detection on each reflection point in the three-dimensional scene to be processed, obtaining voxel detection results, including: The following processing is performed on each of the reflection points in the three-dimensional scene to be processed: The reflection direction is determined by combining the reflection point position and reflection point normal of the reflection point to be processed, as well as the line of sight. Using the reflection point location as the starting point for reflection detection, reflection detection is performed along the reflection direction in the voxel scene to be processed, and the voxel detection result is obtained.

4. The method according to claim 1 or 2, characterized in that, The process involves performing reflection detection on each reflection point in the three-dimensional scene to be processed, obtaining voxel detection results, including: In the voxel scene to be processed, reflection detection is performed on each reflection point in the three-dimensional scene to be processed to determine the current voxel to be processed corresponding to the current detection position; When the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object, the current voxel to be processed is determined as the reflected voxel; The detected reflective voxel is determined as the voxel detection result.

5. The method according to claim 4, characterized in that, After determining the current voxel to be processed corresponding to the current detection position, the method further includes: When the current voxel distance indicates that the current voxel to be processed is located outside the target scene object to which it belongs, the reflection detection parameters are obtained. The reflection detection parameters include one or both of the reflection detection distance and the number of reflection detections. When the reflection detection end condition is determined based on the reflection detection parameters, the absence of detected voxel is determined as the voxel detection result. The reflection detection end condition includes one or two of the following: the reflection detection distance is greater than the reflection detection distance threshold, and the number of reflection detections is greater than the reflection detection number threshold.

6. The method according to claim 5, characterized in that, After obtaining the reflection detection parameters, the method further includes: When it is determined based on the reflection detection parameters that the reflection detection continuation condition is met, advance a specified step length along the reflection direction to obtain the next voxel to be processed. The reflection detection continuation condition includes one or two of the following: the reflection detection distance is less than or equal to the reflection detection distance threshold, and the number of reflection detections is less than or equal to the reflection detection number threshold. The voxel detection result is determined based on the next voxel to be processed.

7. The method according to claim 4, characterized in that, The step of determining the current voxel to be processed as the reflected voxel when the current voxel distance indicates that the current voxel to be processed is located on the surface or inside the target scene object includes: When the current voxel distance corresponding to the current voxel to be processed indicates that the current voxel to be processed is located on the surface or inside the target scene object to which it belongs, the current voxel detection count is obtained; When the current voxel detection count indicates that the current voxel to be processed is the voxel to be processed after the first voxel to be processed, the current voxel to be processed is determined as the reflected voxel.

8. The method according to claim 7, characterized in that, After obtaining the current number of voxel detections, the method further includes: When the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed, the voxel that was not detected is determined as the voxel detection result.

9. The method according to claim 7, characterized in that, After obtaining the current number of voxel detections, the method further includes: When the current voxel detection count indicates that the current voxel to be processed is the first voxel to be processed, the reflection level is determined based on the reflection parameters of the reflection point to be processed, the reflection parameters including metallicity and roughness; When the reflection level is the first reflection level, the failure to detect the reflected voxel is determined as the voxel detection result. The first reflection level means that the metallicity is greater than the metallicity threshold and the roughness is greater than the roughness threshold. When the reflection level is the second reflection level, the first voxel to be processed is determined as the reflected voxel, and the voxel detection result of the reflected voxel is obtained. The second reflection level refers to the metallicity being greater than the metallicity threshold and the roughness being less than or equal to the roughness threshold.

10. The method according to claim 1 or 2, characterized in that, Determining the reflection color of the reflection point to be processed based on the voxel detection results includes: When the voxel detection result indicates that the reflected voxel is not detected, the specified color is determined as the reflection color of the reflection point to be processed; When the voxel detection result indicates that the reflected voxel has been detected, the reflection color of the reflection point to be processed is determined based on the reflected voxel.

11. The method according to claim 10, characterized in that, Determining the reflected color of the reflection point to be processed based on the reflected voxel includes: The reflection level is determined based on the reflection parameters of the reflection point to be processed; When the reflection level is the first reflection level, the voxel color of the reflected voxel is determined as the reflection color of the reflection point to be processed.

12. The method according to claim 11, characterized in that, After determining the reflection level based on the reflection parameters of the reflection point to be processed, the method further includes: When the reflection level is the second reflection level, the voxel initiation and voxel end points into which the reflected light enters the voxel are determined; Determine the reflection detection range of the voxel start point and the voxel end point in screen space; Reflection detection is performed within the reflection detection range to obtain pixel detection results, which indicate whether a reflected pixel used to reflect to the reflection point to be processed is detected. Based on the pixel detection results, the reflected color of the reflection point to be processed is determined.

13. The method according to claim 12, characterized in that, Determining the reflection color of the reflection point to be processed based on the pixel detection results includes: When the pixel detection result indicates that the reflected pixel is not detected, the voxel color of the reflected voxel is determined as the reflection color of the reflection point to be processed; When the voxel detection result indicates that the reflected pixel has been detected, the pixel color of the reflected pixel is determined as the reflection color of the reflection point to be processed.

14. The method according to claim 1 or 2, characterized in that, The process of reconstructing a 3D scene of the screen to be reflected to obtain the 3D scene to be processed includes: For each pixel of the screen to be reflected in the scene, the following processing is performed: By combining the scene depth image and spatial transformation parameters corresponding to the scene to be reflected, the reflection point position and reflection point normal of the pixel of the screen to be reflected in world space are determined; By combining the location of the reflection point and the normal of the reflection point, the reflection point to be processed is determined, and the three-dimensional scene to be processed, including multiple reflection points to be processed, is obtained.

15. A reflection data processing device, characterized in that, The reflection data processing device includes: The scene reconstruction module is used to reconstruct the 3D scene of the screen to be reflected, and obtain the 3D scene to be processed. A voxel representation module is used to voxelize the screen scene to be reflected to obtain a voxel scene to be processed. The reflection detection module is used to perform reflection detection on each reflection point in the three-dimensional scene to be processed in the scene to be processed, and obtain a voxel detection result, wherein the voxel detection result indicates whether a voxel to be reflected toward the reflection point to be processed is detected. The color determination module is used to determine the reflection color of the reflection point to be processed based on the voxel detection results. The scene rendering module is used to obtain a reflection color image corresponding to the three-dimensional scene to be processed from the reflection color of the reflection point to be processed, and to render the screen scene to be reflected based on the reflection color image to obtain the target virtual scene.

16. An electronic device for reflective data processing, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the reflection data processing method according to any one of claims 1 to 14.

17. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the reflection data processing method according to any one of claims 1 to 14 is implemented.

18. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the reflection data processing method according to any one of claims 1 to 14 is implemented.

Images