Image rendering method, apparatus, device, storage medium and program product

Abstract

The application provides an image rendering method and device, electronic equipment, computer readable storage medium and computer program product, comprising: based on the position of a first virtual object in a virtual scene and the perspective of a virtual camera, performing parallax correction on a reflection probe arranged in the virtual scene; through the reflection probe after parallax correction, collecting illumination information of reflected light of the first virtual object; obtaining a texture image of the first virtual object, and based on the texture image and the illumination information, determining a color value of an image pixel point; based on the color value of the image pixel point, rendering to obtain a reflection image of the first virtual object. Through the application, the rendering effect of the reflection image can be improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to an image rendering method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In computer graphics, common reflection schemes include probe-based and ray-based schemes. Probe-based schemes typically use a probe to collect lighting information based on the reflection direction. However, this can lead to errors in the sampling location, resulting in poor rendering of the reflected image. Ray-based schemes obtain the accurate hit point of the reflected ray in world space by emitting a ray, and then sample the corresponding lighting information from the world space lighting storage structure. However, the world space lighting storage itself has a significant performance overhead, so its accuracy cannot be very high. This results in poor rendering even when the sampling location is correct, leading to poor overall rendering of the reflected image. Summary of the Invention

[0003] This application provides an image rendering method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve the rendering effect of reflected images.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an image rendering method, including:

[0006] Based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, parallax correction is performed on the reflection probes set in the virtual scene;

[0007] The illumination information of the reflected light from the first virtual object is collected using the parallax-corrected reflection probe.

[0008] Obtain the texture image of the first virtual object, and determine the color value of the image pixel based on the texture image and the lighting information;

[0009] Based on the color values ​​of the image pixels, the reflection image of the first virtual object is rendered.

[0010] This application provides an image rendering apparatus, including:

[0011] The correction module is used to perform parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera.

[0012] The acquisition module is used to acquire the illumination information of the reflected light from the first virtual object through the parallax-corrected reflection probe;

[0013] The determining module is used to acquire the texture image of the first virtual object and determine the color value of the image pixel based on the texture image and the lighting information;

[0014] The rendering module is used to render the reflection image of the first virtual object based on the color values ​​of the image pixels.

[0015] In the above scheme, the correction module is further configured to determine the intersection point of the reflected light from the first virtual object and the second virtual object in the virtual scene based on the position of the first virtual object and the viewpoint of the virtual camera; determine a first direction from the position of the reflection probe to the intersection point, and adjust the direction of the illumination information of the reflected light from the first virtual object collected through the reflection probe to the first direction.

[0016] In the above scheme, the correction module is further used to construct the spatial coordinate system of the virtual scene, and determine the first coordinate of the reflection probe in the spatial coordinate system; obtain the second coordinate of the intersection point in the spatial coordinate system; subtract the first coordinate from the second coordinate to obtain the vector of the reflection probe pointing to the intersection point, and determine the direction indicated by the vector as the first direction.

[0017] In the above scheme, the device further includes an acquisition module, which is used to acquire the position of the virtual camera in the virtual scene; and generate the reflection probe based on the position of the virtual camera.

[0018] In the above scheme, the device further includes a first update module, which is used to detect the position of the virtual camera; in response to the virtual camera's position changing from a first position to a second position, the position of the reflection probe is updated based on the second position of the virtual camera.

[0019] In the above scheme, the first update module is further configured to determine the difference between the first position and the second position in response to the virtual camera's position changing from the first position to the second position; and update the position of the reflection probe in response to the difference reaching a difference threshold.

[0020] In the above scheme, the virtual scene is divided into at least two grid cells, and the first update module is also used to detect the position of the virtual camera; in response to the virtual camera changing from being in the first grid cell to being in the second grid cell, the position of the reflection probe is updated based on the second grid cell.

[0021] In the above scheme, the virtual scene is divided into at least two grid units. The acquisition module is further used to acquire the grid unit in which the virtual camera is located in the virtual scene. The first update module is further used to acquire the duration for which the virtual camera is in the grid unit. When the duration reaches the target duration, the position of the reflection probe is updated.

[0022] In the above scheme, the parallax-corrected reflection probe records the illumination information of virtual objects within the target range in the virtual scene; the device further includes a second update module, which is used to update the illumination information collected by the reflection probe when the update conditions of the reflection probe are met, to obtain updated illumination information; wherein, the update conditions include at least one of the following: the relative positional relationship between the parallax-corrected reflection probe and the virtual objects within the target range in the virtual scene changes; the light at the virtual objects within the target range in the virtual scene changes; the acquisition module is further used to obtain the illumination information of the reflected light of the first virtual object from the updated illumination information.

[0023] In the above scheme, the lighting information includes color information obtained by coloring the virtual objects within the target range based on the texture information of the virtual objects within the target range; the second update module is further configured to update the texture information corresponding to the lighting information to obtain new texture information when the update condition is that the relative position relationship changes; update the color information based on the new texture information to obtain new color information, and use the new color information as the updated lighting information.

[0024] In the above scheme, the lighting information includes color information obtained by coloring the virtual objects within the target range based on the texture information of the virtual objects within the target range; the second update module is further configured to obtain the texture information corresponding to the lighting information when the update condition is that the light changes; update the color information based on the texture information to obtain new color information, and use the new color information as the updated lighting information.

[0025] In the above scheme, the second update module is further configured to obtain the texture image of the virtual object within the target range in the virtual scene based on the texture information, and determine the color value of each pixel in the texture image based on the texture image and the light at the virtual object; and use the color value of each pixel as the new color information.

[0026] In the above scheme, the shape of the reflection probe is a polyhedron, and different faces of the polyhedron record local lighting information in the virtual scene. The local lighting information recorded by different faces is different, and the lighting information includes the local lighting information recorded by each face. The second update module is further used to determine the update cycle of the reflection probe. The update cycle is used to indicate the minimum update interval between any two faces of the polyhedron. Based on the update cycle, the local lighting information recorded by each face of the polyhedron is updated sequentially to obtain the updated lighting information.

[0027] In the above scheme, the device further includes a completion module, which is used to obtain the acquisition direction of the reflection probe after parallax correction for collecting illumination information at the intersection point corresponding to the reflected light; wherein, the intersection point corresponding to the reflection probe is the intersection position of the reflected light of the first virtual object and the second virtual object in the virtual scene; if there is an obstruction in the acquisition direction, the illumination information collected by the parallax-corrected reflection probe is completed; the acquisition module is also used to obtain the illumination information of the reflected light obtained through completion.

[0028] In the above scheme, the device further includes a second determining module, which is used to construct a spatial coordinate system of the virtual scene, and in the spatial coordinate system, determine the depth of the reflective probe in the acquisition direction; obtain the third coordinate of the reflective probe in the spatial coordinate system, and obtain the fourth coordinate of the intersection point in the spatial coordinate system; determine the distance between the reflective probe and the intersection point based on the third coordinate and the fourth coordinate; when the distance is greater than the depth of the reflective probe in the acquisition direction, determine that there is an obstruction to the reflective probe in the acquisition direction.

[0029] In the above scheme, the completion module is further used to adjust the position of the reflection probe to the target position; wherein, the reflection probe at the target position has no obstruction in the acquisition direction for the intersection point, and the target position is used to indicate the position where the reflection probe can acquire the illumination information for the intersection point; and the illumination information of the reflected light of the first virtual object acquired by the reflection probe at the target position is obtained.

[0030] In the above scheme, the completion module is further used to mark the intersection point to obtain a mark for indicating the intersection point; obtain the lighting information library of the virtual scene, the lighting information library is used to store the lighting information in the virtual scene; and sample the lighting information at the intersection point from the lighting information library based on the mark.

[0031] This application provides an electronic device, including:

[0032] Memory is used to store executable instructions or computer programs.

[0033] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the image rendering method provided in the embodiments of this application.

[0034] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the image rendering method provided in this application.

[0035] This application provides a computer program product comprising computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the image rendering method provided in this application.

[0036] The embodiments of this application have the following beneficial effects:

[0037] Based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, parallax correction is applied to the reflection probes set in the virtual scene. The reflected light information of the first virtual object is then collected through the parallax-corrected reflection probes. Based on the acquired texture image of the first virtual object and the collected lighting information, the color values ​​of the image pixels are determined. Based on the color values ​​of the image pixels, the reflected image of the first virtual object is rendered. Thus, compared to reflection schemes based on reflection probes, this application, by correcting the parallax of the reflection probes, corrects the position of the lighting information collection, improving the accuracy of the collected lighting information and thus improving the rendering effect of the reflected image. Compared to reflection schemes based on emitted rays, it improves the precision of the collected lighting information, thereby improving the rendering effect of the reflected image. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the architecture of the image rendering system 100 provided in an embodiment of this application;

[0039] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0040] Figure 3 This is a schematic flowchart of the image rendering method provided in the embodiments of this application;

[0041] Figure 4This is a schematic diagram of the viewpoint of the virtual camera provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram showing how the first virtual object, as provided in this embodiment, reflects the line-of-sight light emitted by the virtual camera;

[0043] Figure 6 This is a schematic diagram of the intersection points of the reflected rays of the virtual object provided in the embodiments of this application;

[0044] Figure 7 This is a schematic diagram of the first direction provided in the embodiments of this application;

[0045] Figure 8 This is a schematic diagram of the reflective probe provided in an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of at least two mesh cells provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram illustrating the process of updating the position of the reflective probe according to an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the process for updating the lighting information of a virtual scene collected by a reflection probe, provided in an embodiment of this application.

[0049] Figure 12 This is a schematic diagram of the process for completing the lighting information of a virtual scene collected by a reflection probe, provided in an embodiment of this application.

[0050] Figure 13 This is a schematic diagram of the obstruction provided in the embodiments of this application;

[0051] Figure 14 This is a schematic diagram of the target location provided in the embodiments of this application;

[0052] Figure 15 This is a schematic diagram of a reflection scheme based on a reflection probe provided in an embodiment of this application;

[0053] Figure 16 This is an effect diagram of the reflection scheme based on the reflection probe provided in the embodiments of this application;

[0054] Figure 17 This is a schematic diagram of a reflection scheme based on an emitted ray provided in an embodiment of this application;

[0055] Figure 18 This is an effect diagram of the reflection scheme based on the emitted ray provided in the embodiments of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0058] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0060] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0061] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0062] 2) Client, also known as user terminal, refers to the program that provides local services to users in contrast to the server. Except for some applications that can only run locally, it is generally installed on the terminal and needs to work in conjunction with the server. That is, there needs to be a corresponding server and service program on the network to provide the corresponding services. Thus, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application, such as virtual scene clients (such as game clients) and video clients.

[0063] 3) Artificial Intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0064] 4) A virtual scene is a virtual scene displayed (or provided) by an application when it runs on a terminal. This virtual scene can be a simulation of the real world, a semi-simulated virtual environment, or a purely fictional virtual environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move within this virtual scene.

[0065] 5) Virtual light source, a type of graphic element that provides lighting effects in virtual scenes, is obtained by simulating real light sources in virtual scenes.

[0066] 6) Virtual camera, a component that simulates the function of a real camera. It determines the viewing angle of a virtual scene by simulating the shooting function of a real camera, and defines the position and orientation of the virtual object controlled by the user in the virtual environment. In other words, it is used to determine the scene content of the virtual scene observed by the virtual object controlled by the user.

[0067] 7) Reflection probes are used to capture lighting information from the surrounding environment. Specifically, reflection probes are placed in the 3D scene to capture lighting information at specific locations and generate environment maps. These maps are typically stored as cubemaps, containing six faces, each indicating the lighting information captured in one direction. Thus, during rendering, object surfaces use these environment maps to simulate reflection effects.

[0068] 8) Cube Map: Used to store lighting information in all directions in a virtual scene. It consists of six squares, each corresponding to one of the six faces of a cube (front, back, left, right, top, and bottom). Different faces store lighting information in different directions. During the rendering process, the reflection and refraction of an object's surface can be simulated by looking up the lighting information in a specific direction stored in the cube map.

[0069] 9) World Space Lighting Storage Structure, a lighting information database used to store lighting information in the world space, including lighting information not in the screen space.

[0070] 10) The world coordinate system defines the position and orientation of all objects in a 3D scene. It serves as a global reference frame, providing a common positioning benchmark for all objects in the 3D scene. The world coordinate system is a right-handed coordinate system, typically composed of the following key elements: the origin, the z-axis (vertical axis), the x-axis (horizontal axis), and the y-axis (vertical axis). The origin is the reference point in the world coordinate system, with coordinates (0,0,0). The position of all objects is defined relative to this origin. The x-axis is a horizontal axis, usually pointing to the right or left (the specific direction depends on the definition, but is usually to the right), used to represent the horizontal position of an object. The y-axis is also a horizontal axis, usually pointing forward or backward (the specific direction depends on the definition, but is usually forward), used to represent the forward / backward position of an object. The z-axis is the vertical axis, usually pointing upward. It is used to represent the vertical position of an object.

[0071] 11) The camera coordinate system is a reference system used to describe the position and orientation of the camera in three-dimensional space. It is defined from the camera's perspective and is used to simulate the viewpoint and field of view when the camera observes a three-dimensional scene. The camera coordinate system typically consists of the following key elements: origin, z-axis (vertical axis), x-axis (horizontal axis), and y-axis (vertical axis). The origin of the camera coordinate system is the center of the camera itself, usually located at the focal point of the camera lens. The z-axis of the camera coordinate system points forward of the camera, i.e., the camera's orientation, and is usually aligned with the camera's optical axis. In perspective projection, the z-axis determines the direction of the line of sight. The x-axis of the camera coordinate system usually points to the right of the camera and is perpendicular to the z-axis, forming a right-handed coordinate system (in some cases, such as in OpenGL, a left-handed coordinate system may be used). The y-axis of the camera coordinate system usually points upward of the camera and is perpendicular to both the x-axis and z-axis, completing the definition of a right-handed coordinate system.

[0072] 12) Screen coordinate system, a two-dimensional reference system that describes the position of touch points (such as cursors) on a physical device (such as a monitor); wherein the origin of the screen coordinate system is located at the upper left corner of the screen, the X-axis is positive to the right and the Y-axis is positive downward, the unit of measurement is pixels, and it is used to locate points on the screen corresponding to the terminal, such as the position of a mouse click, the position of a window, etc.

[0073] See Figure 1 , Figure 1This is a schematic diagram of the architecture of the image rendering system 100 provided in the embodiments of this application. The terminal (terminal 400 is shown as an example) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using wireless or wired links.

[0074] The server 200 is used to: perform parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera; collect the illumination information of the reflected light of the first virtual object through the parallax-corrected reflection probes; obtain the texture image of the first virtual object, and determine the color value of the image pixels based on the texture image and illumination information; render the reflection image of the first virtual object based on the color value of the image pixels; and send the reflection image of the first virtual object to the terminal 400.

[0075] Terminal 400 is used to receive and display the reflection image of the first virtual object.

[0076] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, set-top box, smart voice interaction device, smart home appliance, virtual reality device, vehicle terminal, aircraft, portable music player, personal digital assistant, dedicated messaging device, portable gaming device, smart speaker, and smartwatch, but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0077] The electronic device implementing the image rendering method provided in the embodiments of this application will now be described. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a server or a terminal. The electronic device is used as an example. Figure 1 Taking the server shown as an example, Figure 2The illustrated electronic device includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0078] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0079] User interface 430 includes one or more output devices 431 that enable the display of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0080] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0081] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0082] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0083] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0084] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0085] Presentation module 453 is configured to enable the display of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0086] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0087] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An image rendering apparatus 455 stored in memory 450 is shown. This apparatus can be software in the form of programs and plugins, and includes the following software modules: a correction module 4551, an acquisition module 4552, a determination module 4553, and a rendering module 4554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0088] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the image rendering apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image rendering method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0089] In some embodiments, the terminal or server can implement the image rendering method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a local client, i.e., a program that needs to be installed in the operating system to run, such as an instant messaging APP or a web browser APP; it can also be a mini-program, i.e., a program that only needs to be downloaded into a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of client, module, or plugin.

[0090] Based on the foregoing description of the image rendering system and electronic device provided in the embodiments of this application, the image rendering method provided in the embodiments of this application will be described below. In actual implementation, the image rendering method provided in the embodiments of this application can be implemented by a terminal or a server alone, or by a terminal and a server working together, so that... Figure 1 The following description uses the example of server 200 executing the image rendering method provided in this embodiment of the application independently. See also... Figure 3 , Figure 3 This is a schematic flowchart of the image rendering method provided in the embodiments of this application. Next, it will be combined with... Figure 3 The steps shown are explained.

[0091] Step 101: The server performs parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera.

[0092] In practical implementation, the virtual scene can be a virtual game scene or an audio / video live streaming scene, etc. The first virtual object in the virtual scene refers to an object with high reflectivity in the virtual scene. High reflectivity refers to an object whose reflectivity reaches the target reflectivity, such as a mirror, flowing water, glass, or metal. Before the server performs parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the virtual camera's perspective, the server first obtains an image rendering request sent by the terminal. The image rendering request is used to request the reflection image of the first virtual object in the virtual scene. Specifically, taking a virtual game scene as an example, the terminal first displays the virtual scene. When the view of the player character controlled by the terminal turns to the virtual object with high reflectivity, the terminal sends an image rendering request to the server. Then, in response to the image rendering request, the server performs parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the virtual camera's perspective.

[0093] It should be noted that before performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, the server first determines the virtual camera in the virtual scene. The virtual camera acts as the viewpoint here, and the viewpoint of the virtual camera is also the field of view of the virtual camera, which is the observation range. The virtual scene is displayed based on the content of the virtual scene within the viewpoint of the virtual camera. In other words, the screen used to present the virtual scene will only display the content of the virtual scene within the viewpoint of the virtual camera.

[0094] It should be noted that the virtual camera emits line-of-sight rays in the virtual scene to collect lighting information. Specifically, starting from the virtual camera's position, a ray is emitted to each location within the virtual scene's field of view, and the path of this ray is determined. When the ray intersects an object, the color at that location is calculated based on the object's material and lighting conditions, and this color is assigned to the corresponding pixel. As mentioned earlier, the screen used to present the virtual scene displays the content of the virtual scene within the virtual camera's viewpoint. Therefore, when displaying the content of the virtual scene within the virtual camera's viewpoint on the screen, the color value of each pixel within the virtual camera's viewpoint on the screen is first determined. Based on the color value of each pixel, the content of the virtual scene within the virtual camera's viewpoint can be displayed on the screen.

[0095] It's important to clarify that the gaze light emitted by the virtual camera in a virtual scene can be understood as follows: outside the virtual scene, there can exist a virtual light source to illuminate it. The virtual light emitted by this source illuminates the virtual scene, generating lighting information. This lighting information is used to indicate the color information at the corresponding location when the virtual light shines upon it—that is, the color value of each pixel at that location. Therefore, when the virtual light from the virtual source illuminates the virtual scene, it is reflected at the corresponding location and then back to the virtual camera, allowing the virtual camera to "see" and collect the lighting information at that location, thus displaying it on the screen. The gaze light emitted by the virtual camera can be understood as light rays traveling in the opposite direction to the reflected virtual light. In other words, the gaze light emitted by the virtual camera and the reflected virtual light from the virtual source are the same light ray, just in different directions. The lighting information corresponding to the gaze light emitted by the virtual camera and the reflected virtual light from the virtual source is the same lighting information at the same location.

[0096] For example, see Figure 4 , Figure 4 This is a schematic diagram of the viewpoint of the virtual camera provided in the embodiments of this application, based on Figure 4 The range between 401 and 402 is the virtual camera's field of view, i.e., the observation range, and 403 is the line of sight emitted by the virtual camera.

[0097] In actual implementation, the content of the virtual scene within the virtual camera's viewpoint will change. For example, when a user is playing a game, they can change the content of the presented virtual scene by adjusting the viewpoint. Thus, the process of parallax correction for the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera is to obtain the position of the first virtual object in the virtual scene. Based on the position of the first virtual object, when it is determined that there is a first virtual object within the viewpoint of the virtual camera, parallax correction is performed on the reflection probes set in the virtual scene.

[0098] It should be noted that, because the first virtual object has reflective capabilities, it will reflect the line-of-sight light emitted by the virtual camera. For example, see... Figure 5 , Figure 5 This is a schematic diagram illustrating how a first virtual object reflects the line-of-sight light emitted by a virtual camera, based on an embodiment of this application. Figure 5 When the line of sight emitted by the virtual camera indicated by 501 is directed toward the first virtual object, it is reflected at the position indicated by the dashed box 502, thus forming the reflected line of sight indicated by 503.

[0099] Therefore, after the first virtual object reflects the line of sight light emitted by the virtual camera, the reflected light obtained by the first virtual object will also illuminate other positions in the virtual scene. That is, the reflected light also has a hit position in the virtual scene. Here, the hit position is the position illuminated by the reflected light, thereby determining the hit position of the reflected light in the virtual scene, and based on the hit position, parallax correction is performed on the reflection probe.

[0100] Meanwhile, as mentioned earlier, outside the virtual scene, there exists a virtual light source used to illuminate the virtual scene, and the reflection probe is used to collect the lighting information in the virtual scene, that is, the lighting information generated when the virtual light emitted by the virtual light source illuminates the virtual scene; where the reflection probe can collect the lighting information generated when the virtual light illuminates the virtual scene means that when the virtual light illuminates the virtual scene, it is reflected by virtual objects in the virtual scene to the reflection probe, so that the reflection probe can "see" the lighting information of the virtual objects, that is, it can collect the lighting information in the virtual scene.

[0101] Therefore, parallax correction of the reflection probe, that is, correction of the direction in which the reflection probe collects lighting information in the virtual scene, and thus the process of parallax correction of the reflection probe set in the virtual scene based on the hit position, can be the process of parallax correction of the reflection probe set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera. That is, based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, the intersection point of the reflected light of the first virtual object and the second virtual object in the virtual scene is determined; the first direction from the position of the reflection probe to the intersection point is determined, and the direction of collecting the lighting information of the reflected light of the first virtual object through the reflection probe is adjusted to the first direction.

[0102] It should be noted that the second virtual object here refers to any virtual object in the virtual scene other than the first virtual object. The intersection of the reflected light from the first virtual object and the second virtual object in the virtual scene is the hit position mentioned above. The hit position can be a point or a region, and this embodiment does not limit this. The size of the hit position is related to the illumination range of the reflected light. For example, when the illumination range of the reflected light is a pixel, the hit position is a point; when the illumination range of the reflected light is a region, the hit position is a region.

[0103] For example, see Figure 6 , Figure 6 This is a schematic diagram of the intersection points of reflected rays from a virtual object provided in this application embodiment, based on... Figure 6 When the line of sight emitted by the virtual camera indicated by 601 is directed toward a virtual object, it is reflected to form the reflected line indicated by 602. This reflected line is directed toward the virtual scene and hits a certain position outside the virtual camera's field of view, namely the intersection of the reflected line indicated by the dashed box 603 and the second virtual object in the virtual scene.

[0104] It should be noted that the direction of collecting the illumination information of the reflected light from the first virtual object through the reflection probe is adjusted to the first direction. In other words, the collection direction of the reflection probe for the intersection point, i.e. the hit position, is adjusted to the first direction. The collection direction is the direction of the illumination information generated when the reflected light shines on the intersection point through the reflection probe.

[0105] For example, see Figure 7 , Figure 7 This is a schematic diagram of the first direction provided in the embodiments of this application, based on Figure 7 ,exist Figure 7In step a, before parallax correction is applied to the reflective probe, 701 indicates the acquisition direction for the intersection point, i.e., the hit position, via the reflective probe. After parallax correction, by connecting the reflective probe to the intersection point, the direction for acquiring illumination information at the intersection point via the reflective probe is adjusted as follows: Figure 7 The direction indicated by 702 in b is the direction from the position of the reflective probe to the intersection.

[0106] Thus, by pointing the position of the reflection probe in the first direction of the aforementioned intersection, which serves as the direction for collecting illumination information at the hit point of the reflected light, the accuracy of the reflection probe's acquisition position is ensured, thereby improving the accuracy of the illumination information collected by the reflection probe and ultimately enhancing the rendering effect of the reflected image.

[0107] In actual implementation, before performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the perspective of the virtual camera, it is also necessary to determine the position of the reflection probes. Specifically, a spatial coordinate system of the virtual scene is constructed, and the first coordinate of the reflection probe is determined in the spatial coordinate system. Thus, the process of determining the first direction from the position of the reflection probe to the intersection point can be to obtain the second coordinate of the intersection point in the spatial coordinate system; subtract the first coordinate from the second coordinate to obtain the vector pointing the reflection probe to the hit position, and determine the direction indicated by the vector as the first direction.

[0108] In this way, by constructing a virtual scene spatial coordinate system and then using vector calculation to determine the first direction from the position of the reflection probe to the intersection point, the accuracy of the determined first direction is ensured, thereby improving the accuracy of the illumination information collected based on the first direction and thus improving the rendering effect of the reflection image.

[0109] In actual implementation, before performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the perspective of the virtual camera, it is also necessary to obtain the reflection probes of the virtual scene. As mentioned above, the reflection probes are used to collect lighting information in the virtual scene.

[0110] It should be noted that, for the process of obtaining reflection probes in a virtual scene, when there are no reflection probes in the virtual scene yet, the process of obtaining reflection probes in the virtual scene can be: obtaining the position of the virtual camera in the virtual scene; generating reflection probes based on the position of the virtual camera; when there are already reflection probes in the virtual scene, the process of obtaining reflection probes in the virtual scene is simply to directly obtain the already generated reflection probes.

[0111] It should be noted that the process of generating a reflection probe based on the virtual camera's position can involve: acquiring the set attribute information of the reflection probe, including its type, size, sampling range, and intensity; and then, based on this attribute information, generating the reflection probe at the associated location of the virtual camera. The associated location of the virtual camera refers to its surrounding area, such as any location within a certain distance from the virtual camera target. For example, see [link to example]. Figure 8 , Figure 8 This is a schematic diagram of the reflective probe provided in an embodiment of this application, based on Figure 8 The dashed box 801 indicates the reflection probe generated at the associated location of the virtual camera.

[0112] In this way, generating a reflection probe based on the position of the virtual camera ensures that the reflection probe can collect all the lighting information within the observation range of the virtual camera, thereby improving the rendering effect of the reflection image obtained based on the lighting information.

[0113] In actual implementation, after generating the reflection probe, the position of the reflection probe can be updated. There are several ways to update the position of the reflection probe. Next, we will take three of them as examples to explain the process of updating the position of the reflection probe.

[0114] In some embodiments, updating the position of the reflective probe may involve detecting the position of the virtual camera; and updating the position of the reflective probe based on the second position of the virtual camera in response to the virtual camera's position changing from a first position to a second position.

[0115] It should be noted that the detection of the virtual camera's position can be real-time or periodic, and this application embodiment does not limit this; the first position and the second position are two different positions in the virtual scene; the process of updating the position of the reflection probe based on the second position of the virtual camera is that the reflection probe is placed at the associated position of the second position of the virtual camera. Here, the associated position of the second position refers to the surrounding position of the second position, such as any position at a distance from the target of the second position.

[0116] In this way, when the position of the virtual camera changes, the position of the reflection probe is updated synchronously. This ensures that all lighting information within the virtual camera's field of view can be collected through the reflection probe, thus avoiding the reflection probe missing any lighting information that should be collected. This improves the rendering effect of the reflection image obtained based on lighting information. At the same time, compared with related technologies that require artists to manually adjust the position of the reflection probe and place a large number of reflection probes in the virtual scene, this not only improves the rendering effect but also reduces labor costs and resource consumption.

[0117] In actual implementation, the process of updating the position of the reflection probe based on the second position of the virtual camera in response to the change of the virtual camera's position from the first position to the second position can be as follows: in response to the change of the virtual camera's position from the first position to the second position, determine the difference between the first position and the second position; in response to the difference reaching the difference threshold, update the position of the reflection probe.

[0118] It should be noted that the process of determining the difference between the first and second positions can involve: constructing a spatial coordinate system for the virtual scene; obtaining the coordinates of the first and second positions within this system; and determining the distance, or difference, between the first and second positions based on these coordinates.

[0119]

[0120] Where P is the distance or difference between the first position and the second position, the spatial coordinate system here is the world coordinate system in the virtual scene, which is used to define the position and orientation of all objects in the three-dimensional scene. It is a global reference frame that provides a common positioning reference for all objects in the three-dimensional scene; the coordinates of the first position are (x1, y1, z1), and the coordinates of the second position are (x2, y2, z2).

[0121] Then, when the difference reaches the difference threshold, the position of the reflection probe is updated. Here, the difference threshold is preset. The process of updating the position of the reflection probe in response to the difference reaching the difference threshold is that, in response to the difference reaching the difference threshold, the position of the reflection probe is updated based on the second position of the virtual camera.

[0122] Thus, when the virtual camera moves far enough, that is, beyond the difference threshold, the position of the reflection probe is updated, avoiding the omission of lighting information around the virtual camera that should be collected by the reflection probe, thereby improving the rendering effect of the reflection image obtained based on the lighting information.

[0123] In some embodiments, the virtual scene is divided into at least two grid cells, so that the process of updating the position of the reflection probe can be to detect the position of the virtual camera; and in response to the virtual camera changing from being in the first grid cell to being in the second grid cell, to update the position of the reflection probe based on the second grid cell.

[0124] It should be noted that the detection of the virtual camera's position can be real-time or periodic, and this application does not limit this; and in the process of dividing the virtual scene into at least two grid units, the virtual scene can be evenly divided into at least two grid units of the same size, or the virtual scene can be directly divided into at least two grid units, and the different grid units can be the same size or different sizes.

[0125] For example, see Figure 9 , Figure 9 This is a schematic diagram of at least two mesh cells provided in an embodiment of this application, based on Figure 9 The virtual scene is divided into blocks to obtain at least two grid cells of equal size, such as... Figure 9 There are nine grid cells of equal size.

[0126] In actual implementation, when the virtual camera changes from being in the first grid cell to being in the second grid cell, the position of the reflection probe is updated based on the second grid cell. The process of updating the position of the reflection probe based on the second grid cell is similar to the process of updating the position of the reflection probe based on the second position of the virtual camera mentioned earlier. Specifically, the process of updating the position of the reflection probe based on the second grid cell is to place the reflection probe at the associated position of the second grid cell. Here, the second grid cell refers to the surrounding position of the second grid cell, such as any position at a distance from the target at the second position, or it can be the middle position of the second grid cell, that is, within the second grid cell. This application embodiment does not limit this.

[0127] For example, see Figure 10 , Figure 10 This is a schematic diagram illustrating the process of updating the position of the reflection probe according to an embodiment of this application, based on Figure 10 The dashed box 1001 indicates the virtual camera, and 1003 indicates the movement trajectory of the virtual camera in the virtual scene. Based on the movement trajectory, when the virtual camera moves to different grid cells, the position of the virtual probe is also updated. Taking the grid cell indicated by the dashed box 1004 as an example, when the position of the virtual camera moves to the grid cell indicated by the dashed box 1004, the position of the reflection probe is updated from the position indicated by 1002 to the position indicated by 1005.

[0128] The change in the virtual camera's position during the above process is due to a change in the position of the player character controlled by the terminal. Since the virtual camera's field of view is also the player character's field of view, the virtual camera's position will change accordingly once the player character's position changes. Figure 10 The virtual camera indicated by the dashed box 1001 is displayed in the form of a human figure.

[0129] Thus, when the virtual camera changes from the first grid cell to the second grid cell, the position of the reflection probe is updated synchronously. This ensures that all lighting information within the virtual camera's field of view can be collected through the reflection probe, thus avoiding any omission of lighting information that should be collected. This improves the rendering effect of the reflection image obtained based on lighting information. At the same time, compared with related technologies that require artists to manually adjust the position of the reflection probe and place a large number of reflection probes in the virtual scene, this not only improves the rendering effect but also reduces labor costs and resource consumption.

[0130] In other embodiments, the virtual scene is divided into at least two grid cells. The process of obtaining the position of the virtual camera in the virtual scene may be to obtain the grid cell in which the virtual camera is located. Therefore, the process of updating the position of the reflection probe may be to obtain the duration for which the virtual camera is in the grid cell. When the duration reaches the target duration, the position of the reflection probe is updated.

[0131] It should be noted that the target duration can be preset, such as five minutes; and in the process of dividing the virtual scene into at least two grid units, the virtual scene can be evenly divided into at least two grid units of the same size, or the virtual scene can be directly divided into at least two grid units. The different grid units can be the same size or different sizes.

[0132] In actual implementation, when the target duration is reached, the position of the reflection probe is updated. Specifically, the process of updating the position of the reflection probe involves obtaining the real-time position of the virtual camera in the corresponding grid cell and placing the reflection probe at the associated position of the virtual camera's real-time position. Here, the associated position of the real-time position refers to the surrounding positions of the real-time position, such as any position at a distance from the target real-time position.

[0133] Thus, when the virtual camera spends too long in a grid cell, exceeding the target duration, updating the position of the reflection probe avoids the situation where the reflection probe misses the lighting information it should collect if the virtual camera moves away from the reflection probe in a larger grid cell when the grid cell size is different. This ensures that all lighting information within the virtual camera's field of view can be collected through the reflection probe, thereby improving the rendering effect of the reflection image obtained based on lighting information. At the same time, compared with related technologies that require artists to manually adjust the placement of reflection probes and place a large number of reflection probes in the virtual scene, this not only improves the rendering effect but also reduces labor costs and resource consumption.

[0134] It should be noted that all the above-mentioned processes for updating the position of the reflection probe can be either by directly placing the original reflection probe in the new position, thereby updating the position of the reflection probe, or by deleting the original reflection probe and generating a new reflection probe at the new position, thereby updating the position of the reflection probe. This application does not limit the specific method. The process of generating a new reflection probe at the new position is similar to the process of generating a reflection probe based on the position of the virtual camera mentioned above. Specifically, the attribute information of the set reflection probe is obtained, including the type, size, sampling range, intensity, etc. of the reflection probe. Then, based on the attribute information of the reflection probe, the reflection probe is generated at the new position. Here, the new position can be, for example, the associated position of the second position of the virtual camera, the associated position of the second grid cell, the associated position of the real-time position of the virtual camera, etc.

[0135] Step 102: Collect the illumination information of the reflected light from the first virtual object using the parallax-corrected reflection probe.

[0136] It should be noted that the lighting information of the reflected light here is used to indicate the color information at the intersection of the reflected light and the second virtual object in the virtual scene, that is, the color value of each pixel at the intersection. At the same time, as mentioned above, the reflection probe is used to collect lighting information in the virtual scene. Therefore, the lighting information of the reflected light of the first virtual object collected by the parallax-corrected reflection probe, that is, the lighting information at the intersection of the virtual light source and the virtual scene collected by the parallax-corrected reflection probe, is used as the lighting information of the reflected light collected by the parallax-corrected reflection probe.

[0137] In some embodiments, in addition to updating the position of the reflection probe, all the lighting information collected by the reflection probe can also be updated. As mentioned above, the reflection probe is used to collect lighting information in the virtual scene, that is, the reflection probe records the collected lighting information in the virtual scene. The lighting information in the virtual scene here does not refer to the lighting information of all virtual objects in the virtual scene, but to the lighting information of virtual objects within the target range in the virtual scene (that is, all the lighting information within the target range in the virtual scene).

[0138] It should be noted that the target range refers to the area around the virtual camera, that is, a circular area with the virtual camera as the center and the target distance as the radius. The target distance refers to the distance from the virtual camera to the farthest position that the virtual camera can observe. The virtual objects within the target range in the virtual scene include virtual objects with reflective capabilities as mentioned above, as well as virtual objects without reflective capabilities. That is, the virtual objects within the target range in the virtual scene can have high reflective capabilities or low reflective capabilities. For example, they can be forests, water sources, buildings, plants, animals, etc. This application does not limit this. At the same time, the lighting information here refers to the color information of the virtual objects within the target range in the virtual scene when the virtual light emitted by the virtual light source illuminates them. That is, the color value of each pixel of the virtual object, which is equivalent to the color value of each pixel when the virtual object is displayed.

[0139] Therefore, for the process of updating the illumination information collected by the reflection probe, see [link to documentation]. Figure 11 , Figure 11 This is a schematic diagram illustrating the process of updating the lighting information of a virtual scene collected by a reflection probe, as provided in an embodiment of this application. Figure 11 The process of updating the lighting information of the virtual scene collected by the reflection probe can be achieved through the following steps.

[0140] Step 201: When the update conditions of the reflection probe are met, the server updates the illumination information collected by the reflection probe to obtain the updated illumination information.

[0141] The update conditions include at least one of the following: the relative positional relationship between the parallax-corrected reflection probe and the virtual object within the target range in the virtual scene changes; or the light at the virtual object within the target range in the virtual scene changes.

[0142] It should be noted that the illumination information collected by the reflection probe here is also the illumination information of the virtual objects within the target range in the virtual scene (i.e., all illumination information within the target range in the virtual scene). The relative positional relationship between the parallax-corrected reflection probe and the virtual objects within the target range in the virtual scene changes. This can be due to several factors: the position of the reflection probe changes, but the position of the virtual objects within the target range in the virtual scene remains unchanged; or the position of the reflection probe remains unchanged, but the position of the virtual objects within the target range in the virtual scene changes; or the position of the reflection probe changes along with the position of the virtual objects within the target range in the virtual scene. This embodiment of the application does not limit the scope of these variations.

[0143] The light at virtual objects within the target area in a virtual scene refers to the virtual light emitted by a virtual light source when it illuminates virtual objects within the target area in the virtual scene, as described above. This includes direct light and indirect light. Direct light is emitted by a virtual light source and directly illuminates the corresponding virtual object. Indirect light is emitted by a virtual light source and illuminates the corresponding virtual object after at least one reflection.

[0144] Thus, when the relative positional relationship between the reflection probe and the virtual object within the target range in the virtual scene changes, and / or the light at the virtual object within the target range in the virtual scene changes, the lighting information collected by the reflection probe is updated, thereby ensuring the accuracy of the lighting information collected by the reflection probe, which in turn improves the effect of the reflection image rendered based on the reflection probe.

[0145] In practice, the process of updating the illumination information collected by the reflection probe and obtaining the updated illumination information varies depending on the update conditions. The following sections will explain the process of updating the illumination information collected by the reflection probe and obtaining the updated illumination information under different update conditions.

[0146] In some embodiments, the lighting information includes color information obtained by coloring virtual objects within the target range based on texture information of virtual objects within the target range; thus, the process of updating the lighting information collected by the reflection probe to obtain updated lighting information may involve updating the texture information corresponding to the lighting information when the update condition is a change in relative position relationship, obtaining new texture information; updating the color information based on the new texture information, obtaining new color information, and using the new color information as the updated lighting information.

[0147] It should be noted that the texture information here includes the shape, normal direction, depth value, reflection properties, refractive index, etc. of the corresponding virtual object. The process of updating the texture information corresponding to the lighting information to obtain new texture information can be to re-render the virtual objects within the target range to obtain new texture information. Here, geometric rendering refers to rendering each virtual object within the target range once, but only processing and storing the texture information of the virtual objects, without performing lighting calculations, i.e., shading.

[0148] The process of geometrically rendering virtual objects within the target area to obtain texture information involves: acquiring the attribute information (including position, size, shape, etc.) of the virtual objects in the model space corresponding to the virtual scene; determining at least two vertices of the virtual objects in the model space based on the attribute information, constructing a spatial coordinate system for the corresponding model space, and determining the coordinates and attributes (including normals, texture coordinates, tangents, etc.) of each vertex in the spatial coordinate system; performing coordinate transformation on the coordinates of each vertex in the spatial coordinate system to obtain the coordinates of each vertex in the screen coordinate system; and using the coordinates of each vertex in the screen coordinate system and the attributes of each vertex as the texture information of the virtual objects.

[0149] In this context, model space refers to the coordinate space used when modeling virtual objects in a virtual scene. It contains the geometric information and surface normals of the virtual objects. During rendering, the geometry of the virtual objects is initially stored in model space, which is associated with the virtual objects and is often called local space or object space. The process involves transforming the coordinates of each vertex in the model space coordinate system to obtain the coordinates of each vertex in the screen coordinate system. Specifically, based on the model matrix, the coordinates of each vertex in the model space coordinate system are transformed to the world coordinate system corresponding to the world space, resulting in the vertex coordinates of each vertex in the world coordinate system. Then, based on the view matrix, the coordinates of each vertex in the world coordinate system are transformed to the camera coordinate system corresponding to the camera space, resulting in the vertex coordinates of each vertex in the camera coordinate system. Finally, based on the projection matrix, the coordinates of each vertex in the camera coordinate system are transformed to the screen coordinate system corresponding to the screen space, resulting in the vertex coordinates of each vertex in the screen coordinate system.

[0150] The model matrix includes information such as the translation, rotation, and scaling of virtual objects; the view matrix defines the position and orientation (i.e., the viewpoint) of the virtual camera; and the projection matrix is ​​used to project three-dimensional space onto a two-dimensional screen.

[0151] In practice, the process of updating color information based on new texture information to obtain new color information can be as follows: based on the new texture information, determine the texture image of the virtual object within the target range in the virtual scene, and based on the texture image and the light at the virtual object, determine the color value of each pixel in the texture image; and use the color value of each pixel as the new color information.

[0152] It should be noted that, as mentioned earlier, the process of determining the texture image of virtual objects within the target range in a virtual scene based on new texture information involves using the coordinates of each vertex in the screen coordinate system and the attributes of each vertex as the texture information of the virtual object. Therefore, rasterization is performed based on the texture information of the virtual object to obtain the texture image of the virtual object within the target range in the virtual scene, which is also the uncolored two-dimensional image of the virtual object in screen space. Here, rasterization is used to convert the geometric data of the virtual object into pixel data, that is, to convert vector graphics (such as lines and curves) into a raster image composed of pixels.

[0153] The process of determining the color value of each pixel in a texture image based on the light rays at the texture image and the virtual object involves performing lighting calculations (i.e., lighting rendering and shading) based on the light rays at the texture image and the virtual object to obtain the color value of each pixel in the texture image, thereby determining the color value of each pixel as new color information.

[0154] It should be noted that the reflection probe records information by first performing geometric rendering on the virtual object to obtain texture information, and then performing lighting rendering based on the texture information to obtain the final recorded color information. When the relative positional relationship changes, it means that the texture information of the virtual object recorded by the reflection probe, such as shape information and depth information, will change. Therefore, the texture information recorded by the reflection probe needs to be updated. At the same time, if the texture information changes, the color information obtained by shading based on the texture information will also change. Based on this, when the relative positional relationship changes, a complete rendering process is required, that is, updating both the texture information and the color information together.

[0155] In this way, when the relative positional relationship changes, the texture and color information of the virtual object are updated together, ensuring the accuracy of the lighting information collected by the reflection probe, thereby improving the rendering effect of the reflection image.

[0156] In other embodiments, the lighting information includes color information obtained by coloring the virtual objects within the target range based on the texture information of the virtual objects within the target range; thus, the process of updating the lighting information collected by the reflection probe to obtain the updated lighting information may be as follows: when the update condition is a change in light, the texture information corresponding to the lighting information is obtained; based on the texture information, the color information is updated to obtain new color information, and the new color information is used as the updated lighting information.

[0157] It should be noted that, as mentioned above, texture information includes the shape, normal direction, depth value, reflection properties, refractive index, etc. of the corresponding virtual object; and the texture information corresponding to the acquired lighting information is the pre-rendered texture information; therefore, the process of updating the color information based on the texture information to obtain new color information can be as follows: based on the texture information, acquire the texture image of the virtual object within the target range in the virtual scene, and determine the color value of each pixel in the texture image based on the texture image and the light at the virtual object; and use the color value of each pixel as the new color information; or it can be that the texture image of the virtual object within the target range acquired before the light change is directly acquired, and the color value of each pixel in the texture image is determined based on the texture image and the light at the virtual object; and the color value of each pixel is used as the new color information.

[0158] It should be noted that, as mentioned earlier, the way the reflection probe records information is to first perform geometric rendering on the virtual object to obtain texture information, and then perform lighting rendering based on the texture information to obtain the final recorded color information. When the light changes, it means that only the color information of the virtual object recorded by the reflection probe changes, while the texture information of the virtual object, such as shape information and depth information, does not change. Therefore, it is not necessary to update the texture information recorded by the reflection probe, but only to update the color information of the virtual object. Based on this, when the light at the virtual object changes, it is not necessary to perform a complete rendering process, that is, it is not necessary to update the texture information. It is only necessary to re-color based on the original texture information, that is, only to update the color information.

[0159] In this way, when the light at the virtual object changes, the color information of the virtual object is updated only based on the original texture information. This not only ensures the accuracy of the lighting information collected by the reflection probe, but also reduces the resource consumption of the lighting information update process of the reflection probe. Thus, the rendering effect of the reflection image is guaranteed with minimal resource consumption.

[0160] In other embodiments, when the update conditions are changes in relative position and changes in light, the process of updating the illumination information collected by the reflection probe to obtain updated illumination information is the same as the process described above of updating the illumination information collected by the reflection probe to obtain updated illumination information when the update conditions are changes in relative position and light. Therefore, this embodiment will not be described in detail.

[0161] In some embodiments, the reflective probe is shaped like a polyhedron, with different faces of the polyhedron recording local illumination information from the virtual scene. The local illumination information recorded on different faces is different, and the illumination information includes the local illumination information recorded on each face. Therefore, the process of updating the illumination information collected by the reflective probe to obtain updated illumination information can be as follows: determining the update period of the reflective probe; wherein, the update period is used to indicate the minimum update interval between any two faces of the polyhedron; based on the update period, the local illumination information recorded on each face of the polyhedron is updated sequentially to obtain updated illumination information.

[0162] It should be noted that a polyhedron can include a cube. Since the shape of the reflective probe is a polyhedron, each face captures the lighting information of the virtual object in one direction. Therefore, the reflective probe can collect lighting information in 360°, that is, it can collect the lighting information of all virtual objects within the target range. Each face of the reflective probe only records the lighting information of the virtual object in the direction corresponding to that face, that is, local lighting information. In this way, the local lighting information recorded by each face constitutes the total lighting information within the target range collected by the reflective probe.

[0163] In practical implementation, since the reflection probe includes multiple faces, updating the lighting information of the reflection probe is also updating the local lighting information of multiple faces of the reflection probe. Taking a cube as an example, if the cube is directly rendered six times in one frame to update, it will result in a very long frame time. If updating a certain frame of the reflection probe takes a long time, and the frame time is very short when not updating, it will cause stuttering during the update. Therefore, the six faces are updated in six times, and only the local lighting information of one face is updated each time, and the update is performed once every update cycle. The process of updating the local lighting information of one face is as described above, that is, based on the update conditions (including changes in relative position relationship and / or changes in the light at the virtual object), the specific update process of the local lighting information is determined. This embodiment of the application will not be described in detail here.

[0164] It should be noted that the update interval refers to the frame difference or time difference between updating two faces. For example, when the update interval is 10 frames, each face is updated once every 10 frames. Thus, when the reflection probe is a cube, the reflection probe can be completely updated within 60 frames. The update period can be determined based on the scene frame rate in the virtual scene or it can be preset. For example, if the update interval refers to the frame difference between updating two faces, the update period is preset, such as 10 frames. If the update interval refers to the time difference between updating any two faces, the update period is negatively correlated with the scene frame rate. For example, when the scene frame rate is 60, the update period can be 1 / 6 second, and when the scene frame rate is 120, the update period can be 1 / 12 second.

[0165] In practice, based on the update cycle, the local illumination information recorded on each face of the polyhedron is updated sequentially until the local illumination information of each face is updated, thus obtaining the updated illumination information. In this way, the update of the reflection probe is distributed across multiple frames, avoiding the stuttering problem caused by updating the reflection probe, and enabling the update of the reflection probe without reducing the smoothness.

[0166] Step 202: Obtain the lighting information of the reflected light of the first virtual object from the updated lighting information.

[0167] In practice, if the lighting information of the virtual scene collected by the parallax-corrected reflection probe is updated before the probe collects the lighting information of the reflected light from the first virtual object, then the process of collecting the lighting information of the reflected light from the first virtual object using the parallax-corrected reflection probe is equivalent to obtaining the lighting information of the reflected light from the first virtual object from the updated lighting information. This ensures that the obtained lighting information is the most up-to-date lighting information.

[0168] In some embodiments, since there are many virtual objects in the virtual scene, the acquisition direction of the reflection probe may be obstructed. If the acquisition direction of the reflection probe is obstructed, the lighting information acquired by the reflection probe needs to be supplemented; specifically, see [link to relevant documentation]. Figure 12 , Figure 12 This is a flowchart illustrating the process of completing the lighting information of a virtual scene collected by a reflection probe, as provided in an embodiment of this application. Figure 12 The process of completing the lighting information of the virtual scene collected by the reflection probe can be achieved through the following steps.

[0169] Step 301: The server obtains the acquisition direction for collecting illumination information at the intersection of the reflected light rays by the reflection probe after parallax correction.

[0170] It should be noted that the intersection point corresponding to the reflection probe is the intersection position of the reflected light from the first virtual object and the second virtual object in the virtual scene; the process of obtaining the acquisition direction of the reflection probe for collecting illumination information at the intersection point corresponding to the reflected light after parallax correction is the same as described above. Specifically, after determining the position of the reflection probe, the direction from the position of the reflection probe to the intersection point will be used as the acquisition direction of the reflection probe for collecting illumination information at the intersection point.

[0171] Step 302: If there is an obstruction in the acquisition direction, the illumination information acquired by the parallax-corrected reflection probe is supplemented.

[0172] It should be noted that occlusions are used to indicate virtual objects in a virtual scene that can obstruct the propagation of light. These can be plants, animals, buildings, boxes, etc. As mentioned earlier, the virtual light emitted by a virtual light source when it illuminates a virtual object within the target range in a virtual scene includes direct light and indirect light. Direct light is emitted by the virtual light source and directly illuminates the corresponding virtual object, such as the virtual light at the intersection. Indirect light is emitted by the virtual light source and illuminates the corresponding virtual object, such as the virtual light at the intersection, after at least one reflection. Therefore, when there are no occlusions, the reflection probe can collect the lighting information at the intersection through the virtual light. However, when there are occlusions, the reflection probe cannot collect the lighting information at the intersection through the virtual light, but can only collect the lighting information at the occlusion through the virtual light.

[0173] Specifically, the ability of a reflection probe to collect lighting information at an intersection means that when direct or indirect light rays illuminate the intersection, they are reflected from the intersection on the virtual object to the reflection probe, allowing the reflection probe to "see" the lighting information at the intersection, thus enabling it to collect the lighting information at the intersection. However, due to obstructions in the collection direction, direct or indirect light rays cannot be reflected from the intersection on the virtual object to the reflection probe when illuminating the intersection, preventing the reflection probe from "seeing" the lighting information at the intersection, which is equivalent to the reflection probe being unable to collect the lighting information at the intersection. On the other hand, when direct or indirect light rays illuminate an obstruction, they may be reflected from the obstruction to the reflection probe, allowing the reflection probe to "see" the lighting information at the obstruction, which means the reflection probe can collect the lighting information at the obstruction.

[0174] For example, see Figure 13 , Figure 13 This is a schematic diagram of the obstruction provided in the embodiments of this application, based on Figure 13 The dashed box 1301 indicates an obstruction that prevents the reflective probe from collecting light in the virtual scene.

[0175] Therefore, if there is an obstruction in the acquisition direction, the lighting information acquired by the parallax-corrected reflection probe is supplemented, enabling the reflection probe to acquire lighting information at the intersection point. Thus, when there is an obstruction in the acquisition direction, supplementing the acquired lighting information allows the reflection probe to acquire the lighting information blocked by the obstruction, improving the comprehensiveness of the lighting information acquired by the reflection probe and avoiding the problem of poor rendering of the reflected image due to missing lighting information, thereby improving the rendering effect of the reflected image.

[0176] In practical implementation, to determine the presence of an obstruction in the acquisition direction, specifically before supplementing the illumination information acquired by the parallax-corrected reflective probe, a spatial coordinate system of the virtual scene can be constructed. Within this spatial coordinate system, the depth of the reflective probe in the acquisition direction is determined; the third coordinate of the reflective probe in the spatial coordinate system and the fourth coordinate of the intersection point in the spatial coordinate system are obtained; based on the third and fourth coordinates, the distance between the reflective probe and the intersection point is determined; when the distance is greater than the depth of the reflective probe in the acquisition direction, it is determined that an obstruction exists in the acquisition direction.

[0177] It should be noted that the spatial coordinate system here is the camera coordinate system, that is, the coordinate system corresponding to the virtual camera. It is a reference system used to describe the position and orientation of the virtual camera in three-dimensional space. It is defined from the camera's perspective and is used to simulate the perspective and field of view when the virtual camera observes a three-dimensional scene. Since the reflective probe is a polyhedron such as a cube, each face of the reflective probe has a corresponding depth. The depth of each face of the reflective probe is also the vertical coordinate of the coordinates of each face of the reflective probe in the camera coordinate system. In the camera coordinate system, the process of determining the depth of the reflective probe in the acquisition direction is to take the vertical coordinate of the face of the reflective probe corresponding to the acquisition direction, which is the depth, as the depth of the reflective probe in the acquisition direction.

[0178] It should be noted that after determining the third coordinate of the reflective probe in the spatial coordinate system and obtaining the fourth coordinate of the intersection point in the spatial coordinate system, the process of determining the distance between the reflective probe and the intersection point based on the third and fourth coordinates can refer to the formula (1) mentioned above. Thus, when the distance is greater than the depth of the reflective probe in the acquisition direction, it is determined that the intersection point is behind the obstruction, that is, it is determined that there is an obstruction in the acquisition direction of the reflective probe towards the intersection point.

[0179] For example, see [link to previous article] Figure 13 Based on the third and fourth coordinates, the distance between the reflective probe and the intersection point is determined as shown in AC, while the depth of the reflective probe in the acquisition direction is shown in AB. Figure 13When the distance is greater than the depth of the reflective probe in the acquisition direction, the intersection point is determined to be behind the obstruction, that is, the reflective probe is determined to be behind the obstruction in the acquisition direction of the intersection point.

[0180] In this way, determining whether there is an obstruction in the acquisition direction of the reflective probe based on the depth of the reflective probe avoids misjudgment when determining whether there is an obstruction in the acquisition direction, improves the accuracy of determining whether there is an obstruction, and thus can promptly supplement the illumination information acquired by the reflective probe when it is determined that there is an obstruction in the acquisition direction.

[0181] In actual implementation, if there are obstructions in the acquisition direction, there are multiple processes for completing the illumination information acquired by the parallax-corrected reflection probe. The following will use two of them as examples to explain the process of completing the illumination information acquired by the parallax-corrected reflection probe.

[0182] In some embodiments, the process of supplementing the illumination information collected by the parallax-corrected reflection probe may involve adjusting the position of the reflection probe to a target position; wherein, at the target position, there are no obstructions on the acquisition direction for the intersection point, and the target position is used to indicate the position where the reflection probe can collect illumination information for the intersection point; and obtaining the illumination information of the reflected light of the first virtual object collected by the reflection probe at the target position.

[0183] It should be noted that the process of determining the target location can be as follows: First, based on the size and intersection of the occluding object, determine its occlusion range. Specifically, radiate rays from the intersection point outwards, then select the ray that intersects with the occluding object from at least two rays, thus defining the range formed by the intersecting rays as the occlusion range. Then, outside the occlusion range, any location around the virtual camera can be selected as the target location. For example, see [link to example]. Figure 14 , Figure 14 This is a schematic diagram of the target location provided in the embodiments of this application, based on Figure 14 If there is an obstruction in the acquisition direction, the position of the reflective probe will be changed as follows: Figure 14 Move to the position indicated by 1401 in a, as shown Figure 14 The target location indicated by 1402 in b.

[0184] Then, the illumination information of the intersection point of the reflected light rays collected by the reflection probe at the target location is used as the lighting information for completion. Thus, when there is an obstruction in the acquisition direction, the position of the reflection probe is adjusted to complete the missing illumination information, improving the comprehensiveness of the illumination information collected by the reflection probe and avoiding the problem of poor rendering of the reflection image due to missing illumination information, thereby improving the rendering effect of the reflection image. At the same time, compared with the scheme of completing the missing illumination information of the reflection probe based on an illumination information database, the accuracy of the completed illumination information is improved, further enhancing the rendering effect of the reflection image.

[0185] In other embodiments, the process of completing the illumination information collected by the parallax-corrected reflection probe may involve: marking the intersection points to obtain markers for indicating the intersection points; acquiring a virtual scene illumination information library, which stores the illumination information of reflected light rays in the virtual scene; and sampling the illumination information at the intersection points from the illumination information library based on the markers.

[0186] It should be noted that the lighting information library is pre-set, that is, it includes the lighting information generated when all virtual objects in the virtual scene are illuminated by the virtual light source; after the hit position is marked, the lighting information of the corresponding hit position is sampled from the lighting information library, and the sampled lighting information is used as the lighting information for completion.

[0187] Thus, when there are obstructions in the acquisition direction, the missing lighting information of the reflection probe is supplemented by the lighting information database, improving the comprehensiveness of the lighting information collected by the reflection probe and avoiding the problem of poor rendering effect of the reflection image due to missing lighting information, thereby improving the rendering effect of the reflection image. At the same time, compared with the solution of supplementing the missing lighting information of the reflection probe by adjusting the position of the reflection probe, since there are multiple objects in the virtual object, there may still be obstructions after adjusting the position, making it difficult to find a suitable target position. Therefore, supplementing the missing lighting information of the reflection probe by the lighting information database simplifies the process of supplementing the missing lighting information of the reflection probe and reduces resource consumption.

[0188] Step 303: Obtain the illumination information of the reflected light obtained through completion.

[0189] In practice, if the lighting information of the virtual scene collected by the reflection probe is supplemented before acquiring the lighting information of the reflected light collected by the parallax-corrected reflection probe, then the process of acquiring the lighting information of the reflected light collected by the parallax-corrected reflection probe is also equivalent to acquiring the lighting information of the reflected light obtained through supplementation. This ensures that the acquired lighting information is accurate.

[0190] Step 103: Obtain the texture image of the first virtual object, and determine the color value of the image pixels based on the texture image and lighting information.

[0191] In practice, the process of acquiring the texture image of the first virtual object is similar to the process described above of performing geometric rendering on virtual objects within the target range to obtain texture information, and determining the texture image of virtual objects within the target range in the virtual scene based on the texture information. Specifically, the process involves acquiring the attribute information (including position, size, shape, etc.) of the first virtual object in the model space corresponding to the virtual scene; determining at least two vertices of the first virtual object in the model space based on the attribute information of the first virtual object, constructing a spatial coordinate system for the corresponding model space, and determining the coordinates and attributes (including normals, texture coordinates, tangents, etc.) of each vertex in the spatial coordinate system; performing coordinate transformation on the coordinates of each vertex in the spatial coordinate system to obtain the coordinates of each vertex in the screen coordinate system; and using the coordinates of each vertex in the screen coordinate system and the attributes of each vertex as the texture information of the first virtual object.

[0192] Then, rasterization is performed based on the texture information of the virtual object to obtain the texture image of the first virtual object in the virtual scene, that is, the uncolored two-dimensional image of the first virtual object in the screen space; here, rasterization is used to convert the geometric data of the first virtual object into pixel data, that is, to convert vector graphics (such as lines and curves) into a raster image composed of pixels.

[0193] Next, for the process of determining the color value of image pixels based on texture image and lighting information, specifically, for each image pixel in the texture image, lighting calculation (i.e., lighting rendering and shading) is performed based on the texture information of the first virtual object to obtain the basic color value of each image pixel; based on the reflection position of the first virtual object, the target image pixel in the texture image used to represent the color of the intersection corresponding to the reflected light rays is determined, and the reflection color value of the target pixel is determined based on the lighting information; for each target image pixel, the basic color value and the reflection color value are merged to obtain the color value of the target image pixel; based on the color value of the target image pixel and the basic color values ​​of the other image pixels in at least two image pixels in the texture image besides the target image pixel, the color value of each image pixel in the texture image is obtained.

[0194] It should be noted that the reflection position is the position where the line of sight emitted by the virtual camera hits the first virtual object, that is, the position where the first virtual object reflects the line of sight emitted by the virtual camera. At the same time, the line of sight will be illuminated to the corresponding intersection point by the reflection of the first virtual object. Therefore, the target image pixel point used to present the color of the intersection point corresponding to the reflected light in the texture image of the first virtual object is the image pixel point corresponding to the reflection position in the texture image.

[0195] Furthermore, since the lighting information is the color information at the intersection of the reflected light rays, the reflection color value of the target pixel can be determined based on the lighting information. The base color value of each image pixel is used to indicate the color of the first virtual object itself when the reflection effect is not considered. The reflection color value is used to indicate the color at the intersection of the reflected light rays. Since the reflection position on the first virtual object corresponds to the intersection of the reflected light rays, that is, the image pixels in the texture image used to represent the color of the intersection of the reflected light rays are the same as the image pixels in the texture image used to represent the reflection position. Therefore, the corresponding image pixels include both the color of the first virtual object itself and the color of the intersection of the reflected light rays. Therefore, it is necessary to merge the colors of the corresponding image pixels to obtain the color value of each image pixel at the reflection position in the texture image.

[0196] In practice, there are several ways to combine the base color value and the reflected color value to obtain the color value of the target image pixel. Next, we will take two of them as examples to explain the process of combining the base color value and the reflected color value to obtain the color value of the target image pixel.

[0197] In some embodiments, the process of merging the base color value and the reflected color value to obtain the color value of the target image pixel may be to add the base color value and the reflected color value to obtain the color value of each target pixel.

[0198] In other embodiments, the process of merging the base color value and the reflected color value to obtain the color value of the target image pixel may involve: obtaining the reflectivity of the reflection location; obtaining a first weight corresponding to the reflected color value and a second weight corresponding to the base color value based on the reflectivity; and performing a weighted sum based on the first weight, the second weight, the reflected color value, and the base color value to obtain the color value of each target pixel.

[0199] Step 104: Render the reflection image of the first virtual object based on the color values ​​of the image pixels.

[0200] In actual implementation, after determining the color value of each image pixel in the texture image, the image is colored in the frame buffer space based on the color value of the image pixel to obtain the reflection image of the first virtual object.

[0201] The frame buffer space, also known as the frame buffer, is a memory buffer used to store rendering results. It is used by the graphics processor to draw images on top of the rendering results and then present the drawing results on the screen.

[0202] By applying the above embodiments of this application, based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, parallax correction is performed on the reflection probe set in the virtual scene. Then, through the parallax-corrected reflection probe, the illumination information of the reflected light from the first virtual object is collected. Based on the acquired texture image of the first virtual object and the collected illumination information, the color values ​​of the image pixels are determined. Based on the color values ​​of the image pixels, the reflection image of the first virtual object is rendered. Thus, compared to a reflection scheme based on a reflection probe, this application, by correcting the parallax of the reflection probe, corrects the acquisition position of the illumination information, improving the accuracy of the acquired illumination information, and thereby improving the rendering effect of the reflection image; and compared to a reflection scheme based on emitted rays, it improves the precision of the acquired illumination information, thereby improving the rendering effect of the reflection image.

[0203] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0204] Reflection schemes in computer graphics generally suffer from two problems: insufficient accuracy and high performance consumption. Common schemes include reflection probe-based schemes and ray-based schemes. In reflection probe-based schemes, reflection probes are typically placed manually. Each reflection probe acts like a camera looking at six faces, recording the color information of those six faces into a cube map. When calculating the color of a smooth surface, the color in the same direction is directly sampled from nearby reflection probes based on the reflection direction of the smooth surface; for example... Figure 15 As shown, Figure 15 This is a schematic diagram of a reflection scheme based on a reflection probe provided in an embodiment of this application. Figure 15 Typically, reflection probes only collect lighting information based on the direction of reflection. For example, if a reflection probe collects lighting information according to the sampling direction indicated by 1501, since the sampling direction of the reflection probe and the direction of reflection at the reflection point are the same, it is obvious that the world space location corresponding to the specular reflection hit point (hit position, intersection) and the color (lighting information) sampled from the reflection probe are inconsistent. This leads to a situation where the further the surface reflection point (reflection position) is from the reflection probe, the worse the final effect becomes. Figure 16 As shown, Figure 16This is a rendering of the reflection scheme based on the reflection probe provided in the embodiments of this application; based on Figure 16 , Figure 16 The value indicated by 'a' is the accurate reflected image. Figure 16 The value of 'b' refers to the reflection image rendered by a reflection scheme based on a reflection probe in related technologies. Figure 16 b. Because the world space location corresponding to the specular reflection hit point and the color sampled from the reflection probe is inconsistent, an accurate reflection image cannot be rendered in the mirror as indicated by dashed box 1602. The upper part of the mirror can only display an incorrect image, as indicated by dashed box 1601.

[0205] In a ray-based reflection scheme, ray emission can accurately determine the world spatial location. Then, by sampling the corresponding world spatial illumination storage structure (optical information database) based on that location, illumination information can be obtained, such as... Figure 17 As shown, Figure 17 This is a schematic diagram of a reflection scheme based on an emitted ray provided in an embodiment of this application. Figure 17 After obtaining the accurate hit point (reflection position) of the reflected ray in world space by emitting rays, the illumination information of the corresponding hit point is sampled from the world space illumination storage structure. However, this depends entirely on the accuracy of the world space illumination storage structure. Higher accuracy means greater performance overhead, so its accuracy cannot be very high. This leads to the problem that even if the sampling position is correct, the effect is still very poor. Figure 18 As shown, Figure 18 This is a rendering of the reflection scheme based on the emitted ray provided in the embodiments of this application; based on Figure 18 , Figure 18 The value indicated by 'a' is the accurate reflected image. Figure 18 The value of 'b' refers to the reflection image rendered using a reflection scheme based on emitted light rays in related technologies. Figure 18 b. Due to the low precision of the world space lighting storage structure, in the mirror indicated by dashed box 1801, the reflection image obtained by sampling the lighting information in the world space lighting storage structure and then rendering it will display a mosaic.

[0206] Therefore, the drawback of the reflection scheme based on reflection probes is that it only achieves good results when the reflection probes are very close to the mirror surface; otherwise, the effect is poor. This necessitates manual adjustment of the probe positions by artists, resulting in both subpar results and high labor costs. The drawback of the reflection scheme based on emitted rays is that it requires a high-precision world-space lighting storage structure to achieve good reflection effects, but this high-precision structure imposes a significant performance burden.

[0207] Based on this, this application provides an image rendering method that combines the advantages of reflection schemes based on reflection probes and reflection schemes based on emission rays. It uses the emission ray method to obtain the accurate position in world space, and uses this accurate position to perform parallax correction on the sampling direction of the reflection probe. At the same time, it adaptively updates the position of the reflection probe to reduce labor costs and the memory usage of the reflection probe. Ultimately, it achieves a much better effect than the reflection scheme based on reflection probes with a much lower overhead than the reflection scheme based on emission rays.

[0208] In practical applications, the technical solution of this application can be started by starting a command. For example, the command r.GI.DynamicUpdateReflectionProbe 0 is used to close the technical solution of this application, and the command r.GI.DynamicUpdateReflectionProbe 1 is used to start the technical solution of this application.

[0209] In practical implementation, addressing the pain points of poor performance and high performance overhead of common reflection schemes in related technologies, this application uses ray casting to emit rays from the surface reflection points (reflection positions) seen within the screen space (the viewing range of the virtual camera) towards the reflection direction, and obtains the reflection hit points (hit positions, intersections) in world space. The line connecting the hit point and the reflection probe serves as the sampling direction of the reflection probe. This achieves very high-quality reflection effects with only the overhead of ray casting. Simultaneously, this application automatically maintains globally unique reflection probes. The positions of these probes change as the character (virtual camera) moves. When the position of objects (virtual objects within the target range) or the lighting (light at virtual objects within the target range) changes, the information stored in the reflection probes is updated in real time. This eliminates the need for artists to manually place hundreds or thousands of reflection probes and manually bake lighting information onto cube maps when objects or lighting are updated. Furthermore, the memory usage is reduced from hundreds of probes to just one.

[0210] The technical solution of this application includes the following five parts:

[0211] 1. Adaptive placement of reflection probes: This application adopts a heuristic method to automatically update the position of globally unique reflection probes. When implementing the technical solution of this application, a global reflection probe is automatically generated at the position of the character camera (virtual camera), and the world space is divided into tiles. When the character stays in the same tile for a long time, the probe position is updated. Similarly, when the character moves to another tile or the moving distance reaches a certain threshold, the probe position is updated. This ensures that the reflection probe is always near the character camera, thereby maximizing the use of the information of the reflection probe.

[0212] 2. Adaptive Update of Reflection Probes: The reflection probes record information by rendering the six faces of the image and then storing the information of the six faces in a cube map. This rendering method uses deferred rendering, a modern graphics rendering technique that separates geometry processing and lighting calculations to improve the efficiency of lighting calculations in complex scenes. First, in the geometry phase, the geometric information of all visible objects (such as position, normals, material properties, etc.) is rendered and stored in a set of textures called G-Buffers. Then, in the lighting phase, lighting calculations are performed using this pre-stored geometric information, without needing to re-traverse and process all the geometry in the scene. This allows for efficient handling of a large number of light sources because lighting calculations are performed only in screen space, avoiding the overhead of calculating for each object for each light source in traditional forward rendering. Specifically, the G-Buffers of the six faces need to be rendered first, and then shading is performed using the G-Buffers. The G-Buffers record the geometric information (texture information) of the objects. In this application, object movement and lighting updates are distinguished. When an object moves (the relative position of the virtual camera and the virtual object changes), the geometric information of the six faces observed by the reflection probe changes, so a complete deferred rendering process is required at this time. When the lighting is updated (the light at the virtual object changes), since the relative position of the object in the world does not change, the information recorded in the G-Buffer does not need to be updated. Therefore, the original G-Buffer is retained, and only the final rendered image is updated.

[0213] 3. Frame-by-frame update of reflection probe: Updating reflection probe often requires updating information on six faces. However, if the update is done by rendering six times in one frame, the frame will take a very long time. If the frame for updating the reflection probe takes a long time, the frame for not updating will take a very short time, which will cause stuttering during the update. Therefore, in this application, the six faces are updated in six times, with only one face updated each time, and the update is done every 10 frames. This way, the update can be completed within 60 frames. The updated content will be replaced in the original cube map. This is equivalent to spreading the update of the screen probe across multiple frames, which can update the probe in real time without reducing the smoothness.

[0214] 4. Calculate the reflection color: The technical solution of this application uses ray query to quickly obtain the reflection hit point (hit position, intersection) in world space from the surface reflection point (reflection position) towards the reflection direction. The direction vector formed by connecting the reflection hit point in world space with the world position of the reflection probe is the sampling direction of the reflection probe. The color is sampled directly from the cube map of the probe.

[0215] 5. Completing Missing Illumination Information: In most cases, the reflection probe can obtain the correct reflected color through parallax correction. However, in some cases, an object may block the reflection probe from a certain direction. In this situation, the reflection probe cannot see the information behind the obstruction. If the specular reflection hit point (hit position) happens to be in this situation, the reflection probe will lack information for this part, such as... Figure 13 As shown. In this case, if the reflection probe is directly sampled using the original method, the lighting information used for the reflection hit point behind the occluder will actually be from the surface of the occluder, resulting in a problem similar to ghosting. To address this issue, this application stores the depth of the six surfaces seen in the reflection probe. When using the color of the reflection probe, the depth is used to determine if the distance from the specular reflection hit point to the reflection probe is greater than the depth in the corresponding direction of the reflection probe. This indicates that the specular reflection hit point is actually behind the occluder, and the reflection probe cannot be directly sampled. These specular reflection hit points are then marked, and a backtracking scheme is applied to them, which involves using a world space lighting storage structure (lighting information database) with lower sampling accuracy.

[0216] By applying the above embodiments of this application, based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, parallax correction is performed on the reflection probe set in the virtual scene. Then, through the parallax-corrected reflection probe, the illumination information of the reflected light from the first virtual object is collected. Based on the acquired texture image of the first virtual object and the collected illumination information, the color values ​​of the image pixels are determined. Based on the color values ​​of the image pixels, the reflection image of the first virtual object is rendered. Thus, compared to a reflection scheme based on a reflection probe, this application, by correcting the parallax of the reflection probe, corrects the acquisition position of the illumination information, improving the accuracy of the acquired illumination information, and thereby improving the rendering effect of the reflection image; and compared to a reflection scheme based on emitted rays, it improves the precision of the acquired illumination information, thereby improving the rendering effect of the reflection image.

[0217] The following description continues to illustrate the exemplary structure of the image rendering apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the image rendering device 455 of the memory 450 may include:

[0218] The correction module 4551 is used to perform parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera.

[0219] The acquisition module 4552 is used to acquire the illumination information of the reflected light of the first virtual object through the parallax-corrected reflection probe;

[0220] The determining module 4553 is used to acquire the texture image of the first virtual object and determine the color value of the image pixel based on the texture image and the lighting information;

[0221] The rendering module 4554 is used to render the reflection image of the first virtual object based on the color values ​​of the image pixels.

[0222] In some embodiments, the correction module 4551 is further configured to determine the intersection point of the reflected light from the first virtual object and the second virtual object in the virtual scene based on the position of the first virtual object and the viewpoint of the virtual camera; determine a first direction from the position of the reflection probe to the intersection point; and adjust the direction of the illumination information of the reflected light from the first virtual object collected through the reflection probe to the first direction.

[0223] In some embodiments, the correction module 4551 is further configured to construct a spatial coordinate system of the virtual scene, and determine the first coordinate of the reflection probe in the spatial coordinate system; obtain the second coordinate of the intersection point in the spatial coordinate system; subtract the first coordinate from the second coordinate to obtain the vector of the reflection probe pointing to the intersection point, and determine the direction indicated by the vector as the first direction.

[0224] In some embodiments, the apparatus further includes an acquisition module, which is configured to acquire the position of the virtual camera in the virtual scene and generate the reflection probe based on the position of the virtual camera.

[0225] In some embodiments, the apparatus further includes a first update module, which is configured to detect the position of the virtual camera; and update the position of the reflection probe based on the second position of the virtual camera in response to the virtual camera's position changing from a first position to a second position.

[0226] In some embodiments, the first update module is further configured to determine the difference between the first position and the second position in response to the virtual camera's position changing from a first position to a second position; and to update the position of the reflection probe in response to the difference reaching a difference threshold.

[0227] In some embodiments, the virtual scene is divided into at least two grid cells, and the first update module is further configured to detect the position of the virtual camera; in response to the virtual camera changing from being in the first grid cell to being in the second grid cell, the position of the reflection probe is updated based on the second grid cell.

[0228] In some embodiments, the virtual scene is divided into at least two grid cells, and the acquisition module is further configured to acquire the grid cell in which the virtual camera is located in the virtual scene; the first update module is further configured to acquire the duration for which the virtual camera is in the grid cell; when the duration reaches a target duration, the position of the reflection probe is updated.

[0229] In some embodiments, the parallax-corrected reflection probe records illumination information of virtual objects within a target range in the virtual scene; the device further includes a second update module, which is used to update the illumination information collected by the reflection probe when the update conditions of the reflection probe are met, to obtain updated illumination information; wherein, the update conditions include at least one of the following: the relative positional relationship between the parallax-corrected reflection probe and the virtual objects within a target range in the virtual scene changes; the light at the virtual objects within the target range in the virtual scene changes; the acquisition module 4552 is further used to obtain illumination information of the reflected light of the first virtual object from the updated illumination information.

[0230] In some embodiments, the lighting information includes color information obtained by coloring virtual objects within the target range based on texture information of virtual objects within the target range; the second update module is further configured to update the texture information corresponding to the lighting information to obtain new texture information when the update condition is that the relative position relationship changes; update the color information based on the new texture information to obtain new color information, and use the new color information as the updated lighting information.

[0231] In some embodiments, the lighting information includes color information obtained by coloring virtual objects within the target range based on texture information of virtual objects within the target range; the second update module is further configured to acquire the texture information corresponding to the lighting information when the update condition is that the light changes; update the color information based on the texture information to obtain new color information, and use the new color information as the updated lighting information.

[0232] In some embodiments, the second updating module is further configured to obtain a texture image of a virtual object within the target range in the virtual scene based on the texture information, and determine the color value of each pixel in the texture image based on the texture image and the light at the virtual object; and use the color value of each pixel as the new color information.

[0233] In some embodiments, the shape of the reflective probe is a polyhedron, and different faces of the polyhedron record local lighting information in the virtual scene. The local lighting information recorded by different faces is different, and the lighting information includes the local lighting information recorded by each face. The second update module is further configured to determine the update period of the reflective probe. The update period is used to indicate the minimum update interval between any two faces of the polyhedron. Based on the update period, the local lighting information recorded by each face of the polyhedron is updated sequentially to obtain the updated lighting information.

[0234] In some embodiments, the device further includes a completion module, which is used to obtain the acquisition direction of the reflection probe after parallax correction for acquiring illumination information at the intersection point corresponding to the reflected light; wherein the intersection point corresponding to the reflection probe is the intersection position of the reflected light of the first virtual object and the second virtual object in the virtual scene; if there is an obstruction in the acquisition direction, the illumination information acquired by the parallax-corrected reflection probe is completed; the acquisition module 4552 is also used to acquire the illumination information of the reflected light obtained by completion.

[0235] In some embodiments, the device further includes a second determining module, which is configured to construct a spatial coordinate system of the virtual scene and determine the depth of the reflective probe in the acquisition direction in the spatial coordinate system; obtain a third coordinate of the reflective probe in the spatial coordinate system and obtain a fourth coordinate of the intersection point in the spatial coordinate system; determine the distance between the reflective probe and the intersection point based on the third coordinate and the fourth coordinate; and determine that there is an obstruction to the reflective probe in the acquisition direction when the distance is greater than the depth of the reflective probe in the acquisition direction.

[0236] In some embodiments, the completion module is further configured to adjust the position of the reflection probe to a target position; wherein, the reflection probe at the target position has no obstruction in the acquisition direction for the intersection point, and the target position is used to indicate the position where the reflection probe can acquire the illumination information for the intersection point; and to acquire the illumination information of the reflected light of the first virtual object acquired by the reflection probe at the target position.

[0237] In some embodiments, the completion module is further configured to mark the intersection point to obtain a mark for indicating the intersection point; obtain the lighting information library of the virtual scene, the lighting information library being used to store lighting information in the virtual scene; and sample the lighting information at the intersection point from the lighting information library based on the mark.

[0238] This application provides a computer program product comprising computer-executable instructions or a computer program stored in a computer-readable storage medium. An electronic device's processor reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the image rendering method described in this application.

[0239] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the image rendering method provided in this application. For example, ... Figure 3 The image rendering method shown.

[0240] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or CD-ROM, etc.; or it may be a device that includes one or any combination of the above-mentioned memories.

[0241] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0242] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0243] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0244] In summary, the embodiments of this application have the following beneficial effects:

[0245] (1) Compared with the reflection scheme based on the reflection probe, this application corrects the acquisition position of the illumination information by performing parallax correction on the reflection probe, thereby improving the accuracy of the acquired illumination information and thus improving the rendering effect of the reflection image; and compared with the reflection scheme based on the emission ray, it improves the accuracy of the acquired illumination information, thereby improving the rendering effect of the reflection image.

[0246] (2) When the position of the virtual camera changes, the position of the reflection probe is updated synchronously. This ensures that all lighting information within the view of the virtual camera can be collected through the reflection probe, thus avoiding the reflection probe from missing the lighting information that should be collected, thereby improving the rendering effect of the reflection image obtained based on the lighting information. At the same time, compared with the related technologies that require artists to manually adjust the position of the reflection probe and place a large number of reflection probes in the virtual scene, this not only improves the rendering effect, but also reduces labor costs and resource consumption.

[0247] (3) When the relative positional relationship between the reflection probe and the virtual object within the target range in the virtual scene changes, and / or the light at the virtual object within the target range in the virtual scene changes, the lighting information collected by the reflection probe is updated, thereby ensuring the accuracy of the lighting information collected by the reflection probe, which in turn improves the effect of the reflection image rendered based on the reflection probe.

[0248] (4) The update of the reflection probe is spread across multiple frames, which avoids the stuttering problem caused by updating the reflection probe and can update the reflection probe without reducing the smoothness.

[0249] (5) When there is an obstruction in the acquisition direction, the acquired lighting information is supplemented so that the reflection probe can acquire the lighting information blocked by the obstruction. This improves the comprehensiveness of the lighting information acquired by the reflection probe and avoids the problem of poor rendering effect of the reflection image due to missing lighting information, which improves the rendering effect of the reflection image.

[0250] It should be noted that in the embodiments of this application, data such as the attribute information of the pre-set reflective probe are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0251] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An image rendering method, characterized in that, The method includes: Based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, parallax correction is performed on the reflection probes set in the virtual scene; The illumination information of the reflected light from the first virtual object is collected using the parallax-corrected reflection probe. Obtain the texture image of the first virtual object, and determine the color value of the image pixel based on the texture image and the lighting information; Based on the color values ​​of the image pixels, the reflection image of the first virtual object is rendered.

2. The method as described in claim 1, characterized in that, The parallax correction of the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera includes: Based on the position of the first virtual object and the viewpoint of the virtual camera, the intersection point of the reflected light from the first virtual object and the second virtual object in the virtual scene is determined. A first direction is determined from the position of the reflection probe pointing to the intersection, and the direction of the illumination information of the reflected light from the first virtual object collected through the reflection probe is adjusted to the first direction.

3. The method as described in claim 2, characterized in that, Before performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, the process includes: Construct a spatial coordinate system for the virtual scene, and determine the first coordinates of the reflection probe within the spatial coordinate system; Determining the first direction from the position of the reflective probe to the intersection includes: Obtain the second coordinates of the intersection point in the spatial coordinate system; Subtracting the first coordinate from the second coordinate yields the vector pointing from the reflection probe to the intersection point, and the direction indicated by the vector is determined as the first direction.

4. The method according to any one of claims 1 to 3, characterized in that, Before performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, the method further includes: Obtain the position of the virtual camera in the virtual scene; The reflection probe is generated based on the position of the virtual camera.

5. The method as described in claim 4, characterized in that, After generating the reflection probe based on the position of the virtual camera, the method further includes: The position of the virtual camera is detected; In response to the virtual camera's position changing from a first position to a second position, the position of the reflection probe is updated based on the second position of the virtual camera.

6. The method as described in claim 5, characterized in that, The step of updating the position of the reflection probe in response to a change in the position of the virtual camera from a first position to a second position, based on the second position of the virtual camera, includes: In response to the virtual camera's position changing from a first position to a second position, the difference between the first position and the second position is determined; In response to the difference reaching a difference threshold, the position of the reflective probe is updated.

7. The method as described in claim 4, characterized in that, The virtual scene is divided into at least two grid cells. After generating the reflection probe based on the position of the virtual camera, the method further includes: The position of the virtual camera is detected; In response to the virtual camera changing from being in the first grid cell to being in the second grid cell, the position of the reflective probe is updated based on the second grid cell.

8. The method as described in claim 4, characterized in that, The virtual scene is divided into at least two grid cells, and obtaining the position of the virtual camera in the virtual scene includes: Obtain the grid cell where the virtual camera is located in the virtual scene; After generating the reflection probe based on the position of the virtual camera, the method further includes: Obtain the duration for which the virtual camera is in the grid cell; When the duration reaches the target duration, the position of the reflection probe is updated.

9. The method according to any one of claims 1 to 8, characterized in that, The parallax-corrected reflection probe records the lighting information of virtual objects within the target range in the virtual scene; Before acquiring the illumination information of the reflected light from the first virtual object using the parallax-corrected reflection probe, the method further includes: When the update condition of the reflection probe is met, the illumination information collected by the reflection probe is updated to obtain updated illumination information; wherein, the update condition includes at least one of the following: The relative positional relationship between the parallax-corrected reflection probe and the virtual objects within the target range in the virtual scene changes; The light at virtual objects within the target area in the virtual scene changes; The process of collecting illumination information of reflected light from the first virtual object using a parallax-corrected reflection probe includes: From the updated lighting information, obtain the lighting information of the reflected light from the first virtual object.

10. The method as described in claim 9, characterized in that, The lighting information includes color information obtained by coloring the virtual objects within the target range based on the texture information of the virtual objects within the target range; The step of updating the illumination information collected by the reflection probe to obtain updated illumination information includes: When the update condition is that the relative positional relationship changes, the texture information corresponding to the lighting information is updated to obtain new texture information; Based on the new texture information, the color information is updated to obtain new color information, and the new color information is used as the updated lighting information.

11. The method as described in claim 9, characterized in that, The lighting information includes color information obtained by coloring the virtual objects within the target range based on the texture information of the virtual objects within the target range; The step of updating the illumination information collected by the reflection probe to obtain updated illumination information includes: When the update condition is that the light changes, the texture information corresponding to the lighting information is obtained; Based on the texture information, the color information is updated to obtain new color information, and the new color information is used as the updated lighting information.

12. The method according to any one of claims 9 to 11, characterized in that, The reflective probe is in the shape of a polyhedron. Different faces of the polyhedron record local lighting information in the virtual scene. The local lighting information recorded by different faces is different. The lighting information includes the local lighting information recorded by each face. The step of updating the illumination information collected by the reflection probe to obtain updated illumination information includes: Determine the update cycle of the reflection probe; The update period is used to indicate the minimum update interval between any two faces of the polyhedron; Based on the update cycle, the local illumination information recorded on each face of the polyhedron is updated sequentially to obtain the updated illumination information.

13. The method according to any one of claims 1 to 12, characterized in that, After performing parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera, the method further includes: The acquisition direction for collecting illumination information by the reflection probe at the intersection point corresponding to the reflected light rays after parallax correction is obtained; Wherein, the intersection point corresponding to the reflection probe is the intersection position of the reflected light of the first virtual object and the second virtual object in the virtual scene; If there is an obstruction in the acquisition direction, the illumination information acquired by the parallax-corrected reflection probe is supplemented. The process of collecting illumination information of reflected light from the first virtual object using a parallax-corrected reflection probe includes: Obtain the illumination information of the reflected light obtained through completion.

14. The method as described in claim 13, characterized in that, Before completing the illumination information acquired by the parallax-corrected reflection probe, the method further includes: Construct a spatial coordinate system for the virtual scene, and determine the depth of the reflective probe in the acquisition direction within the spatial coordinate system; Obtain the third coordinate of the reflective probe in the spatial coordinate system, and obtain the fourth coordinate of the intersection point in the spatial coordinate system; determine the distance between the reflective probe and the intersection point based on the third coordinate and the fourth coordinate; When the distance is greater than the depth of the reflective probe in the acquisition direction, it is determined that there is an obstruction to the reflective probe in the acquisition direction.

15. The method as described in claim 13 or 14, characterized in that, The process of completing the illumination information collected by the parallax-corrected reflection probe includes: Adjust the position of the reflective probe to the target position; Wherein, the reflective probe at the target location has no obstruction in the acquisition direction for the intersection, and the target location is used to indicate the position where the reflective probe can acquire the illumination information for the intersection; The illumination information of the reflected light from the first virtual object is obtained by the reflection probe at the target location.

16. The method as described in claim 13 or 14, characterized in that, The process of completing the illumination information collected by the parallax-corrected reflection probe includes: The intersection points are marked to obtain markers used to indicate the intersection points; Obtain the lighting information library of the virtual scene, wherein the lighting information library is used to store the lighting information in the virtual scene; Based on the marker, the illumination information at the intersection is sampled from the illumination information database.

17. An image rendering apparatus, characterized in that, The device includes: The correction module is used to perform parallax correction on the reflection probes set in the virtual scene based on the position of the first virtual object in the virtual scene and the viewpoint of the virtual camera. The acquisition module is used to acquire the illumination information of the reflected light from the first virtual object through the parallax-corrected reflection probe; The determining module is used to acquire the texture image of the first virtual object and determine the color value of the image pixel based on the texture image and the lighting information; The rendering module is used to render the reflection image of the first virtual object based on the color values ​​of the image pixels.

18. An electronic device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the image rendering method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, It stores computer-executable instructions or computer programs for inducing a processor to execute and implement the image rendering method according to any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the image rendering method according to any one of claims 1 to 16 is implemented.

Images