Rendering method and apparatus

Abstract

A rendering method and apparatus, which are used for reducing the service latency of end-cloud collaborative rendering. The method comprises: a first apparatus sending to a server information for describing a first video scene; the first apparatus receiving from the server a first lighting parameter of the first video scene; the first apparatus acquiring user perspective information for indicating a user perspective; on the basis of the user perspective information and the first lighting parameter, the first apparatus determining a second lighting parameter of the first video scene at the user perspective; and the first apparatus performing shading on the basis of the second lighting parameter.

Description

A rendering method and apparatus

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510201496.7, filed on February 24, 2025, entitled "A Rendering Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of image rendering technology, and in particular to a rendering method and apparatus. Background Technology

[0004] In fields involving computer graphics, such as film, animation, and games, ray tracing technology is commonly used to render images. Ray tracing can realistically simulate lighting effects in the real world and can handle complex lighting scenes, such as indoor, outdoor, and natural landscapes. Therefore, ray tracing technology has been widely applied in the field of computer graphics.

[0005] Due to power consumption and manufacturing process limitations, the computing power of terminal devices has increased slowly, and the high frame rate and high image quality rendering requirements of services such as mobile games cannot be met locally on the terminal device. Therefore, edge-cloud collaborative rendering introduces cloud-side devices to support rendering, ensuring high-quality rendering requirements through cloud-side computing power, while the terminal device is generally only used to display the rendered image from the cloud side.

[0006] However, the real-time performance of edge-cloud collaborative rendering faces challenges and is easily affected by network quality fluctuations, leading to service latency. Reducing the latency of edge-cloud collaborative rendering is a pressing technical problem that needs to be solved. Summary of the Invention

[0007] This application provides a rendering method and apparatus to reduce the service latency of edge-cloud collaborative rendering.

[0008] Firstly, a rendering method is provided to reduce the latency of edge-cloud collaborative rendering. This method can be executed by a first device. The first device is, for example, a device that supports rendering and / or display, or a component within that device. The device can be a terminal device or other electronic device. The component within the device can be a processor, a chip, or a chip system, etc. Taking the first device as the executing entity as an example, the method may include:

[0009] The first device sends information describing a first video scene to the server; the first device receives first lighting parameters of the first video scene from the server; the first device obtains user view information indicating the user's viewpoint; the first device determines second lighting parameters of the first video scene under the user's viewpoint based on the user view information and the first lighting parameters; the first device performs coloring based on the second lighting parameters.

[0010] Based on the method described in the first aspect, the first device can send information describing the first video scene to the cloud server and obtain the intermediate rendering result, i.e., the first lighting parameters of the first video scene, from the cloud server. The first device can also, after obtaining user viewpoint information, determine second lighting parameters from the user viewpoint based on the user viewpoint information and the first lighting parameters, and perform shading according to the second lighting parameters to achieve rendering from the user viewpoint. It is evident that the first lighting parameters obtained by the first device from the cloud server are not determined based on user viewpoint information; that is, the step of the first device obtaining the first lighting parameters does not depend on the user's real-time operation. Therefore, it can reduce the network latency requirements of edge-cloud collaborative rendering, thereby reducing service latency.

[0011] In one possible implementation, acquiring user perspective information for indicating the user's perspective includes: acquiring the user perspective information in real time.

[0012] Based on this implementation, user perspective information can be collected in real time by the first device, meaning it changes dynamically according to the application scenario. This means the user's perspective doesn't need to be collected in real time during the stage of reporting the first video scene to the cloud server; subsequent rendering only requires combining the real-time user perspective information with the previously obtained first lighting parameters, thus reducing business latency. Optionally, the first lighting parameters can be obtained non-real-time. For example, the first device can obtain the first lighting parameters at relatively long time intervals and use the already obtained first lighting parameters in any rendering before the next time the first lighting parameters are obtained.

[0013] In one possible implementation, the user perspective information includes one or more of the following: user location parameters; and / or, user perspective parameters.

[0014] Based on this implementation, the first device can flexibly collect user position parameters and / or user view parameters to support the determination of second lighting parameters based on user position and / or view, and to perform rendering based on the second lighting parameters.

[0015] In one possible implementation, the first lighting parameter includes one or more of the following: indirect lighting parameter; ambient light occlusion parameter; direct lighting parameter; specular reflection parameter; refracted light parameter; volumetric effect parameter. Correspondingly, the second lighting parameter may include one or more of the following: indirect lighting parameter; ambient light occlusion parameter; direct lighting parameter; specular reflection parameter; refracted light parameter; volumetric effect parameter.

[0016] Based on this implementation method, the first device can perform one or more of the following rendering methods: indirect lighting, ambient occlusion, direct lighting, specular reflection, refraction, and volumetric effects, in order to obtain better rendering results.

[0017] Secondly, a rendering apparatus is provided. The apparatus can implement the method described in any possible implementation of the first aspect. The apparatus possesses the functions of the first apparatus described above. The apparatus is, for example, a terminal device, or a functional module within a terminal device. The terminal device can be a device that supports rendering and display functions. The terminal device can also be replaced with other electronic devices.

[0018] In one optional implementation, the device may include modules corresponding one-to-one with the methods / operations / steps / actions described in the first aspect and any possible implementation thereof. These modules may be hardware circuits, software, or a combination of hardware circuits and software. In another optional implementation, the device includes a processing unit (sometimes also called a processing module, rendering unit, rendering module, etc.) and a communication unit (sometimes also called a transceiver module, communication module, etc.). The transceiver unit is capable of both sending and receiving functions. When the transceiver unit performs the sending function, it may be called a sending unit (sometimes also called a sending module); when the transceiver unit performs the receiving function, it may be called a receiving unit (sometimes also called a receiving module). The sending unit and the receiving unit may be the same functional module, which is called the transceiver unit and can perform both sending and receiving functions; or, the sending unit and the receiving unit may be different functional modules, with the transceiver unit being a collective term for these functional modules.

[0019] For example, when the apparatus is used to perform the method described in the first aspect, the apparatus may include a communication unit and a processing unit.

[0020] Thirdly, embodiments of this application also provide a rendering apparatus, including a processor for executing a computer program (or computer-executable instructions) stored in a memory, which, when executed, causes the apparatus to perform the method described in the first aspect and any possible implementation thereof. The processor may include a graphics processing unit (GPU) and / or a central processing unit (CPU).

[0021] In one possible implementation, the processor and memory are integrated together;

[0022] In another possible implementation, the memory is located outside the rendering device.

[0023] The rendering apparatus also includes a communication interface for communicating with other devices, such as sending or receiving data and / or signals. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0024] Fourthly, a computer-readable storage medium is provided for storing a computer program or instructions that, when executed, cause the methods described in the first aspect and any possible implementation thereof, as well as the methods shown in any possible implementation thereof, to be implemented.

[0025] Fifthly, a computer program product containing instructions is provided that, when run on a computer, enables the methods described in the first aspect and any possible implementation thereof to be implemented.

[0026] Sixthly, embodiments of this application also provide a rendering apparatus for performing the method described in the first aspect and any possible implementation thereof.

[0027] In a seventh aspect, a chip system is provided, comprising logic circuitry (or, as understood, a processor, which may include logic circuitry, etc.), and further comprising input / output interfaces. The processor may include a graphics processor and / or a central processing unit. The input / output interface can be used to input messages or to output messages. The input / output interface may be the same interface, i.e., the same interface can implement both sending and receiving functions; or, the input / output interface includes an input interface and an output interface, the input interface being used to implement the receiving function, i.e., to receive messages; and the output interface being used to implement the sending function, i.e., to send messages. The logic circuitry can be used to perform operations other than the sending and receiving functions in the methods described in the first aspect and any possible implementation thereof; the logic circuitry can also be used to transmit messages to the input / output interface or to receive messages from other communication devices from the input / output interface. The chip system can be used to implement the methods described in the first aspect and any possible implementation thereof. The chip system may include one or more chips, or may include chips and other discrete devices.

[0028] Optionally, the chip system may also include a memory, which can be used to store instructions, and the logic circuits can call the instructions stored in the memory to implement the corresponding functions.

[0029] Eighthly, a rendering method is provided, which may include the method implemented by a first device and the actions performed by a cloud server as shown in the first aspect and any possible implementation thereof. For example, the cloud server may be used to send first lighting parameters to the first device after receiving information describing a first video scene.

[0030] A ninth aspect provides a rendering system that may include a first device and a second device. The first device is used to implement the method shown in the first aspect and any possible implementation thereof, and the second device is used to perform actions performed by a cloud server. The second device is, for example, a cloud server or a component within a cloud server. For example, the second device may be used to send first lighting parameters to the first device after receiving information describing a first video scene.

[0031] The technical effects brought about by the third to ninth aspects above can be found in the description of the beneficial effects of the corresponding solutions in the first aspect above, and will not be repeated here. Attached Figure Description

[0032] Figure 1 is a schematic diagram of the architecture of an end-to-cloud collaborative system provided in an embodiment of this application;

[0033] Figure 2 is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0034] Figure 3 is a flowchart illustrating a rendering method provided in an embodiment of this application;

[0035] Figure 4 is a schematic diagram of an irradiance representation method provided in an embodiment of this application;

[0036] Figure 5 is a schematic diagram of an embodiment of this application that uses an octahedral texture to represent the irradiance at different locations in space;

[0037] Figure 6 is a schematic diagram of a method for storing specular reflection illumination parameters by a probe according to an embodiment of this application;

[0038] Figure 7 is a schematic diagram of a method for determining the second illumination parameter in a specular reflection scenario provided by an embodiment of this application;

[0039] Figures 8 and 9 are schematic diagrams of the terminal device-side actions and cloud server-side actions of a rendering method provided in an embodiment of this application, respectively.

[0040] Figures 10 and 11 are schematic diagrams of the structure of a rendering device provided in an embodiment of this application. Detailed Implementation

[0041] In this application embodiment, "multiple" refers to two or more. Therefore, in this application embodiment, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, such as one, two, or more. For example, "including at least one" means including one, two, or more, and it does not limit which ones are included. For example, including at least one of A, B, and C, then it could include A, B, C, A and B, A and C, B and C, or A and B and C. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0042] The terms "comprising" and "having," and any variations thereof, used in the following description of embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should be noted that in embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any method or design described as "exemplary" or "for example" in embodiments of this application should not be construed as preferred or advantageous over other methods or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0043] Unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, sequence, priority, or importance of the multiple objects. To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only used to explain specific embodiments of this application and is not intended to limit this application.

[0044] This application will present various aspects, embodiments, or features relating to systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that individual systems may include additional devices, components, modules, etc., and / or may not include all the devices, components, modules, etc. discussed in conjunction with the accompanying drawings. Furthermore, combinations of these approaches are also possible.

[0045] Furthermore, in the embodiments of this application, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as an "example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "example" is intended to present concepts in a concrete manner. In the embodiments of this application, "of," "corresponding, relevant," and "corresponding" may sometimes be used interchangeably, and it should be noted that their intended meanings are consistent unless their distinction is emphasized.

[0046] Before introducing the specific solutions provided in the embodiments of this application, some terms used in this application will be explained to facilitate understanding by those skilled in the art, but the terms used in this application are not limited.

[0047] (1) Ray Tracing: Ray tracing is used to simulate the process of light propagation from a camera to an object in a scene, and then the interaction of light with reflection, refraction, scattering, etc. Ray tracing takes into account the interaction between light and objects, and can produce realistic shadows, reflections, refractions, and other effects. It can simulate all the natural phenomena of interaction between object surfaces in the real physical world.

[0048] (2) Ray intersection, that is, determining the intersection point of the ray with the object in the scene. The more complex the scene and the more objects in the scene, the longer the ray intersection will take.

[0049] (3) Global illumination (GI) is a rendering technique that considers both direct light from light sources in the scene and indirect light reflected from objects in the scene. In other words, global illumination can be understood as the superposition of direct lighting rendering and indirect lighting rendering.

[0050] (4) Dynamic diffuse global illumination (DDGI) is a probe-based GI algorithm that generates diffuse global illumination in real time within dynamic scenes and lighting conditions. It utilizes ray tracing technology to collect lighting parameters. For example, lighting parameters include irradiance and / or distance.

[0051] In DDGI technology, the acquired illumination parameters can be stored on a regular probe grid. The probe grid can be considered a grid consisting of probes arranged in a certain regular pattern. For example, each grid point can be considered a probe, and the probe position can correspond to the acquired illumination parameters. Alternatively, the probe can be considered to store the illumination parameters at that position.

[0052] Further shading can be performed. Shading can refer to introducing light and shadow and color into a pattern, or applying different materials to different objects, as different materials have different effects on light. In DDGI, shading can refer to transferring color to the screen. When shading a point in space (called a shading point), the DDGI algorithm can find one or more probes around that point and use interpolation and other algorithms to calculate the lighting parameters at the shading point using the lighting parameters stored in these probes. This interpolation method allows DDGI to reduce computational costs while maintaining high rendering quality.

[0053] DDGI can achieve lighting rendering based on direct light and indirect light within the scene.

[0054] (4-1) Direct lighting is generally divided into two categories: light source and ambient light. Light sources generally include: parallel light, point light source, spot light source, area light source, etc., while ambient light, which is direct lighting, generally refers to image-based lighting (IBL).

[0055] (4-2) Indirect lighting mainly considers the light reflected from objects within the scene. Specifically, indirect lighting refers to light that has been reflected, refracted, and scattered multiple times within the scene before reaching objects. Indirect lighting is primarily used to simulate the multiple reflections and scattering effects of natural and artificial light sources within a scene, enhancing the scene's three-dimensionality and realism.

[0056] Unless otherwise specified, "rendering" in this application may refer to GI rendering or other types of lighting rendering, or DDGI rendering or other types of rendering.

[0057] (5) End-to-end cloud collaborative rendering and end-to-end cloud collaborative rendering system

[0058] Considering the large computational demands of image rendering and the limited computing resources and capabilities of terminal devices such as mobile phones and tablets, which make it difficult to support image rendering in complex scenes, an edge-cloud collaborative rendering mechanism is introduced. This mechanism leverages the resources of cloud-side devices to assist terminal devices in rendering, thereby improving rendering quality. Edge-cloud collaborative rendering achieves image rendering through the collaboration of cloud-side devices and terminal devices, and the terminal devices can ultimately present the rendered images to the user.

[0059] As an example, Figure 1 illustrates an image rendering system that may include cloud-side devices and terminal devices. Figure 1 exemplarily shows multiple terminal devices; in some embodiments, the cloud-side device may be connected to a single terminal device.

[0060] In this application, cloud-side equipment and terminal equipment can communicate via a network. This network can comply with the communication standards of the Third Generation Partnership Project (3GPP). For example, the mobile communication network can include Universal Mobile Telecommunications System (UMTS), Wireless Local Area Network (WLAN), Wireless Fidelity (Wi-Fi) system, 4th generation (4G) mobile communication system (such as Long Term Evolution (LTE) system), 5th generation (5G) mobile communication system (such as New Radio (NR) system), or future communication systems, etc.

[0061] Cloud-side devices can be carriers of cloud computing resources and can be used to perform cloud computing through these resources. In this application, cloud computing may include cloud-side rendering operations; for details, please refer to the description of the functions of cloud-side devices in the embodiments of this application, which will not be elaborated here. Specifically, cloud-side devices may be cloud servers, cloud platforms, computers deployed in the cloud, or other electronic devices. In the following description, unless otherwise specified, a cloud server will be used as an example of a cloud-side device.

[0062] The terminal device in this application may have real-world functions, such as a display screen or other display components. Furthermore, the terminal device may be connected to the display screen or other display components via wired and / or wireless means; for example, the terminal device may be connected to an external display module.

[0063] For example, the terminal device may be a non-foldable screen mobile phone, a foldable screen mobile phone, a wearable device (such as a smartwatch, smart bracelet, etc.), a tablet computer, a laptop computer, a smart screen, an in-vehicle terminal, a computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a virtual reality (VR) device / augmented reality (AR) device, an artificial intelligence (AI) device, or any other device with computing and display functions. It may also be a server or other similar device; this application does not limit the scope. A smart screen is also called a large-screen device, such as an intuitive color TV or a rear-projection TV with a large screen.

[0064] Figure 2 is a schematic diagram of a terminal device 100. As can be seen, the terminal device 100 may include a processor 110 and off-chip memory 120. The processor 110 and off-chip memory 120 can be connected via a bus 130. The bus 130 may be a data bus, and the processor 110 may include one or more processing units and on-chip memory.

[0065] In some embodiments, as shown in FIG1, the processing unit may include a central processing unit 111 and a graphics processing unit 112. In other embodiments, in addition to including CPU 111 and GPU 112, the processing unit may also include one or more of the following processors: application processor (AP), image signal processing unit (ISP), microprocessor (MP), controller, video codec, baseband processor, embedded neural network processing unit (NPU) in the field of AI, or digital signal processor (DSP). Different processors may be independent devices, such as separate chips, or they may be integrated into the same chip.

[0066] On-chip memory 113 is located inside processor 110 and can be used to store instructions and data. On-chip memory 113 in processor 110 may include one or more of the following: cache, tile buffer, texture buffer, shared cache, or register. Cache is a type of static random access memory (SRAM). CPU 111, GPU 112, and other processing units can access on-chip memory 113. On-chip memory 113 can be used to store instructions or data that have just been used or are being used repeatedly by CPU 111 or GPU 112. Depending on whether it stores instructions or data, on-chip memory 113 can also be divided into instruction cache and data cache. Compared to accessing off-chip memory 120, the time taken for CPU 111 or GPU 112 to access on-chip memory 113 is very short. Therefore, if CPU 111 or GPU 112 needs to use the instruction or data again, it can be directly called from on-chip memory 113, reducing the waiting time of CPU 111 or GPU 112.

[0067] The off-chip memory 120 is located outside the processor 110 and can be the internal memory of the terminal device 100, i.e., the main memory of the terminal device 100. The off-chip memory 120 can be random access memory (RAM). For example, the off-chip memory 120 can include one or more of dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM).

[0068] The off-chip memory 120 can also be used to store computer program instructions and data. The off-chip memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function (such as a camera application or a game application). The data storage area may store data created during the operation of the terminal device 100 (such as image data generated during image rendering). The processor 110 can read data from the off-chip memory 120 and write data to the off-chip memory 120 via the bus 130.

[0069] As an example of edge-cloud collaborative rendering, in current edge-cloud collaborative DDGI rendering, the terminal device can provide scene information and user perspective information to the cloud server. The cloud server then performs DDGI based on the scene information and user perspective information to obtain the rendering result corresponding to that perspective information, such as a shaded image. The cloud server further provides the rendering result to the terminal device via the network, and the terminal device can display the image shown in the rendering result.

[0070] For example, in a game scenario, the process of edge-cloud collaborative rendering may include the following steps:

[0071] S0: The terminal device determines scene information and user perspective information. Scene information can be a game scene, and user perspective information can include the location, orientation, and / or lens parameters of the user's game character or camera.

[0072] S1: The terminal device provides scene information and user perspective information to the cloud server.

[0073] S2: Based on scene information and user perspective information, the cloud server uses DDGI technology to render a local area of ​​the scene corresponding to the user's perspective information, using both direct and indirect lighting, to obtain a local rendering result. This local rendering result can be used as the rendered image.

[0074] S3: The cloud server sends the rendering results to the terminal device.

[0075] S4: The terminal device displays the rendering results.

[0076] As can be seen, in current edge-cloud collaborative rendering, terminal devices need to collect and provide the scene information to be rendered and the user's perspective information to the cloud server in real time, and then display the rendered image based on the rendering results from the cloud server. Since the user's perspective information changes in real time with the user's commands—for example, in a game scene, the user's perspective information changes in real time with the user's character's position or viewpoint—terminal devices need to collect and provide this information to the cloud server in real time based on user actions. To ensure timely display of the rendering results from the current user's perspective, current edge-cloud collaborative rendering has high real-time requirements. In situations with fluctuating network quality, edge-cloud collaborative rendering is prone to rendering latency, impacting the user experience.

[0077] In conclusion, reducing the latency of edge-cloud collaborative rendering is a technical problem that urgently needs to be solved.

[0078] To address the aforementioned technical problems, this application provides a rendering method. This method can be executed by a cloud server and a terminal device. Specifically, the actions of the cloud server can be replaced by modules or chips within the cloud server, and the actions of the terminal device can also be replaced by modules or chips within the terminal device. Furthermore, the cloud server can also be replaced by a cloud-side device.

[0079] The rendering method provided in this application will be described below with reference to Figure 3. This method can be executed by a cloud server and a terminal device.

[0080] S101: The terminal device sends information describing the first video scene.

[0081] Accordingly, the cloud server receives information describing the first video scene.

[0082] S102: The cloud server sends the first illumination parameters of the first video scene to the terminal device based on the information used to describe the first video scene. Correspondingly, the terminal device can receive the first illumination parameters of the first video scene from the cloud server.

[0083] S103: The terminal device acquires user perspective information used to indicate the user's perspective.

[0084] In this application, the user perspective information can be the user's perspective information in the first video scene collected by the terminal device. The user perspective information can be considered dynamic or real-time information. For example, the terminal device can acquire real-time user perspective information in the first video scene through sensors, such as a camera or touchscreen. That is, the user perspective information changes dynamically according to the application scenario and is collected in real-time by the terminal device's sensors.

[0085] S104: The terminal device determines the second illumination parameter from the user's perspective based on the user's viewpoint information and the first illumination parameter.

[0086] S105: The terminal device performs coloring according to the second illumination parameters.

[0087] Based on the method shown in Figure 3, the terminal device can send information describing the first video scene to the cloud server and obtain the intermediate rendering result, i.e., the first lighting parameters of the first video scene, from the cloud server. After obtaining the user's viewpoint information, the terminal device can also determine the second lighting parameters under the user's viewpoint based on the user's viewpoint information and the first lighting parameters, and perform shading based on the second lighting parameters to achieve rendering under the user's viewpoint.

[0088] As can be seen, in the process shown in Figure 3, the first illumination parameters obtained by the terminal device from the cloud server are not determined based on the user's perspective information. That is, the execution of S101 to S103 does not depend on the user's real-time operation. Only the execution of S104 requires the terminal device to collect the user's real-time operation and obtain the user's perspective information within the first video scene based on that operation. Therefore, real-time rendering can be achieved by executing S104 to S105 only after obtaining the real-time user perspective information. In contrast, S0 to S2 all rely on the collection of user perspective information, and the rendering steps after obtaining the user's perspective information in real time are more numerous than S104 and S105, placing higher demands on network latency. Therefore, compared to the process shown in S0 to S2, the scheme shown in Figure 3 can reduce the network latency requirements of edge-cloud collaborative rendering, thus enhancing the rendering service's resilience to network fluctuations.

[0089] In addition, the global illumination parameters in the first video scene in S102 are not intermediate rendering results for the terminal device, but can be applied to multiple terminals and / or users with the same information to describe the first video scene, which can further reduce rendering overhead.

[0090] The following describes each step in the process shown in Figure 3.

[0091] In this application, the first video scene can be a scene of an application running on the terminal device. This application can be a game, film / television, AR, VR, or other application with rendering requirements. Specifically, the first video scene can be a scene in a display screen with rendering requirements, such as a game, image (e.g., a picture or video frame), AR, or VR. For example, the first video scene can be a game scene. The terminal device can obtain information describing the first video scene in the game based on the user's game operations. The first video scene can include one or more objects within a certain spatial range; this space can be a virtual space or an image or effect corresponding to a physical space.

[0092] For example, the first video scene displayed to the user by the terminal device through the screen can be regarded as the image captured by the camera. The first video scene is also the three-dimensional scene in which the camera is located.

[0093] In one possible embodiment, the terminal device can obtain information describing a first video scene based on a first operation performed by the user in the application. The first operation may include entering the first video scene and / or editing the first video scene.

[0094] In this application, the information used to describe the first video scene may also be called video scene description information or scene information, etc., without specific limitation.

[0095] In one possible implementation, the information used to describe the first video scene may include one or more of the following:

[0096] (1) Information used to identify the first video scene, also known as the identifier of the first video scene.

[0097] The identifier for the first video scene can include at least one of the following: the scene's number, name, or location information. For example, if the first video scene is a game scene, the terminal device can obtain this identifier when the user enters the first video scene.

[0098] (2) User's editing information for the first video scene.

[0099] The editing information reflects the user's editing operations on objects within the first video scene. For example, the editing information might include actions such as adding, deleting, moving, or modifying objects within the first video scene. Therefore, the editing information can be used to indicate the type and location of objects within the first video scene after editing.

[0100] The information described above for the first video scene is exemplary, and this application does not limit the information to the content of the above example.

[0101] Optionally, the information used to describe the first video scene can be understood as static information of the first video scene. That is, the information used to describe the first video scene does not change with changes in the user or terminal device; in other words, the information used to describe the first video scene is non-real-time information. For example, multiple users or terminal devices can use the same identifier to indicate the first video scene. Furthermore, for multiple users or terminal devices, when performing the same editing operation on an object, the editing information can be the same.

[0102] In S102, the first lighting parameter may be a lighting parameter obtained based on information used to describe the first video scene. This lighting parameter can be understood as a static lighting parameter of the first video scene or a lighting parameter with a low frequency of change.

[0103] In this application, "static" and "dynamic" are relative terms. Static can be considered as state information that is independent of the user's perspective or does not need to change in real time; that is, static lighting parameters are lighting parameters that are independent of the user's perspective or do not need to change in real time. Static lighting parameters can also be considered as lighting parameters that change at a low frequency.

[0104] Specifically, the first lighting parameter may include one or more of the following: direct lighting parameters, indirect lighting parameters, specular reflection lighting parameters, refractive lighting parameters, ambient occlusion (AO) parameters, and volumetric effect parameters of the first video scene.

[0105] Alternatively, the rendering result of the first video scene in this application can be considered as a superposition of one or more of the rendering results of direct lighting, indirect lighting, specular lighting, refraction lighting, AO and volumetric effects, thus enabling lighting rendering of one or more of the following: direct lighting, indirect lighting, specular lighting, refraction lighting, AO and volumetric effects.

[0106] In this application, direct lighting parameters may include lighting parameters used to indicate direct lighting at one or more locations in a first video scene. These lighting parameters include, for example, irradiance and / or radiance. Irradiance refers to the power received per unit illuminated area. Radiance refers to the power emitted, reflected, transmitted, or received per unit solid angle, per unit time, and per unit vertical area, i.e., the color and brightness perceived by the human eye. Power can also be called radiant flux, which refers to the energy released, reflected, projected, or received per unit time. As an example, direct lighting may include light source illumination and / or ambient light illumination; therefore, direct lighting parameters may include lighting parameters of light sources at one or more locations and / or ambient light illumination parameters. The lighting parameters of light sources may include the irradiance and / or radiance of the light source, and the lighting parameters of ambient light may include the irradiance and / or radiance of the ambient light.

[0107] Indirect lighting parameters may include lighting parameters used to indicate indirect lighting on the surfaces of objects in the first video scene. As an example, indirect lighting parameters may include lighting parameters of light that has been reflected, refracted, and scattered in the scene before reaching the objects.

[0108] Specular reflection refers to the process where, after light strikes an object's surface, a portion of the light is reflected back to the light source or camera. Specular reflection typically involves only one reflection: light emitted from the light source strikes the object's surface and is directly reflected back to the light source or captured by the camera. For example, specular reflection exists on the surface of water, glass, or other mirror-like objects. Specular reflection lighting parameters are primarily used to simulate the glossy effect of smooth surfaces, enhancing the texture and detail of objects. Specular reflection lighting parameters can include lighting parameters describing the illumination of other object surfaces by light reflected from a mirror-like object surface in a first video scene. For example, specular reflection lighting parameters can include the irradiance and / or emissivity of light reflected from a mirror-like object surface onto other object surfaces.

[0109] Refraction refers to the phenomenon where the direction of light changes when it travels from one medium to another. In rendering, refraction can be used to simulate the effect of transparent or translucent objects, such as water or glass. Refracted light parameters are primarily used to simulate the transparency of transparent or translucent objects, such as water and glass, increasing the realism of the scene. Refracted lighting parameters can include lighting parameters describing the illumination of other object surfaces by light refracted from transparent or translucent objects within the first video scene. For example, refraction lighting parameters can include the irradiance and / or emissivity of light refracted from transparent or translucent objects within the first video scene on other object surfaces.

[0110] Ambient Occlusion (AO) simulates the effect of light being occluded between object surfaces in a scene. Due to the occlusion effect around objects, ambient light can only partially reach the objects, creating the AO effect. AO parameters can include AO-related parameters describing the relationships between objects in the first video scene and / or lighting parameters at the occluded location. AO parameters may include, for example, markers representing the occlusion location and / or coefficients representing the impact of occlusion on illuminance (referred to as AO impact coefficients). The lighting parameters at the occluded location can be determined by the lighting parameters before AO correction and / or the AO parameters. The lighting parameters before AO correction may include one or more of the following: direct lighting parameters, indirect lighting parameters, specular reflection lighting parameters, or refraction lighting parameters at the occluded location before AO correction.

[0111] Volumetric effects are properties of light passing through matter, not the light source itself. Volumetric effect rendering can be used to achieve the rendering effect of light absorption by irregularly shaped translucent materials. These materials could be, for example, fire, smoke, fog, or glass. Volumetric effect parameters can include translucency parameters and / or volumetric effect lighting parameters describing the irregularly shaped translucent material within the first video scene. For example, translucency parameters include the irradiance, emissivity, and / or transmittance coefficient of the absorbed light. Volumetric effect lighting parameters can include the emissivity and / or irradiance of the volumetric effect lighting. Volumetric effect lighting parameters can be determined based on the lighting parameters before volumetric effect correction, the irradiance, emissivity, and / or transmittance coefficient of the absorbed light from the translucent material. The lighting parameters before volumetric effect correction may include one or more of the following: direct lighting parameters, indirect lighting parameters, specular reflection lighting parameters, or refractive lighting parameters at that location.

[0112] Unless otherwise specified below, emissivity can be the emissivity of at least one of the following lighting parameters: direct lighting parameter, indirect lighting parameter, specular reflection lighting parameter, refracted lighting parameter, AO lighting parameter at the blocked location, and volume effect lighting parameter of the transparent material; in addition, irradiance can be the irradiance of at least one of the following lighting parameters: direct lighting parameter, indirect lighting parameter, specular reflection lighting parameter, refracted lighting parameter, AO lighting parameter at the blocked location, and volume effect lighting parameter of the transparent material.

[0113] The following section introduces irradiance and emissivity.

[0114] (1) Emissivity

[0115] It can be understood that the rendering equation based on radiometrics in space is described as follows: L(x o ,w o ) = L e (x o ,w o )+∫ Ω f(x o ,w o ,x i ,w i )L(x i ,w i (w) i ·n)dw i (1)

[0116] Here, L(x,w) represents emissivity, used to describe the radiant intensity at a location x along a unit solid angle direction w, per unit time, and per unit area. x, w, n, and Ω represent the position of the first video scene, the unit solid angle direction, the surface normal of the problem, and the sphere's area region, respectively. And L... e (x o ,w o Specifically refers to self-luminous emissivity, that is, the emissivity from position x. o Along w o The emissivity of light emitted in a given direction. f(x) o ,w o ,x i ,w i The incident ray (x) is a bidirectional distribution function used to describe how light interacts with the medium, i.e., the interaction between the incident ray (x) and the medium. i ,w i For the outgoing ray (x) o ,w o The contribution of ); the integral ∫ in the second half of the equation Ω f(x o ,w o ,x i ,w i )L(x i,w i (w) i ·n)dw i This indicates all incident rays (x) distributed along the spherical surface. i ,w i ) all affect the outgoing ray (x) o ,w o He made certain contributions.

[0117] In this example, the illumination parameters may include the emissivity L(x) satisfying the above formula (1). o ,w o It is not excluded that the emissivity can satisfy other formulas besides formula (1), or that the emissivity can be expressed by other formulas or forms besides formula (1), and this application does not impose any restrictions.

[0118] It can be understood that the emissivity L(x) shown in the above formula (1) is... o ,w o Considering various types of incident rays (x) i ,w i The emissivity of light on an object's surface, including one or more of the following: direct lighting, indirect lighting, specular reflection lighting, refracted lighting, AO lighting, or volumetric effect lighting. The emissivity is determined based on the incident ray (x...). i ,w i The different types of L(x) i ,w i L(x) can represent the emissivity of direct illumination, indirect illumination, specular reflection illumination, refracted illumination, AO illumination, or volumetric effect illumination. Therefore, different types of incident light will affect L(x). o ,w o The type of emissivity included. That is, the emissivity L(x) o ,w o The type of illumination can be one or more of the following: direct illumination, indirect illumination, specular reflection illumination, refracted illumination, AO illumination, or volumetric effect illumination.

[0119] (2) Irradiance

[0120] For diffuse reflective surfaces, considering that their bidirectional reflectance distribution function (BRDF) is constant and their emissivity is direction-independent, the illumination parameters do not need to cache the value of indirect illumination (i.e., emissivity); instead, they can cache the irradiance. For example, the irradiance I can satisfy: I=∫ Ω L(x i ,w i (w)i ·n)dw i (2)

[0121] Since the indirect lighting signal itself is relatively smooth, it is further smoothed after integration. During shading, the irradiance can be obtained by interpolation based on the shading point position, and then the indirect lighting of the diffuse surface can be approximately calculated according to the rendering equation. Combined with the results of direct lighting, the complete lighting can be reconstructed. As shown in Figure 4, in one example, the irradiance I at any position x in the first video scene in different directions w can be saved.

[0122] It is understandable that, according to the incident ray (x) i ,w i The different types of L(x) i ,w i I can represent the emissivity of direct lighting, indirect lighting, specular reflection lighting, refracted lighting, AO lighting, or volumetric effect lighting. Therefore, different types of incident light will affect the type of irradiance included in I. That is, the type of irradiance I can be the emissivity of one or more of the following: direct lighting, indirect lighting, specular reflection lighting, refracted lighting, AO lighting, or volumetric effect lighting.

[0123] It's understandable that lighting parameters for one or more of the following types of lighting—direct lighting, indirect lighting, specular lighting, refractive lighting, ambient oscillation (AO) lighting, or volumetric lighting—can include irradiance or radiance to reduce data volume. Furthermore, it's possible that the above lighting parameters include both irradiance and radiance to meet the needs of different scenarios. For example, some scenarios only require irradiance, while others require radiance. When lighting parameters include both irradiance and radiance, the rendering device can flexibly select either irradiance or radiance according to the requirements.

[0124] In one possible implementation, any one of the following illumination parameters—direct illumination, indirect illumination, specular reflection illumination, and refractive illumination—can include an emissivity value and / or an irradiance diagram. That is, the first illumination parameter may include an emissivity value and / or an irradiance diagram. The emissivity value, for example, indicates the emissivity that satisfies formula (1). The irradiance diagram, as shown in Figure 4, can be used to indicate the irradiance at different locations.

[0125] However, if the irradiance is cached directly in the illumination parameters as shown in Figure 4, the required resolution for caching is relatively large, resulting in a large amount of data for the first illumination parameter. This will increase the transmission bandwidth when the cloud server sends the first illumination parameter to the terminal device. At the same time, this caching is difficult to guarantee consistency when the object is deformed, and it is not an ideal representation method.

[0126] Therefore, in another possible implementation, to reduce the amount of data to be stored and the bandwidth required for transmission of the illumination parameters, any one of the following illumination parameters—direct illumination, indirect illumination, specular reflection illumination, and refractive illumination—can include radiance and / or irradiance represented by probes, voxels, surface cache information, or neural network cache information. In other words, the first illumination parameter can include radiance and / or irradiance represented by probes, voxels, surface cache information, or neural network cache information. That is, illumination parameters such as radiance and / or irradiance in the first illumination parameter can be stored through probe caching, voxel caching, surface caching, or neural network caching, or, in other words, radiance and / or irradiance can be stored through irradiance probes, voxels, surface cache information, or neural network cache information.

[0127] The following section uses irradiance as an example to introduce the methods of representing illumination parameters using probes, voxels, surface cache information, or neural network cache information. Correspondingly, the representation of emissivity in the first illumination parameter can be implemented similarly and will not be elaborated further.

[0128] (a) Probe

[0129] In probe caching, irradiance can be cached using a spatially sparse structure. Specifically, multiple probes can be distributed throughout the scene space, and probe irradiance data can be obtained by sampling at probe positions, then stored using an irradiance map. This method can be called the irradiance probe caching method.

[0130] The probe irradiance data may include the irradiance I′ at probe position x′ in different directions w′. Optionally, I′ may satisfy formula (2) or other expressions, without specific limitations.

[0131] For example, as shown in Figure 5, an octahedral texture can be used to store all irradiance along a spherical direction, thus avoiding polar distortion caused by the spherical texture and accelerating the calculation and acquisition of lighting data. Specifically, through octahedral mapping, a position x′ and direction w′ in three-dimensional space can be mapped to a pixel y′ on a two-dimensional octahedral texture, and the irradiance I′ is indicated by the RGB value of pixel y′ on this texture. Therefore, the position x′, direction w′, and corresponding irradiance I′ of any probe within the first video scene can be indicated by the pixel y′ on the octahedral texture and its RGB value. Specifically, RGB values ​​and irradiance are mutually corresponding; for example, the conversion between RGB values ​​and irradiance I′ can be performed through calculation or table lookup.

[0132] In other words, the probe can store the octahedral texture shown in Figure 5, which represents the irradiance and distance of the indirect lighting corresponding to the probe. Any point on this texture corresponds to a point in three-dimensional space, and the irradiance value of that point on the texture is the indirect lighting irradiance value of that point in three-dimensional space.

[0133] It is understood that when using the irradiance probe caching method, the indirect illumination parameters included in the first illumination parameter may include probe data of one or more irradiance probes distributed within the first video scene. The probe data of any probe is, for example, the octahedral texture shown in Figure 5.

[0134] (b) Voxels

[0135] A voxel is short for volume pixel. A solid containing voxels (such as a cube) can be represented through 3D rendering or by extracting polygonal isosurfaces from a given threshold contour. Based on voxels, 3D space can be meshed, and each mesh can be assigned features. During rendering, voxels can be used to store sampled data such as mesh density, color, or material, as well as the radiance and / or irradiance of mesh locations. In the voxel caching method, the first lighting parameters can store this data through a 3D voxel mesh. That is, through the voxel caching method, the indirect lighting parameters of the first video scene can include the coordinates of the 3D voxel mesh within the first video scene and the corresponding object density, color, material, radiance, and / or irradiance lighting parameters.

[0136] (c) Surface caching method

[0137] Surface caching captures the geometric properties of objects in a scene from various angles and stores them as an atlas. When visualized, surface caching resembles a low-resolution version of the rendered scene image. As shown in Figure 4, surface caching can be viewed as a low-resolution rendered version of one or more of the following: direct lighting, indirect lighting, specular lighting, refraction lighting, ambient occlusion (AO), and volumetric effects. In other words, through surface caching, the indirect lighting parameters of the first video scene can include rendered images of the surface properties of objects within the first video scene.

[0138] (d) Neural Network Caching Method

[0139] A neural network caching method is employed, implicitly storing illumination parameters such as radiance and / or irradiance by training specific network structure parameters. In step S102, the first illumination parameter may include low-volume data network data used to implicitly indicate illumination parameters such as radiance and / or irradiance. Correspondingly, the terminal device can perform network inference on the data network data to obtain illumination parameters such as radiance and / or irradiance. Therefore, it is unnecessary to transmit complete illumination data between the terminal device and the cloud server; instead, low-volume data network data can be transmitted, reducing the amount of data transmitted and the required data bandwidth.

[0140] The following describes how, in S102, the cloud server sends the first illumination parameters to the terminal device based on the information of the first video scene.

[0141] In one possible implementation, the cloud server can store first illumination parameters for one or more scenes. After receiving the information describing the first video scene in S101, it queries the first lookup parameters for one or more scenes to obtain the first illumination parameters of the first video scene, so as to support the sending of the first illumination parameters in S102.

[0142] For example, the cloud server can store lighting parameters for one or more scenes. The first lighting parameters for each scene can be found in the description of the first lighting parameters for the first video scene. The one or more scenes may include the first video scene. After obtaining information describing the first video scene through S101, the cloud server can determine the first lighting parameters for the first video scene based on this information.

[0143] Taking a game scene as an example, the cloud server, as a server rented by the game operator, can store the first lighting parameters of all game scenes. As shown in Table 1, the cloud server stores the correspondence between scene identifiers and first lighting parameters of all game scenes. After receiving the identifier of the first video scene from the terminal device, the cloud server can query the above correspondence based on the identifier of the first video scene and determine the first lighting parameters of the first video scene from the first lighting parameters of all scenes.

[0144] Table 1

[0145] Table 1 provides an example of the correspondence between scene identifiers and first lighting parameters. It should not be interpreted as a limitation on this correspondence. In other words, a modified correspondence can be obtained by appropriately modifying Table 1.

[0146] For example, the first lighting parameter #1 may include one or more of the following: direct lighting parameters, indirect lighting parameters, specular lighting parameters, refractive lighting parameters, AO parameters, and volumetric effect parameters of the first video scene corresponding to scene identifier #1; the first lighting parameter #2 may include one or more of the following: direct lighting parameters, indirect lighting parameters, specular lighting parameters, refractive lighting parameters, AO parameters, and volumetric effect parameters of the first video scene corresponding to scene identifier #2. The types of lighting parameters included in the first lighting parameter #1 and the first lighting parameter #2 may be the same or different, without specific limitation. In this application, the types of lighting parameters may be direct lighting parameters, indirect lighting parameters, specular lighting parameters, refractive lighting parameters, AO parameters, and volumetric effect parameters, etc.

[0147] Optionally, the cloud server can also store the first lighting parameters for multiple scenes under at least one scene editing condition. For example, the cloud server can also store the scene identifiers of all game scenes and the correspondence between the first lighting parameters under one or more editing conditions within that scene. The editing method can be indicated by an identifier or number. For example, as shown in Table 2, edit identifier #1 represents adding an object to the scene, identifier #2 represents deleting an object from the scene, and so on. Therefore, when the information used to describe the first video scene includes the user's editing information for the first video scene, the cloud server can determine the first lighting parameters for that first video scene under the corresponding editing operation from the first lighting parameters of all scenes based on the identifier of the first video scene and the editing information.

[0148] Table 2

[0149] Table 2 provides an example of the correspondence between scene identifiers, editing identifiers, and first lighting parameters. It should not be construed as a limitation on this correspondence. In other words, a modified correspondence between these three elements can be obtained by appropriately modifying Table 2.

[0150] As can be seen from Table 2, when the editing operations corresponding to editing identifier #1 and editing identifier #2 are executed in the first video scene corresponding to scene identifier #1, the first lighting parameters are first lighting parameter #1 and first lighting parameter #3, respectively. The first lighting parameter #1 and first lighting parameter #3 can be the same or different.

[0151] Alternatively, the cloud server can determine the edited first video scene based on the scene identifier and the user's editing information for the first video scene, obtain the identifier of the edited first video scene, and then look up Table 1 based on the identifier of the edited first video scene to obtain the first illumination parameter corresponding to the identifier of the edited first video scene. In this implementation, the cloud server may not store the correspondence between scene identifiers, editing identifiers, and first illumination parameters as shown in Table 2. This application does not limit the method by which the cloud server obtains the identifier of the edited first video scene. For example, the cloud server can determine the identifier of the edited first video scene based on the scene identifier and the editing identifier; or, the cloud server can obtain the identifier of the edited first video scene by requesting manual intervention.

[0152] It is understandable that, in addition to the correspondence between the cloud server storage scene and the first illumination parameter, the cloud server can also generate the first illumination parameter for the scene based on the information describing the first video scene received in S101. For example, the cloud server can obtain the first illumination parameter through probes or other means.

[0153] Optionally, in S102, the cloud server can encode and send the first illumination parameters of the first video scene. Correspondingly, the terminal device can decode the received data to obtain the first illumination parameters of the first video scene. This application does not specify the encoding method of the cloud server or the decoding method of the terminal device.

[0154] The following section introduces the user perspective information in S103.

[0155] As described in S103, user perspective information can be used to indicate the user's perspective within the first video scene. In one possible embodiment, the terminal device can obtain the user perspective information based on a second operation performed by the user in the application corresponding to the first video scene. The second operation may include control operations on the user's position and / or perspective within the first video scene. Specifically, the position and / or perspective may be the position and / or perspective of the user's virtual character or the camera within the first video scene.

[0156] Specifically:

[0157] (1) The second operation may include control instructions for the user's position in the first video scene. For example, the control instructions may include instructions for changing the horizontal position of the camera and / or instructions for changing its height.

[0158] Accordingly, user perspective information may include user position parameters. These parameters can be obtained in real-time based on control commands regarding the aforementioned position. The user position parameters can indicate the position within the first video scene, which may be the position of the user's virtual character or the camera in the first perspective. The user position parameters may include, but are not specifically limited to, three-dimensional coordinate information within the first video scene coordinate system. The user position parameters can be represented as x in Formula 1. o .

[0159] (2) The second operation may include control instructions for the viewpoint in the first video scene. For example, the control instructions may include instructions for changing the solid angle direction of the camera.

[0160] Correspondingly, user perspective information may include user perspective parameters. These parameters can be obtained in real-time based on the aforementioned perspective control commands. The user perspective parameters can indicate the perspective orientation within the first video scene, which can be the orientation of the user's virtual character or the camera's perspective in the first viewpoint. User perspective parameters may include angle information such as solid angles, without specific limitations. Taking a solid angle as an example, this solid angle can be used as w in Formula 1. o .

[0161] In addition, user viewpoint parameters may include parameters such as viewpoint range. Viewpoint range can refer to the maximum angle of the user's line of sight radiating from the user's position. Alternatively, the viewpoint range can be considered to be a default value.

[0162] The second illumination parameter in S104 and its determination method are described below.

[0163] The second lighting parameter can be the dynamic lighting parameter of the first video scene. The dynamic lighting parameter can also be considered as a high-frequency changing or real-time changing lighting parameter, obtained in real time by a data acquisition device on the terminal device, such as a sensor or touchscreen. Since the user's perspective information is acquired in real time, the second lighting parameter in S104 can be considered as the lighting parameter of the first video scene obtained based on the dynamic user perspective information, or the real-time rendering result.

[0164] As described in S102, the first lighting parameter may include one or more of the following: direct lighting parameters, indirect lighting parameters, reflected light parameters, refracted light parameters, ambient light occlusion parameters, and volumetric effect parameters of the first video scene. Correspondingly, the second lighting parameter may include one or more of the following: direct lighting parameters, indirect lighting parameters, reflected light parameters, refracted light parameters, ambient light occlusion parameters, and volumetric effect parameters determined based on the user's viewpoint information. Alternatively, the second lighting parameter may include one or more of the following: dynamic direct lighting parameters, indirect lighting parameters, reflected light parameters, refracted light parameters, ambient light occlusion parameters, and volumetric effect parameters. Optionally, the type of the lighting parameter in the first lighting parameter may include the type of the lighting parameter in the second lighting parameter.

[0165] For example, taking the first illumination parameter as including probe data from multiple irradiance probes under indirect illumination, the probe data of any irradiance probe can include all irradiance in the spherical direction at a certain location within the first video scene. In S104, the terminal device can select probe data from one or more irradiance probes based on the user's viewing angle information, and determine the second illumination parameter based on the selected irradiance probe data. Assuming the user's viewing angle information includes user position parameters and user viewing angle parameters, the terminal device can select one or more irradiance probes whose positions are close to the user's position based on the user position parameters, and then determine the irradiance value from a certain angular range of the selected one or more irradiance probes based on the user's viewing angle parameters. This angular range is determined based on the user's viewing angle parameters; for example, this angular range includes or covers the angular range indicated by the user's viewing angle parameters. After merging the irradiance values ​​of the selected one or more probes, the second illumination parameter can be obtained.

[0166] Alternatively, the second illumination parameter can be considered to include probe data of one or more irradiance probes selected by the terminal device from the irradiance probes included in the first illumination parameter based on the user's viewing angle information.

[0167] For example, taking probe data of irradiance probes that include multiple specular reflections as an example, the probe data of any irradiance probe can include the irradiance of specularly reflected light on the object surface observed at the probe position within the first video scene. As shown in Figure 6, ① and ② represent different probes, the camera viewpoint represents the user position parameter, and the solid line range corresponding to the camera viewpoint represents the user's viewpoint, i.e., the user's visible range. Probes ① and ② can respectively store the irradiance of reflected light from the reflecting plane at one or more locations on the object surface near the probe. For example, the object surface that probe ① can observe is shown by the arrow connected to ①, and the object surface that probe ② can observe is shown by the arrow connected to ②. Optionally, the reflected light irradiance can be stored in the form of an irradiance map, including but not limited to the octahedral map method shown in Figure 5.

[0168] Based on Figure 6, in one example of determining the second illumination parameter, the terminal device can determine the mirror position of the camera's viewing angle, i.e., the location of the mirror camera, through optical analysis. The terminal device can further obtain the viewing angle range of the mirror camera (shown by the dashed line), i.e., the actual field of vision the user can see through the mirror, and determine the second illumination parameter of the mirror reflection based on this viewing angle range, at least according to probe ① and probe ②. The object surfaces corresponding to probe ① and probe ② are within the viewing angle range of the mirror camera. The terminal device can also superimpose the irradiance of the mirrored light from the object surface stored by probe ① within the viewing angle range of the mirror camera, and the irradiance of the mirrored light from the object surface stored by probe ② within the viewing angle range of the mirror camera, to obtain the irradiance of the object surface within the viewing angle range. This irradiance can be used as the second illumination parameter, or included in the second illumination parameter.

[0169] For example, Figure 7 shows another example of specular reflection illumination parameters. The ellipse represents the curved specular reflective surface, and the circle at the center of the ellipse represents a specular reflection irradiance probe. The camera view shown in Figure 7 can serve as an example of a user view. In the camera view shown in Figure 7, the solid arrows radiating outwards from the center of the ellipse represent the direction of the camera view after specular reflection. The terminal device can determine the irradiance stored in the same direction (as shown by the dashed arrow) by the specular reflection irradiance probe based on the tangent direction. This irradiance can be used as the irradiance at the reflection point. The reflection point is the position of the tangent between the maximum angle of view of the camera and the object's surface on the object's surface.

[0170] The coloring method in S105 is described below.

[0171] During the coloring process, the final pixel color is calculated based on information such as the pixel's position, lighting, and material on the screen. This pixel color can then be displayed on the screen. A pixel can also be understood as a coloring point.

[0172] In S105, the terminal device can color each pixel in the screen display area one by one. Specifically, during the coloring process, the terminal device can determine the coordinates of the pixel in the first video scene from the user's perspective based on the position of the pixel on the screen, and then determine the pixel color parameters such as color and brightness based on the coordinates and the second lighting parameters as the rendering result. Afterwards, the pixel on the screen can be configured according to the rendering result so that the pixel presents the color corresponding to the rendering result.

[0173] It is understandable that when the second lighting parameters include multiple parameters such as direct lighting parameters, indirect lighting parameters, specular reflection light parameters, refracted light parameters, ambient occlusion parameters, and volumetric effect parameters, the rendering result of a pixel can be a superimposed rendering result obtained by fusing these multiple parameters.

[0174] In this application, the second lighting parameter can also be considered as an intermediate rendering result from the user's perspective.

[0175] As an example, the terminal device can report information describing the first video scene to the cloud server at relatively long time intervals. For instance, within a certain time period T, the terminal device only needs to report and send information describing the first video scene to the cloud server once and receive the first lighting parameters of the first video scene. For example, the time period T is half an hour or an hour, etc. That is to say, the first lighting parameters can be non-real-time. Within this time period T, the terminal device can perform one or more renderings based on the first lighting parameters through steps S103 to S105. In each rendering, the terminal device can obtain real-time user perspective information through S103 and perform shading.

[0176] Figure 8 illustrates a flowchart of the actions on the terminal device and the cloud server sides of a rendering method according to an embodiment of this application. This flowchart can serve as an exemplary implementation of the method flow shown in Figure 3.

[0177] As shown in Figure 8, the method includes the following steps:

[0178] S201: The terminal device obtains the current user scenario information. The user scenario includes scenarios such as game or video playback, displaying the scene in the game or video so that the user can view the relevant scene.

[0179] Taking a game scene as an example, user scene information refers to the information in the game scene screen where the user is currently located. This game scene information can serve as an example of the information used in S101 to describe the first video scene. Referring to the description in S101, the game scene information here may include the game scene identifier and the user's editing information about the game scene.

[0180] In the process shown in Figure 8, the current game scene can be used as an example of the first video scene in S101.

[0181] S202: The terminal device sends a scene update request to the cloud server. This request can be used to request the first illumination parameters for the current scene. The request can carry user scene information.

[0182] Correspondingly, the cloud server receives scene update requests.

[0183] Optionally, the terminal device can pre-store the address information of the cloud server and communicate with the cloud server through the address information.

[0184] S203: The cloud server updates the user scene information and determines the first lighting parameters based on the updated user scene information. The first lighting parameters include at least one of the following: indirect lighting parameters, direct lighting parameters, reflected light parameters, refracted light parameters, ambient occlusion parameters, and volumetric effect parameters of the current scene.

[0185] In S203, taking the caching method of the first illumination parameter as a probe texture as an example, the cloud server can determine the first probe data based on the user's scene information and obtain the texture corresponding to the first probe texture based on the probe data. The first probe data can contain probe data from multiple preset locations in the scene. Among them, the probe data can include irradiance data such as indirect illumination, direct illumination, reflected illumination, and refracted illumination.

[0186] For example, taking the first illumination parameter as including the indirect illumination parameter, any probe data can include the probe indirect illumination irradiance, and correspondingly, the texture can be the octahedral texture shown in Figure 5.

[0187] For example, taking the first illumination parameter as including the reflected illumination parameter, any probe data can include the irradiance of reflected light from the surface of a nearby object detected at the probe's location, i.e., the reflected irradiance. For example, as shown in Figure 6, probe positions ① and ② can detect the reflected light irradiance from different nearby reflective surfaces, respectively. The reflected light irradiance can be stored using the octahedral mapping method shown in Figure 5.

[0188] Optionally, the cloud server can update the locally stored first illumination parameters after S203. For example, if the cloud server has locally stored the first illumination parameters for the user scene information before S203, then after S206, the terminal device can update or replace the newly acquired first illumination parameters with the first illumination parameters for the user scene information.

[0189] S204: Optional, the cloud server can perform scene clipping on the game scene to select lighting parameters suitable for the user's scene information.

[0190] For example, through scene cropping, the cloud server can select objects in the first video scene based on parameters such as viewing distance and obtain the lighting parameters related to the object. In other words, it can delete objects that are beyond the viewing distance and their corresponding lighting parameters.

[0191] S205: The cloud server can send the encoded first illumination parameters to the terminal device after encoding the first illumination parameters.

[0192] This application does not specify the encoding method; existing encoding schemes between cloud servers and terminal devices can be referenced.

[0193] S205 can serve as an example of one implementation of S102.

[0194] S206: After receiving the encoded first illumination parameter, the terminal device can decode it to obtain the first illumination parameter.

[0195] Optionally, the terminal device may update the locally stored first illumination parameters. For example, if the terminal device has locally stored the first illumination parameters for the user scene information before S206, then after S206, the terminal device may update or replace the newly acquired first illumination parameters with the first illumination parameters for the user scene information.

[0196] S207: Terminal devices collect user perspective information in real time.

[0197] Specifically, user perspective information may include user location parameters and / or user perspective parameters.

[0198] S207 can be referred to the description in S103.

[0199] S208: The terminal device determines the second illumination parameter under the user's perspective based on the user's perspective information and the first illumination parameter.

[0200] S208 can also be described as updating the first lighting parameters based on user perspective information. For example, if the first lighting parameters include textures of multiple probes in the first video scene, then in S208, the terminal device can select a portion of textures related to the current user position and / or perspective as the second lighting parameters based on the user perspective information.

[0201] S208 can be referred to the description in S104.

[0202] S209: The terminal device performs coloring according to the second illumination parameters.

[0203] S209 can be described with reference to S105. For example, the terminal device can determine the lighting parameters related to the user's visible range under the current user's viewpoint information based on the user's viewpoint information and the texture related to the current user's position and / or viewpoint selected in S208, and perform shading based on the lighting parameters. Alternatively, the second lighting parameter can be considered to include or be the lighting parameters related to the user's visible range under the current user's viewpoint information.

[0204] Specifically, the terminal device can perform shading (or rendering) based on the second lighting parameters through the rendering pipeline.

[0205] It is understandable that the process shown in Figure 8 can also be applied to scenarios where multiple terminal devices render the same user scene separately. Taking multiple terminal devices, including terminal device #1 and terminal device #2, as an example, when terminal device #1 and terminal device #2 each need to render the same user scene, terminal device #1 and terminal device #2 can obtain the first lighting parameters for the user scene from the cloud server through S201 to S205 respectively. Since terminal device #1 and terminal device #2 request to render the same user scene, the cloud server can provide the same first lighting parameters to terminal device #1 and terminal device #2, without needing to generate multiple lighting parameters for the scene separately. Furthermore, terminal device #1 can collect user view information #1 as shown in S207, determine the second lighting parameter #1 based on the first lighting parameters and user view information #1 as shown in S208, and perform shading based on the second lighting parameter #1 as shown in S209. Similarly, terminal device #2 can collect user view information #2 in accordance with S207, determine second lighting parameters #2 in accordance with the first lighting parameters and user view information #2 in accordance with S208, and perform coloring in accordance with the second lighting parameters #2 in accordance with S209.

[0206] It is understandable that since the user view information #1 and user view information #2 collected by terminal device #1 and terminal device #2 may be different, the second lighting parameter #1 and the second lighting parameter #2 may also be different. Therefore, terminal device #1 and terminal device #2 can achieve real-time rendering based on their respective real-time collected user view information.

[0207] The following section, using Figure 9 as an example, illustrates how the cloud server obtains the probe texture of the first lighting parameters and how the terminal device performs indirect lighting rendering based on the indirect lighting parameters of the first video scene.

[0208] S301: The cloud server counts the objects that need to be updated within the first video scene. This step can be referred to as scene volumes.

[0209] Prior to S301, the cloud server can obtain information describing the first video scene from the terminal device. For example, see the description in S101 for the information describing the first video scene.

[0210] S302: The cloud server continuously updates scene objects. This step can also be called continuously updating volume.

[0211] Taking the irradiance update of indirect lighting as an example, S302 may include one or more of S302-1 to S302-7:

[0212] S302-1: Ray Tracing Radiance: The cloud server can obtain the radiance of each probe and each surface through ray tracing.

[0213] S302-2: Irradiance Blend: Irradiance is obtained by integrating the radiance using the rendering equation.

[0214] S302-3: Distance Blend: Each probe has a blend distance attribute, which is the distance from the surface of the probe volume to the center of the box volume. The blend distance can be used to determine the perceived distance from the probe. For example, if there are other objects near the probe, the perceived distance from the probe will be closer, meaning that the probe has less influence on the illumination of objects at greater distances.

[0215] S302-4: Irradiance Border Update: Primarily used to handle the boundary updates of lightmaps. In GI technology, lightmaps are a method of pre-computed lighting used to simulate indirect lighting effects in a scene. Irradiance Border Update is responsible for handling the boundaries of these lightmaps, ensuring a smooth transition of lighting effects at the boundaries.

[0216] S302-5: DistanceBorderUpdate: Responsible for handling the boundaries of the distance map, ensuring a smooth transition of distance effects at the boundaries.

[0217] S302-6: Optional, Probe Relocate: Lighting probes are used to store the lighting information received at their current location. When the positions of objects in the scene and / or the user's viewpoint change, the probe positions also need to be adjusted accordingly to ensure the accuracy of the lighting information.

[0218] S302-7: Optional, probe position remarking (ClassifyProbes) can be used to prioritize probes based on their importance. Probe importance can be determined based on factors such as distance from the viewpoint.

[0219] Based on S302-1 to S302-7, the cloud server can obtain the irradiance map of the indirect lighting for each updated scene, that is, obtain the lighting parameters of the indirect lighting. It is understood that the execution order of the multiple items in S302-1 to S302-7 can be changed and is not limited to the order shown in Figure 9.

[0220] It is understandable that if the first lighting parameters also include one or more parameters such as direct lighting, specular lighting, refracted lighting, AO parameters, or volumetric effect parameters, the scene can be updated by referring to S302-1 to S302-7, and one or more parameters such as direct lighting, specular lighting, refracted lighting, AO parameters, or volumetric effect parameters under the corresponding scene can be obtained.

[0221] S303: The cloud server encodes the indirect lighting map of the first video scene.

[0222] S304: The cloud server sends the encoded texture map to the terminal device.

[0223] This texture map, or the encoded texture map, can be used as the first lighting parameter in S102.

[0224] Optionally, some or all of the steps in S301 to S304 may be executed by the cloud server after receiving information from the terminal device describing the first video scene, or they may be executed according to a certain period, such as a time period T, without specific requirements.

[0225] S305: The terminal device receives and decodes the data to obtain the texture map.

[0226] The texture map from the user's perspective can be used as the second lighting parameter in S104.

[0227] S306: Calculate the rendering volume of the terminal device and fill in the description information (descriptor).

[0228] In S306, the terminal device can collect user perspective information in real time to obtain descriptive information, and the first video scene information and / or user perspective information can be used as descriptive information.

[0229] S307: Terminal device renders global illumination.

[0230] In S307, the terminal device can obtain second lighting parameters based on the user's perspective information and the first lighting parameters of the first video scene obtained from the cloud server, and perform rendering based on the second lighting parameters. Alternatively, the terminal device can be considered to perform global illumination rendering based on the user's perspective information and the first lighting parameters.

[0231] Specifically, the terminal device can obtain the indirect lighting map from the user's perspective based on the user's viewpoint information and the indirect lighting map of the first video scene, and then perform corresponding indirect lighting rendering operations. Similarly, the terminal device can also obtain one or more of the following parameters: direct lighting, specular lighting, refraction lighting, AO (Ambient Occlusion) parameters, or volumetric effect parameters, and perform one or more of the following rendering methods: direct lighting rendering, specular lighting rendering, refraction lighting rendering, AO rendering, or volumetric effect rendering. When performing multiple rendering methods, including direct lighting rendering, indirect lighting rendering, specular lighting rendering, refraction lighting rendering, AO rendering, and volumetric effect rendering, the terminal device can also overlay the rendering results of multiple methods.

[0232] Alternatively, it can be considered that when the terminal device obtains multiple of the following lighting parameters from the cloud server: direct lighting parameters, indirect lighting parameters, specular reflection lighting parameters, refraction lighting parameters, AO parameters, and volumetric effect parameters, the terminal device can perform merged rendering based on these multiple lighting parameters.

[0233] The rendering method can be referred to in the descriptions of S104 and S105, and will not be repeated here.

[0234] S308: Optionally, the terminal device reconstructs the low-resolution textures of the first video scene into a high-resolution image using super-resolution and displays it on the display screen.

[0235] The low-resolution texture can be an image of the first video scene before rendering. This texture may be stored on the terminal device or obtained from the network based on information from the first video scene.

[0236] Based on the same technical concept, embodiments of this application provide a rendering apparatus to implement the terminal device in this application. The apparatus includes modules, units, or means that execute the method steps in the above method embodiments. These functions, units, or means can be implemented by software, hardware, or hardware executing corresponding software.

[0237] For example, referring to FIG10, the rendering apparatus 1000 may include a processing unit 1001 (or rendering unit) and a transceiver unit 1002 (or communication unit).

[0238] Optionally, the processing unit 1001 can be used to perform rendering and other processing operations in the above embodiments. The transceiver unit 1002 may include a sending unit and / or a receiving unit. The sending unit is used to perform the sending operation in the above method embodiments. The receiving unit is used to perform the receiving operation in the above method embodiments.

[0239] It should be noted that the rendering apparatus 1000 may include a sending unit but not a receiving unit. Alternatively, the rendering apparatus 1000 may include a receiving unit but not a sending unit. Specifically, it depends on whether the above-described scheme executed by the rendering apparatus 1000 includes both sending and receiving actions.

[0240] The processing unit 1001 is used for data processing, such as rendering. The transceiver unit 1002 can implement corresponding communication functions.

[0241] Optionally, the rendering apparatus 1000 may further include a storage unit, which can be used to store instructions and / or data. The processing unit 1001 can read the instructions and / or data in the storage unit so that the rendering apparatus 1000 can implement the aforementioned method embodiments.

[0242] For example, the rendering device 1000 may be a first device or a component configurable on a first device. The first device may be, for example, a terminal device. The processing unit 1001 is used to perform processing-related operations of the terminal device in the above method embodiments. The transceiver unit 1002 is used to perform sending and / or receiving-related operations of the terminal device in the above method embodiments.

[0243] For example, when implementing the steps shown in Figure 3 performed by the terminal device: the processing unit 1001 can be used to obtain information describing the first video scene in S101. The transceiver unit 1002 can be used to send information describing the first video scene to the cloud server, see S101. The transceiver unit 1002 can also be used to receive first illumination parameters of the first video scene from the cloud server, see the description in S102. In addition, the processing unit 1001 can also be used to execute S103 to S105, such as obtaining user viewpoint information, determining second illumination parameters under the user viewpoint based on the user viewpoint information and the first illumination parameters, and performing coloring based on the second illumination parameters.

[0244] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0245] The processing unit 1001 in the above embodiments can be implemented by at least one processor or processor-related circuitry. The transceiver unit 1002 can be implemented by a transceiver or transceiver-related circuitry. The transceiver unit 1002 can also be referred to as a communication module or a communication interface.

[0246] The following is another structural schematic diagram of the rendering apparatus according to an embodiment of this application, used to implement the first apparatus and / or the second apparatus in this application. As shown in FIG11, an embodiment of this application also provides a rendering apparatus 1100, including:

[0247] At least one processor 1101; and a communication interface 1103 communicatively connected to the at least one processor 1101; the at least one processor 1101 causes the device to perform the method steps in the above method embodiments via the communication interface 1103 by executing instructions stored in at least one memory 1102. The at least one processor 1101 may include a graphics processor and / or a central processing unit.

[0248] Optionally, the at least one memory 1102 is located outside the device 1100.

[0249] Optionally, the device 1100 includes at least one memory 1102, which is connected to at least one processor 1101, and stores instructions executable by the at least one processor 1101. Figure 11 shows, with dashed lines, that the memory 1102 is optional for the device 1100.

[0250] The processor 1101 and the memory 1102 can be coupled through an interface circuit or integrated together; no restriction is imposed here.

[0251] This application embodiment does not limit the specific connection medium between the processor 1101, memory 1102, and communication interface 1103. In Figure 11, the processor 1101, memory 1102, and communication interface 1103 are connected via a bus 1104. The bus is represented by a straight line in Figure 11. The connection methods between other components are only illustrative and not intended to be limiting. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, a straight line is used to represent the bus in Figure 11, but this does not mean that there is only one bus or one type of bus.

[0252] When the rendering device 1100 is the first device, the first device may include a processor, a memory, and a transceiver. The memory may store computer program code, and the transceiver includes a transmitter and a receiver.

[0253] The processor is primarily used for processing communication protocols and data; for example, controlling the first device, executing software programs, and processing data from those programs. The memory is primarily used for storing software programs and data. The transmitter is used to send signals to other devices or equipment, and the receiver is used to receive signals from other devices or equipment. Other devices or equipment may include, for example, cloud servers.

[0254] When the rendering device 1100 is a chip in the first device, the chip may include a processor, a memory, and a transceiver. The transceiver may be an input / output circuit or a communication interface. The processor may be a processing module integrated on the chip, a microprocessor, or an integrated circuit. In the above method embodiments, the transmitting operation of the first device can be understood as the output of the chip, and the receiving operation of the first device in the above method embodiments can be understood as the input of the chip.

[0255] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0256] For example, the processor can be a central processing unit, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0257] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0258] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0259] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0260] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium, including a program or instructions, which, when run on a computer, cause the methods in the above method embodiments to be executed.

[0261] This application also provides a chip or chip system, including circuitry (such as analog circuitry and / or logic circuitry; or understood as the chip system including one or more processors, which may include circuitry, etc.), or understood as the chip including a processor. The circuitry or processor is coupled to a memory for executing computer programs or instructions stored in the memory, thereby implementing the methods shown in FIG3, FIG8, or FIG9, or the various embodiments of this application. The chip or chip system may also include input / output interfaces. For example, taking the implementation of terminal device functions as an example, the chip can receive information from other modules (such as radio frequency or antenna) of the terminal device through the input / output interface, and this information may be sent to the terminal device by other devices or equipment such as a cloud server. Alternatively, the chip can send information to other modules (such as radio frequency or antenna) in the terminal device through the input / output interface, and this information may be sent by the terminal device to other devices or equipment such as a cloud server.

[0262] Based on the same technical concept, embodiments of this application also provide a computer program product, including instructions that, when run on a computer, cause the methods in the above method embodiments to be executed.

[0263] Based on the same technical concept, embodiments of this application also provide a rendering system, which may include a first device and a second device. The first device can be used to implement the method implemented by the terminal device in the above method embodiments, and the second device can be used to implement the method implemented by the cloud server in the above method embodiments. The second device may be a cloud server or a component of a cloud server.

[0264] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0265] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0266] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0267] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

Claims

1. A rendering method, characterized in that, include: Send information describing the first video scene to the server; Receive the first illumination parameters of the first video scene from the server; Obtain user perspective information used to indicate the user's perspective; The second lighting parameters of the first video scene under the user's perspective are determined based on the user's perspective information and the first lighting parameters. Coloring is performed according to the second lighting parameters.

2. The method as described in claim 1, characterized in that, The acquisition of user perspective information used to indicate the user's perspective includes: The user's perspective information is acquired in real time.

3. The method as described in claim 1 or 2, characterized in that, The user-perspective information includes one or more of the following: User location parameters; User-perspective parameters.

4. The method according to any one of claims 1-3, characterized in that, The first illumination parameter includes one or more of the following: Indirect lighting parameters; Ambient light shading parameters; Direct lighting parameters; Specular reflection parameters; Refracted light parameters; Volume effect parameters.

5. A rendering apparatus, characterized in that, include: A communication unit is used to send information describing a first video scene to a server and to receive first illumination parameters of the first video scene from the server. The rendering unit is used to acquire user view information to indicate the user's viewpoint, determine the second lighting parameters of the first video scene under the user's viewpoint based on the user view information and the first lighting parameters, and perform shading based on the second lighting parameters.

6. The apparatus as claimed in claim 5, characterized in that, The rendering unit is used to acquire user perspective information that indicates the user's perspective, including: The rendering unit is used to acquire the user's perspective information in real time.

7. The apparatus as described in claim 5 or 6, characterized in that, The user-perspective information includes one or more of the following: User location parameters; User-perspective parameters.

8. The apparatus as described in any one of claims 5-7, characterized in that, The first illumination parameter includes one or more of the following: Indirect lighting parameters; Ambient light shading parameters; Direct lighting parameters; Specular reflection parameters; Refracted light parameters; Volume effect parameters.

9. A rendering apparatus, characterized in that, include: A transceiver is used to send information describing a first video scene to a server and to receive first illumination parameters of the first video scene from the server. The processor is configured to determine the second lighting parameters corresponding to the user's perspective information based on the user's perspective information and the first lighting parameters in the first video scene, and to perform shading based on the second lighting parameters.

10. The apparatus as claimed in claim 9, characterized in that, The processor is also used for: The user's perspective information is acquired in real time.

11. The apparatus as claimed in claim 9 or 10, characterized in that, The user-perspective information includes one or more of the following: User location parameters; Viewpoint parameters.

12. The apparatus as described in any one of claims 9-11, characterized in that, The first illumination parameter includes one or more of the following: Indirect lighting parameters; Ambient light shading parameters; Direct lighting parameters; Specular reflection parameters; Refracted light parameters; Volume effect parameters.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, and when the computer program or instructions are executed by a communication device, the method as described in any one of claims 1-4 is implemented.

14. A computer program product, characterized in that, When the computer program product is executed by a computer, the computer executes the method as described in any one of claims 1-4.

Images