Image rendering method and related device thereof

Abstract

The application discloses an image rendering method and a related device thereof, which can avoid a large amount of I / O overhead in the process of information transmission, thereby reducing the power consumption of image rendering. The method comprises the following steps: performing ray tracing according to rendering information of a reflecting object, and storing an image of a reflection in the reflecting object obtained in the ray tracing in a first frame buffer, the first frame buffer being used for storing images that are temporarily not needed to be displayed; performing rasterization according to the rendering information of the reflecting object and rendering information of a reflected object respectively, and storing an image of the reflecting object and an image of the reflected object obtained in the rasterization in a second frame buffer, the second frame buffer being used for storing images that are about to be displayed; obtaining the image of the reflection in the reflecting object from the first frame buffer, and storing the image of the reflection in the reflecting object in the second frame buffer, so that the image of the reflecting object, the image of the reflected object and the image of the reflection in the reflecting object are fused to obtain a target image.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and in particular to an image rendering method and related equipment. Background Technology

[0002] With the rapid development of the computer industry, users' demands for image quality are increasing. Currently, rasterization technology combined with ray tracing technology is commonly used to render 3D scenes, thereby obtaining more realistic images and improving the user experience.

[0003] Specifically, after the central processing unit (CPU) sends the rendering information of the 3D scene to the graphics processing unit (GPU), the GPU first performs rasterization based on the rendering information and stores the resulting G-buffer (containing images of various objects in the scene and their geometric information in screen space) in the first frame buffer. The first frame buffer is used to store images that do not need to be displayed temporarily, as well as other information. Then, the GPU retrieves the G-buffer from the first frame buffer and performs ray tracing based on the G-buffer to obtain images of reflections in certain objects (e.g., shadows of people on the ground, reflections of people in mirrors, etc.). In this way, the GPU can obtain a target image simulating the 3D scene, which contains images of various objects in the scene and images of reflections in certain objects, and stores the target image in the second frame buffer. Since the second frame buffer is used to store images to be displayed, the CPU can retrieve the target image from the second frame buffer and display it on the screen for the user to view.

[0004] In the above process, before the target image to be displayed is obtained, the GPU needs to complete the transfer of the G-buffer (that is, to store the G-buffer into the first frame buffer and to retrieve the G-buffer from the first frame buffer). Since the G-buffer contains a lot of information such as the images of all objects, the transfer process of the G-buffer will generate a lot of input / output (I / O) overhead, resulting in excessive power consumption for image rendering. Summary of the Invention

[0005] This application provides an image rendering method and related equipment that can avoid generating a large amount of I / O overhead during information transmission, thereby reducing the power consumption of image rendering.

[0006] A first aspect of this application provides an image rendering method, the method comprising:

[0007] The CPU can acquire rendering information of a 3D scene. Since a 3D scene typically contains reflective objects (e.g., the ground, mirrors, and lake surfaces) and objects that are reflected (e.g., people, tables, and chairs), the CPU can divide the rendering information of the 3D scene into rendering information for reflective objects and rendering information for objects that are reflected. Then, the CPU sends the rendering information of reflective objects and rendering information of objects that are reflected to the GPU.

[0008] After the GPU obtains the rendering information of both the reflecting object and the object being reflected, it first performs ray tracing based on the rendering information of the reflecting object to obtain the image of its reflection. For example, the GPU can perform ray tracing based on the rendering information of the ground to obtain the image of a person's shadow on the ground. After obtaining the image of the reflection, the GPU can store this image in the first frame buffer. It should be noted that the first frame buffer is an off-screen rendering buffer, which is used to store images that do not need to be displayed on the screen immediately. In other words, after the GPU renders the image of the reflection, it does not need to immediately display this image on the monitor screen, so it can temporarily store this image in the first frame buffer.

[0009] Then, the GPU performs rasterization based on the rendering information of the reflecting object and the object being reflected, respectively, thus obtaining images of the reflecting object and the object being reflected. For example, the GPU can perform rasterization based on the rendering information of the ground and the rendering information of the person, thereby obtaining images of the ground and the person. After obtaining the images of the reflecting object and the object being reflected, the GPU can store these images in the second frame buffer. It should be noted that the second frame buffer is the current screen rendering buffer, that is, it is used to store the images that will be displayed on the screen.

[0010] When the GPU stores the image of the reflecting object into the second frame buffer, it needs to retrieve the image of the reflection of the reflecting object from the first frame buffer and store it in the second frame buffer. In this way, the images of the reflecting object, the reflected object, and the reflection can be fused in the second frame buffer to form the target image for display. For example, the target image may contain images of the ground, a person, and the person's shadow on the ground, thus presenting a realistic image of a person on the ground. Afterward, the CPU can retrieve the target image from the second frame buffer and send it to the screen for display by the user.

[0011] As can be seen from the above method, after the GPU obtains the rendering information of the reflecting object and the reflected object, it can first perform ray tracing based on the rendering information of the reflecting object and store the image of the reflection in the reflecting object in the first frame buffer. Then, the GPU performs rasterization based on the rendering information of the reflecting object and the reflected object respectively. When storing the images of the reflecting object and the reflected object in the second frame buffer, the GPU needs to retrieve the image of the reflection in the reflecting object from the first frame buffer and store it in the second frame buffer. Therefore, the images of the reflecting object, the reflected object, and the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image to be displayed. In the aforementioned process, before obtaining the target image to be displayed, the GPU only needs to complete the transmission of the image of the reflection in the reflecting object (that is, storing the image of the reflection in the reflecting object into the first frame buffer and retrieving the image of the reflection in the reflecting object from the first frame buffer). Since the reflection in the reflecting object is only a partial shadow or mirror image of the reflected object, the image of the reflection in the reflecting object contains less information. Therefore, the I / O overhead generated by the transmission process of the image of the reflection in the reflecting object is small, which can reduce the power consumption of image rendering.

[0012] In one possible implementation, ray tracing based on the rendering information of the reflecting object specifically includes: determining the rendering information of the target region of the reflecting object, where the target region contains the reflection within the reflecting object; and performing ray tracing based on the rendering information of the target region. In this implementation, since the target region of the reflecting object contains the reflection, the remaining areas of the reflecting object do not contain reflections. If the GPU performs ray tracing on these areas, it would perform invalid calculations (including ray path calculations, intersection calculations, etc.), wasting GPU computing power. Therefore, the GPU can determine the rendering information of the target region of the reflecting object from its own rendering information, and then perform ray tracing based on this target region, without performing ray tracing on the rendering information of the remaining areas. This is equivalent to the GPU only performing ray tracing on the target region of the reflecting object, avoiding unnecessary calculations and saving GPU computing power, further reducing the power consumption of image rendering.

[0013] In one possible implementation, determining the target region of the reflecting object within its rendering information specifically includes: After obtaining the rendering information of the reflected object, the CPU constructs a bounding volume hierarchies (BVH) tree based on this information and acquires the spatial information of the camera capturing the target image. The CPU then sends the BVH tree and the camera's spatial information to the GPU. Since the BVH tree contains spatial information for multiple bounding boxes, the GPU determines the spatial information of the largest bounding box from the BVH tree. Finally, the GPU determines the target region of the reflecting object within its rendering information based on the largest bounding box's spatial information and the camera's spatial information. In this implementation, the GPU can divide the reflecting object into two parts based on the largest bounding box's spatial information and the camera's spatial information: one part is the target region, and the other part is the remaining region. Therefore, the GPU can determine the rendering information of the target region of the reflecting object within its rendering information and perform ray tracing based on this information to obtain an image of the reflection within the reflecting object.

[0014] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and a first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the reflecting object, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and a third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point. In the aforementioned implementation, since the target region of the reflecting object contains the reflection of the reflecting object, the light rays emitted by the camera, after being reflected at the target region of the reflecting object, will intersect with the maximum bounding box. At this point, there is a certain mathematical relationship between the camera's vertical height, the vertical height of the first target point (any vertex of the maximum bounding box), the distance between the second target point (any vertex in the target area) and the camera, and the distance between the camera and the first target point. Therefore, the target area of ​​the reflecting object can be accurately determined through this mathematical relationship.

[0015] In one possible implementation, the rendering information of a reflecting object includes its metallicity, and the rendering information of a reflected object includes its metallicity. The metallicity of the reflecting object is greater than or equal to a preset threshold, while the metallicity of the reflected object is less than the preset threshold. In this implementation, the CPU can classify multiple objects in a 3D scene into two categories based on their metallicity: reflecting objects and reflected objects. In this way, the CPU can divide the rendering information of the 3D scene into rendering information for reflecting objects and rendering information for reflected objects. Since reflected objects typically do not have reflections, after obtaining the rendering information for both reflecting and reflected objects, the GPU performs ray tracing only based on the rendering information for reflecting objects, not based on the rendering information for reflected objects. That is, the GPU only performs ray tracing on reflecting objects, avoiding unnecessary calculations, saving GPU computing power, and reducing the power consumption of image rendering.

[0016] In one possible implementation, the resolution of the image of the reflecting object is the same as the resolution of the image of the object being reflected, while the resolution of the image of the reflection in the reflecting object is lower than the resolution of the image of the reflecting object. In the aforementioned implementation, when performing ray tracing, the higher the resolution of the image to be rendered, the more rays need to be emitted, and the greater the computational load involved. Therefore, the resolution of the image to be rendered can be reduced to decrease the computational load, thereby further saving GPU computing power and reducing the power consumption of image rendering.

[0017] In one possible implementation, the rendering information of the reflecting object also includes the coordinates of the vertices of the reflecting object, the normals of the vertices of the reflecting object, the triangle index of the reflecting object, the roughness of the reflecting object, and the color of the reflecting object. The rendering information of the reflected object also includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, the roughness of the reflected object, and the color of the reflected object.

[0018] A second aspect of this application provides an image rendering method, the method comprising:

[0019] The CPU can acquire rendering information of a 3D scene. Since a 3D scene typically contains secondary reflective objects (e.g., mirrors and lake surfaces) and objects that are reflected (e.g., people, tables, and chairs), the CPU can divide the rendering information of the 3D scene into rendering information for secondary reflective objects and rendering information for objects that are reflected. Then, the CPU sends the rendering information of the secondary reflective objects and the rendering information of objects that are reflected to the GPU.

[0020] After acquiring the rendering information of the second reflecting object and the reflected object, the GPU performs rasterization based on these two rendering information, resulting in images of the second reflecting object and the reflected object, respectively. For example, the GPU can perform rasterization based on the rendering information of a mirror and a person, thereby obtaining images of the mirror and the person. The GPU then stores these images in a second frame buffer, which is the current screen rendering buffer used to store the images that will be displayed on the screen.

[0021] While the GPU performs rasterization, it can also perform ray tracing based on the rendering information of the second reflecting object to obtain an image of its reflection. For example, the GPU can perform ray tracing based on the rendering information of a mirror to obtain an image of a person in the mirror. After obtaining the image of the reflection, the GPU can also store this image in the second frame buffer. In this way, the images of the second reflecting object, the reflected object, and the reflection can be fused in the second frame buffer to obtain the target image for display. For example, the target image contains the image of the mirror, the image of the person, and the image of the person in the mirror, thus presenting a realistic image of a person in front of a mirror. Subsequently, the CPU can retrieve the target image from the second frame buffer and send it to the screen for display by the user.

[0022] As can be seen from the above method, after the GPU obtains the rendering information of the second reflecting object and the reflected object, it performs rasterization based on these rendering information and stores the resulting images of the second reflecting object and the reflected object in the second frame buffer. Simultaneously, the GPU can also perform ray tracing based on the rendering information of the second reflecting object and store the resulting image of the reflection in the second reflecting object in the second frame buffer. In this way, the images of the second reflecting object, the reflected object, and the reflection in the second reflecting object can be fused in the second frame buffer to obtain the target image for display. In the aforementioned process, before obtaining the target image for display, the GPU does not need to store the image in the first frame buffer (used to store images that are not yet displayed on the screen) or retrieve the image from the first frame buffer, which effectively reduces I / O overhead and thus lowers the power consumption of image rendering.

[0023] In one possible implementation, the method further includes: acquiring rendering information of a first reflecting object, wherein the reflection type of the first reflecting object is different from that of a second reflecting object; performing ray tracing based on the rendering information of the first reflecting object, and storing the image of the reflection in the first reflecting object in a first frame buffer, wherein the first frame buffer is used to store images that do not need to be displayed temporarily; performing rasterization based on the rendering information of the first reflecting object, and storing the image of the first reflecting object in a second frame buffer; acquiring the image of the reflection in the first reflecting object from the first frame buffer, and storing the image of the reflection in the first reflecting object in the second frame buffer, so that the image of the second reflecting object, the image of the reflected object, the image of the first reflecting object, the image of the reflection in the second reflecting object, and the image of the reflection in the first reflecting object are fused together to obtain a target image.

[0024] In the aforementioned implementation, the GPU can simultaneously acquire the rendering information of the second reflecting object, the rendering information of the reflected object, and the rendering information of the first reflecting object. Generally, the first reflecting object has diffuse reflection, and the second reflecting object has specular reflection; for example, the first reflecting object is the ground, and the second reflecting object is a mirror. Next, the GPU can first perform ray tracing based on the rendering information of the first reflecting object and store the image of the reflection in the first reflecting object in the first frame buffer. Then, the GPU performs rasterization based on the rendering information of the second reflecting object, the first reflecting object, and the reflected object, respectively, and performs ray tracing based on the rendering information of the second reflecting object. When storing the images of the second reflecting object, the first reflecting object, the reflected object, and the reflection in the second reflecting object in the second frame buffer, the GPU needs to retrieve the image of the reflection in the first reflecting object from the first frame buffer and store it in the second frame buffer. Therefore, the images of the second reflecting object, the first reflecting object, the reflected object, the reflection in the second reflecting object, and the reflection in the first reflecting object can be fused in the second frame buffer to obtain the target image for display. For example, the target image contains images of the ground, mirrors, people, the shadows of people on the ground, and the image of people in the mirror. Therefore, the target image can present a picture of a person standing on the ground looking in a mirror.

[0025] In the above process, before obtaining the target image to be displayed, the GPU only needs to complete the transmission of the image of the reflection in the first reflecting object (that is, storing the image of the reflection in the first reflecting object into the first frame buffer and retrieving the image of the reflection in the first reflecting object from the first frame buffer). Since the reflection in the first reflecting object is only the shadow of part of the reflected object, etc., that is, the image of the reflection in the reflecting object contains less information, the I / O overhead generated by the transmission process of the image of the reflection in the first reflecting object is small, which can reduce the power consumption of image rendering.

[0026] In one possible implementation, ray tracing based on the rendering information of the second reflecting object specifically includes: determining the rendering information of the target region of the second reflecting object from the rendering information of the second reflecting object, wherein the target region of the second reflecting object contains the reflection in the second reflecting object; and performing ray tracing based on the rendering information of the target region of the second reflecting object.

[0027] In one possible implementation, determining the rendering information of the target region of the second reflecting object in the rendering information of the second reflecting object specifically includes: obtaining the BVH tree and the spatial information of the camera, wherein the BVH tree is constructed based on the rendering information of the reflected object and the camera is used to capture the target image; determining the spatial information of the maximum bounding box from the BVH tree; and determining the rendering information of the target region of the second reflecting object in the rendering information of the second reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0028] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the second reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the second reflecting object, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and the third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point.

[0029] In one possible implementation, ray tracing based on the rendering information of the first reflecting object specifically includes: determining the rendering information of the target area of ​​the first reflecting object from the rendering information of the first reflecting object, wherein the target area of ​​the first reflecting object contains the reflection in the first reflecting object; and performing ray tracing based on the rendering information of the target area of ​​the first reflecting object.

[0030] In one possible implementation, determining the rendering information of the target region of the first reflecting object in the rendering information of the first reflecting object specifically includes: obtaining the spatial information of the BVH tree and the camera; determining the spatial information of the maximum bounding box from the BVH tree; and determining the rendering information of the target region of the first reflecting object in the rendering information of the first reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0031] In one possible implementation, the rendering information of the target area of ​​the first reflecting object includes the coordinates of a third target point. The third target point satisfies the following conditions: the ratio between the vertical height of the camera and the fourth distance is equal to the ratio between the vertical height of the first target point and the fifth distance. The third target point is any vertex in the target area of ​​the first reflecting object. The fourth distance is determined based on the coordinates of the third target point and the coordinates of the camera. The fifth distance is the difference between the fourth distance and the third distance.

[0032] In one possible implementation, the rendering information of the second reflecting object includes the metallicity and roughness of the second reflecting object; the rendering information of the reflected object includes the metallicity and roughness of the reflected object; and the rendering information of the first reflecting object includes the metallicity and roughness of the first reflecting object. Specifically, the metallicity of the first reflecting object is greater than or equal to a preset first threshold and less than a preset second threshold; the metallicity of the second reflecting object is greater than or equal to the preset second threshold; the metallicity of the reflected object is less than the preset first threshold; and the roughness of the second reflecting object is less than a preset third threshold, while the roughness of the first reflecting object is greater than or equal to the preset third threshold.

[0033] In one possible implementation, the resolution of the image of the second reflecting object, the resolution of the image of the reflected object, the resolution of the image of the first reflecting object, and the resolution of the image of the reflection in the second reflecting object are the same, while the resolution of the image of the reflection in the first reflecting object is less than the resolution of the image of the first reflecting object.

[0034] In one possible implementation, the rendering information of the second reflecting object also includes the coordinates of the vertices of the second reflecting object, the normals of the vertices of the second reflecting object, the triangle index of the second reflecting object, and the color of the second reflecting object. The spatial information of the reflected object includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, and the color of the reflected object. The spatial information of the first reflecting object includes the coordinates of the vertices of the first reflecting object, the normals of the vertices of the first reflecting object, the triangle index of the first reflecting object, and the color of the first reflecting object.

[0035] A third aspect of this application provides an electronic device including an acquisition module and a processing module. The acquisition module is used to acquire rendering information of a reflecting object and rendering information of a reflected object. The processing module is used to perform ray tracing based on the rendering information of the reflecting object and store the image of the reflection in the reflecting object in a first frame buffer, the first frame buffer being used to store images that do not need to be displayed temporarily. The processing module is also used to perform rasterization based on the rendering information of the reflecting object and the reflected object respectively, and store the images of the reflecting object and the reflected object in a second frame buffer, the second frame buffer being used to store images that are about to be displayed. The processing module is also used to acquire the image of the reflection in the reflecting object from the first frame buffer and store the image of the reflection in the reflecting object in the second frame buffer, so that the images of the reflecting object, the reflected object, and the reflection in the reflecting object are fused to obtain a target image.

[0036] In one possible implementation, the processing module is specifically used to: determine the rendering information of the target region of the reflecting object from the rendering information of the reflecting object, the target region of the reflecting object containing the reflection in the reflecting object; and perform ray tracing based on the rendering information of the target region of the reflecting object.

[0037] In one possible implementation, the processing module is specifically used to: obtain the hierarchical bounding box (BVH) tree and the spatial information of the camera, wherein the BVH tree is constructed based on the rendering information of the reflected object and the camera is used to capture the target image; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target region of the reflecting object from the rendering information of the reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0038] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the reflecting object, the first distance is the distance between the second target point and the camera, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and the third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point.

[0039] In one possible implementation, the rendering information of the reflecting object includes the metallicity of the reflecting object, the rendering information of the reflected object includes the metallicity of the reflected object, the metallicity of the reflecting object is greater than or equal to a preset threshold, and the metallicity of the reflected object is less than the preset threshold.

[0040] In one possible implementation, the resolution of the image of the reflecting object is the same as the resolution of the image of the object being reflected, but the resolution of the image of the reflection in the reflecting object is smaller than the resolution of the image of the reflecting object.

[0041] In one possible implementation, the rendering information of the reflecting object also includes the coordinates of the vertices of the reflecting object, the normals of the vertices of the reflecting object, the triangle index of the reflecting object, the roughness of the reflecting object, and the color of the reflecting object. The rendering information of the reflected object also includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, the roughness of the reflected object, and the color of the reflected object.

[0042] A fourth aspect of this application provides an electronic device including an acquisition module and a processing module. The acquisition module is used to acquire rendering information of a second reflecting object and rendering information of a reflected object. The processing module is used to perform rasterization based on the rendering information of the second reflecting object and the reflected object, respectively, and store the obtained images of the second reflecting object and the reflected object in a second frame buffer, the second frame buffer being used to store an image to be displayed. The processing module is further used to perform ray tracing based on the rendering information of the second reflecting object, and store the obtained image of the reflection in the second reflecting object in the second frame buffer, so that the images of the second reflecting object, the reflected object, and the reflection in the second reflecting object are fused to obtain a target image.

[0043] In one possible implementation, the processing module is specifically used to: determine the rendering information of the target area of ​​the second reflecting object from the rendering information of the second reflecting object, the target area of ​​the second reflecting object containing the reflection in the second reflecting object; and perform ray tracing based on the rendering information of the target area of ​​the second reflecting object.

[0044] In one possible implementation, the processing module is specifically used to: acquire the BVH tree and the spatial information of the camera, wherein the BVH tree is constructed based on the rendering information of the reflected object and the camera is used to capture the target image; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target region of the second reflecting object from the rendering information of the second reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0045] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the second reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the second reflecting object, the first distance is the distance between the second target point and the camera, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and the third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point.

[0046] In one possible implementation, the acquisition module is further configured to acquire rendering information of the first reflecting object, wherein the reflection type of the first reflecting object is different from that of the second reflecting object; the processing module is further configured to: perform ray tracing based on the rendering information of the first reflecting object, and store the image of the reflection in the first reflecting object in a first frame buffer, wherein the first frame buffer is used to store images that do not need to be displayed temporarily; perform rasterization based on the rendering information of the first reflecting object, and store the image of the first reflecting object in a second frame buffer; acquire the image of the reflection in the first reflecting object from the first frame buffer, and store the image of the reflection in the first reflecting object in the second frame buffer, so that the image of the second reflecting object, the image of the reflected object, the image of the first reflecting object, the image of the reflection in the second reflecting object, and the image of the reflection in the first reflecting object are fused together to obtain the target image.

[0047] In one possible implementation, the processing module is specifically used to: determine the rendering information of the target area of ​​the first reflecting object from the rendering information of the first reflecting object, the target area of ​​the first reflecting object containing the reflection in the first reflecting object; and perform ray tracing based on the rendering information of the target area of ​​the first reflecting object.

[0048] In one possible implementation, the processing module is specifically used to: acquire the spatial information of the BVH tree and the camera; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target area of ​​the first reflecting object from the rendering information of the first reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0049] In one possible implementation, the rendering information of the target area of ​​the first reflecting object includes the coordinates of a third target point. The third target point satisfies the following conditions: the ratio between the vertical height of the camera and the fourth distance is equal to the ratio between the vertical height of the first target point and the fifth distance. The third target point is any vertex in the target area of ​​the first reflecting object. The fourth distance is the distance between the third target point and the camera. The fourth distance is determined based on the coordinates of the third target point and the coordinates of the camera. The fifth distance is the difference between the fourth distance and the third distance.

[0050] In one possible implementation, the rendering information of the second reflecting object includes the metallicity and roughness of the second reflecting object; the rendering information of the reflected object includes the metallicity and roughness of the reflected object; and the rendering information of the first reflecting object includes the metallicity and roughness of the first reflecting object. Specifically, the metallicity of the first reflecting object is greater than or equal to a preset first threshold and less than a preset second threshold; the metallicity of the second reflecting object is greater than or equal to the preset second threshold; the metallicity of the reflected object is less than the preset first threshold; and the roughness of the second reflecting object is less than a preset third threshold, while the roughness of the first reflecting object is greater than or equal to the preset third threshold.

[0051] In one possible implementation, the resolution of the image of the second reflecting object, the resolution of the image of the reflected object, the resolution of the image of the first reflecting object, and the resolution of the image of the reflection in the second reflecting object are the same, while the resolution of the image of the reflection in the first reflecting object is less than the resolution of the image of the first reflecting object.

[0052] In one possible implementation, the rendering information of the second reflecting object also includes the coordinates of the vertices of the second reflecting object, the normals of the vertices of the second reflecting object, the triangle index of the second reflecting object, and the color of the second reflecting object. The spatial information of the reflected object includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, and the color of the reflected object. The spatial information of the first reflecting object includes the coordinates of the vertices of the first reflecting object, the normals of the vertices of the first reflecting object, the triangle index of the first reflecting object, and the color of the first reflecting object.

[0053] A fifth aspect of this application provides an electronic device including a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the electronic device performs the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0054] A sixth aspect of this application provides a computer-readable storage medium including computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0055] A seventh aspect of this application provides a computer program product including computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0056] In this embodiment, after obtaining the rendering information of the reflecting object and the reflected object, the GPU can first perform ray tracing based on the rendering information of the reflecting object and store the image of the reflection in the reflecting object in the first frame buffer. Then, the GPU performs rasterization based on the rendering information of the reflecting object and the reflected object respectively. When storing the images of the reflecting object and the reflected object in the second frame buffer, the GPU needs to retrieve the image of the reflection in the reflecting object from the first frame buffer and store it in the second frame buffer. Therefore, the image of the reflecting object, the image of the reflected object, and the image of the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image to be displayed. In the aforementioned process, before obtaining the target image to be displayed, the GPU only needs to complete the transmission of the image of the reflection in the reflecting object (that is, storing the image of the reflection in the reflecting object into the first frame buffer and retrieving the image of the reflection in the reflecting object from the first frame buffer). Since the reflection in the reflecting object is only a partial shadow or mirror image of the reflected object, the image of the reflection in the reflecting object contains less information. Therefore, the I / O overhead generated by the transmission process of the image of the reflection in the reflecting object is small, which can reduce the power consumption of image rendering. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating the principle of ray tracing technology.

[0058] Figure 2 This is a schematic diagram illustrating the principle of rasterization technology.

[0059] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0060] Figure 4 A schematic flowchart of an image rendering method provided in an embodiment of this application;

[0061] Figure 5 A schematic diagram of a BVH tree provided in an embodiment of this application;

[0062] Figure 6 A schematic diagram illustrating the first application scenario provided in this application embodiment;

[0063] Figure 7 A schematic diagram of the target area provided in the embodiments of this application;

[0064] Figure 8 This is another schematic flowchart of the image rendering method provided in the embodiments of this application;

[0065] Figure 9 A schematic diagram illustrating a second application scenario provided in this application embodiment;

[0066] Figure 10 This is yet another schematic flowchart illustrating the image rendering method provided in an embodiment of this application;

[0067] Figure 11 A schematic diagram illustrating a third application scenario provided in the embodiments of this application;

[0068] Figure 12 A schematic diagram illustrating an application example of the image rendering method provided in this application embodiment;

[0069] Figure 13 Another structural schematic diagram of the electronic device provided in the embodiments of this application;

[0070] Figure 14 This is another structural schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0071] This application provides an image rendering method and related equipment that can avoid generating a large amount of I / O overhead during information transmission, thereby reducing the power consumption of image rendering.

[0072] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved.

[0073] With the development of computer technology, more and more applications, such as games and video applications, require the display of high-quality images on electronic devices. These images are usually rendered by electronic devices based on models in a three-dimensional (3D) scene.

[0074] Traditional image processing methods typically employ rasterization to render 3D scenes, producing images capable of displaying them. However, images rendered using rasterization generally suffer from low quality and often fail to deliver realistic visuals. For instance, rendered images often struggle to accurately reproduce effects such as light reflection, refraction, and shadows within the scene. In response, a new rendering technique—ray tracing—emerged. Both ray tracing and rasterization are methods for image rendering, their primary purpose being to calculate and shade objects to project them into a two-dimensional screen space for display.

[0075] Figure 1 This is a schematic diagram illustrating the principle of ray tracing technology. Figure 1 As shown, the principle of ray tracing is as follows: From the camera's position, a ray of light is emitted through pixel positions on the image plane into the 3D scene. The nearest intersection point between the ray and the geometry is found, and then the shading of that intersection point is calculated. If the material of that intersection point is reflective, ray tracing can continue in the reflection direction from that intersection point, and the shading of the reflected intersection point can be calculated. In other words, the ray tracing method calculates projection and global illumination by tracking the propagation of light in a 3D scene, thereby rendering a 2D image.

[0076] Figure 2This is a schematic diagram illustrating the principle of rasterization technology. (Example) Figure 2 As shown, the principle of rasterization is as follows: objects in a 3D scene are divided into triangles, the 3D coordinates of the triangle vertices are transformed into 2D coordinates on the image through coordinate transformation calculation, and finally, textures are filled into the triangles on the image to achieve image rendering.

[0077] Rasterization, which projects visible content directly onto the screen space to create an image, is relatively simple to process but produces less impressive lighting effects. Ray tracing, on the other hand, tracks every ray of light emitted from the camera to achieve realistic effects such as reflection, refraction, shadows, and ambient occlusion. Therefore, ray tracing provides truly lifelike lighting and shadow effects. Consequently, to render more realistic images, current electronic devices typically combine rasterization with ray tracing to render 3D scenes, enhancing the user's viewing experience.

[0078] Specifically, after the CPU of an electronic device sends the rendering information of the 3D scene to the GPU, the GPU first performs rasterization based on the rendering information and stores the resulting G-buffer (containing images of various objects in the 3D scene and their geometric information in screen space) in the first frame buffer. The first frame buffer is used to store images that do not need to be displayed temporarily, as well as other information. Then, the GPU retrieves the G-buffer from the first frame buffer and performs ray tracing based on the G-buffer to obtain images of reflections in certain objects (e.g., shadows of people on the ground, reflections in mirrors, etc.). In this way, the GPU can obtain a target image simulating the 3D scene, which contains images of various objects in the scene and images of reflections in certain objects, and stores the target image in the second frame buffer. Since the second frame buffer is used to store images to be displayed, the CPU can retrieve the target image from the second frame buffer and display it on the screen of the electronic device for the user to view.

[0079] In the above process, before the target image to be displayed is obtained, the GPU needs to complete the transfer of the G-buffer (that is, to store the G-buffer into the first frame buffer and to retrieve the G-buffer from the first frame buffer). Since the G-buffer contains a lot of information such as the images of all objects, the transfer process of the G-buffer will generate a lot of input / output (I / O) overhead, resulting in excessive power consumption for image rendering.

[0080] To address the aforementioned problems, this application provides an image rendering method that can be executed by an electronic device. This electronic device includes a CPU and a GPU, capable of rendering images. Exemplarily, the electronic device may be a mobile phone, tablet computer, laptop computer, PC, mobile internet device (MID), wearable device, virtual reality (VR) device, augmented reality (AR) device, wireless electronic device in industrial control, wireless electronic device in self-driving, wireless electronic device in remote medical surgery, wireless electronic device in smart grid, wireless electronic device in transportation safety, wireless electronic device in smart city, wireless electronic device in smart home, etc. The electronic device may run Android, iOS, Windows, or other operating systems. Applications that require rendering 3D scenes to obtain 2D images, such as game applications, lock screen applications, map applications, or monitoring applications, can run on this electronic device.

[0081] To facilitate understanding, the following will be combined with... Figure 3 A detailed description of the specific structure of the electronic device is provided. Please refer to [reference needed]. Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 3000 may include: a central processing unit 3001, a graphics processor 3002, a display device 3003, and a memory 3004. Optionally, the electronic device 3000 may further include at least one communication bus. Figure 3 (Not shown in the image) is used to enable communication between various components.

[0082] It should be understood that the various components in the electronic device 3000 can also be coupled to each other via other connectors, which may include various interfaces, transmission lines, or buses. The various components in the electronic device 3000 can also be connected radially with the central processing unit 3001 at its center. In various embodiments of this application, coupling refers to mutual electrical connection or communication, including direct connection or indirect connection via other devices.

[0083] There are various ways to connect the central processing unit 3001 and the graphics processing unit 3002, and it is not limited to these methods. Figure 2 As shown in the diagram. The central processing unit 3001 and the graphics processing unit 3002 in the electronic device 3000 can be located on the same chip or they can be separate chips.

[0084] The functions of the central processing unit 3001, graphics processor 3002, display device 3003, and memory 3004 are briefly introduced below.

[0085] Central Processing Unit 3001: Used to run Operating System 3005 and Application Programming 3006. Application Programming 3006 can be a graphics application, such as a game, video player, etc. Operating System 3005 provides a system graphics library interface. Application Programming 3006 uses this system graphics library interface, as well as the drivers provided by Operating System 3005, such as user-mode and / or kernel-mode drivers for the graphics library, to generate an instruction stream for rendering graphics or image frames, and the necessary rendering data. The system graphics library includes, but is not limited to, OpenGL ES (Open Graphics Library for Embedded System), the Kronos Platform Graphics Interface, or Vulkan (a cross-platform graphics application programming interface). The instruction stream contains a series of instructions, which are typically calls to the system graphics library interface.

[0086] Optionally, the central processing unit 3001 may include at least one of the following types of processors: application processor, one or more microprocessors, digital signal processor (DSP), microcontroller unit (MCU), or artificial intelligence processor, etc.

[0087] The central processing unit 3001 may further include necessary hardware accelerators, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or integrated circuits for implementing logic operations. The processor 3001 may be coupled to one or more data buses for transferring data and instructions between the various components of the electronic device 3000.

[0088] Graphics Processing Unit 3002: Receives the graphics instruction stream sent by Processor 3001, generates rendering targets through the rendering pipeline, and displays the rendering targets on the display device 3003 through the operating system's layer compositing display module. The rendering pipeline, also known as the rendering pipeline, pixel pipeline, or pixel pipeline, is a parallel processing unit within the graphics processing unit 3002 used to process graphics signals. The graphics processing unit 3002 may include multiple rendering pipelines, which can process graphics signals independently and in parallel. For example, the rendering pipeline can perform a series of operations during the rendering of graphics or image frames; typical operations may include vertex processing, primitive processing, rasterization, fragment processing, etc.

[0089] Optionally, the graphics processor 3002 may include a general-purpose graphics processor that executes software, such as a GPU or other types of dedicated graphics processing units.

[0090] Display device 3003: Used to display various images generated by electronic device 3000, which may be the graphical user interface (GUI) of the operating system or image data (including still images and video data) processed by graphics processor 3002.

[0091] Optionally, the display device 3003 may include any suitable type of display screen, such as a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED) display.

[0092] Memory 3004 is the transmission channel between the central processing unit 3001 and the graphics processor 3002, and can be double data rate synchronous dynamic random access memory (DDR SDRAM) or other types of cache.

[0093] The above describes the specific structure of the electronic device used in the image rendering method provided in the embodiments of this application. The following will detail the flow of the image rendering method provided in the embodiments of this application. First, several terms used below will be explained:

[0094] Offscreen render pass: The image rendered through this pass is typically an image that does not need to be displayed on the screen temporarily. This image can be stored in a framebuffer allocated to the offscreen render pass. For ease of explanation, this framebuffer will be referred to as the first framebuffer below, and it can also be called the offscreen render buffer. It is used to store images that do not need to be displayed on the screen temporarily. In this application, the offscreen render pass can be used to implement ray tracing technology.

[0095] The onscreen render pass: The image rendered through this pass is typically the image that will be displayed on the screen (i.e., the image sent for display). This image can be stored in the framebuffer allocated to the onscreen render pass. For ease of explanation, this framebuffer will be referred to as the second framebuffer below, and it can also be called the onscreen render pass. It is used to store the image that will be displayed on the screen. In this application, the onscreen render pass can be used to implement rasterization technology or ray tracing technology.

[0096] Based on the aforementioned rendering pipelines, three different application scenarios can be constructed. In the first scenario, the electronic device can implement ray tracing technology through the off-screen rendering pipeline and rasterization technology through the current-screen rendering pipeline to render the target image for display. In the second scenario, the electronic device can implement both ray tracing and rasterization technologies through the current-screen rendering pipeline to render the target image for display. In the third scenario, the electronic device can implement both ray tracing and rasterization technologies through the off-screen rendering pipeline and the current-screen rendering pipeline to render the target image for display.

[0097] The first application scenario will be introduced below. Figure 4 This is a schematic flowchart of an image rendering method provided in an embodiment of this application, as shown below. Figure 4 As shown, the method includes:

[0098] 401. Obtain the rendering information of the reflecting object and the object being reflected.

[0099] After acquiring a model file of a 3D scene, the CPU of an electronic device can parse the rendering information of each object in the 3D scene. The rendering information of each object includes the object's spatial information and material information. The object's spatial information includes the object's vertex coordinates, vertex normals, and triangle indices, while the object's material information includes the object's color, metallicity, and roughness.

[0100] After obtaining the material information of an object, the CPU can determine the object's category based on this information. Specifically, after obtaining the material information of an object, the CPU can determine whether the object is a reflective object based on its metallicity. If the metallicity of the object is greater than or equal to a preset threshold, it can be determined that the object is a reflective object; if the metallicity of the object is less than the preset threshold, it can be determined that the object is a reflected object. For example, the metallicity of objects such as the ground, mirrors, and lakes is usually greater than the preset threshold, so these objects can be called reflective objects. On the other hand, the metallicity of objects such as people and tables is usually less than the preset threshold, so these objects can be called reflected objects. In this way, the CPU can classify multiple objects in a 3D scene into two categories: reflective objects and reflected objects.

[0101] The CPU can also construct a BVH tree based on the spatial information of the reflected object. The BVH tree can be used to implement subsequent ray tracing operations. Specifically, the CPU can construct a BVH tree based on the vertex coordinates, vertex normals, and triangle indices of the reflected object. It can be understood that the BVH tree contains the spatial information of multiple bounding boxes. The spatial information of each bounding box includes the coordinates of its eight vertices and the vertical height of its eight vertices. Each bounding box is used to enclose at least one reflected object. To further understand the aforementioned BVH tree, the following section will combine... Figure 5 To provide further details. Figure 5 A schematic diagram of a BVH tree provided in an embodiment of this application is shown below. Figure 5 As shown, suppose there are 6 reflected objects in a 3D scene. Based on the spatial information of these 6 objects, the spatial information of bounding box A (enclosing these 6 objects) can be determined. Based on the spatial information of 4 of these objects, the spatial information of bounding box B (enclosing these 4 objects) can be determined. Based on the spatial information of the remaining 2 objects, the spatial information of bounding box C (enclosing these 2 objects) can be determined. The spatial information of bounding box A, bounding box B, and bounding box C are managed using a binary tree structure, resulting in a BVH tree. It can be seen that the BVH tree can centrally manage the spatial information of bounding boxes A, B, and C, which is equivalent to centrally managing bounding boxes A, B, and C. Bounding box A is the largest bounding box among all bounding boxes.

[0102] When GPUs in electronic devices implement ray tracing based on BVH trees, it's necessary to calculate whether a ray intersects with a reflected object in the 3D scene (i.e., intersection calculation). Due to the existence of the BVH tree, the GPU can determine a bounding box surrounding the reflected object based on the BVH tree, and then determine whether the ray intersects with that bounding box. If the ray doesn't touch the bounding box, it means the ray will definitely not intersect with the reflected object within that bounding box; if the ray does touch the bounding box, then it calculates whether the ray intersects with the reflected object within that bounding box. For example, if it detects that a ray does not intersect with bounding box B in the binary tree, it means the ray will definitely not intersect with the four reflected objects within bounding box B. Therefore, the step of detecting whether the ray intersects with the four reflected objects in bounding box B can be omitted, and only the step of detecting whether the ray intersects with the two reflected objects in bounding box C can be performed.

[0103] In addition, the CPU can acquire spatial information from a camera used to capture target images of a simulated 3D scene. This spatial information may include the camera's vertical height and coordinates.

[0104] After the CPU obtains the rendering information of the reflecting object, the rendering information of the reflected object, the BVH tree, and the spatial information of the camera, it sends this information to the GPU so that the GPU can perform image rendering based on this information.

[0105] 402. Perform ray tracing based on the rendering information of the reflecting object, and store the image of the reflection in the reflecting object in the first frame buffer. The first frame buffer is used to store images that do not need to be displayed temporarily.

[0106] After the GPU obtains the rendering information of the reflecting object (including the spatial and material information of the reflecting object), the rendering information of the reflected object (including the spatial and material information of the reflected object), the BVH tree, and the spatial information of the camera, it can perform image rendering based on this information.

[0107] For ease of understanding, the following text will combine... Figure 6 The aforementioned image rendering process will be described in detail. Figure 6 This is a schematic diagram illustrating a first application scenario provided by an embodiment of this application. For example... Figure 6As shown, the GPU has a rendering pipeline 601 for the off-screen rendering channel. Rendering pipeline 601 includes a vertex shader (VS) 6011 and a fragment shader (FS) 6012. The fragment shader 6012 contains a ray tracing algorithm, and therefore, it can be used to perform ray tracing steps. Thus, the GPU can first input the rendering information of the reflecting object into the rendering pipeline 601, so that the rendering pipeline 601 can perform ray tracing based on the rendering information of the reflecting object, that is, perform ray tracing on the reflecting object to obtain an image of the reflection in the reflecting object.

[0108] However, performing ray tracing on the entire reflecting object—that is, emitting rays from every vertex of the reflecting object and then calculating the path and intersection for each ray—involves a considerable amount of computation. Furthermore, not every ray intersects the reflected object; therefore, calculations based on invalid rays (i.e., rays that do not intersect the reflected object) are invalid. It's understandable that the reflection in a reflecting object is essentially the effect or mirror image of the reflected object within the reflecting object. Therefore, a specific region within the reflecting object containing the reflection can be defined as the target region. If a ray is directed towards the target region, this portion of the ray (the effective ray) will intersect the reflected object after passing through that region. If a ray is directed towards any other region of the reflecting object besides the target region, this portion of the ray will not intersect the reflected object after passing through that region. Thus, performing ray tracing only on the target region can effectively reduce the computational load of ray tracing, thereby saving GPU computation time and reducing the power consumption of image rendering.

[0109] It is worth noting that the GPU can determine the target area of ​​a reflecting object in the following way:

[0110] The GPU can determine the spatial information of the maximum bounding box from the BVH tree, and then, based on the spatial information of the maximum bounding box and the camera's spatial information, determine the spatial information of the target region of the reflecting object from the spatial information of the reflecting object. Obtaining the spatial information of the target region of the reflecting object is equivalent to determining the target region of the reflecting object. Specifically, let any vertex in the maximum bounding box be the first target point, and any vertex in the target region of the reflecting object be the second target point. Then, the vertical height of the camera (H1), the vertical height of the first target point (H2), the distance between the second target point and the camera (X1), and the distance between the camera and the first target point (X2) have the following mathematical relationship: H1 / X1 = H2 / (X2-X1). It should be noted that X1 is the aforementioned first distance, X2-X1 is the aforementioned second distance, and X2 is the aforementioned third distance. X1 is determined based on the coordinates of the camera and the second target point, and X2 is determined based on the coordinates of the camera and the first target point.

[0111] Since the camera coordinates, camera vertical height, coordinates of the first target point, and vertical height of the first target point are all known values, only the coordinates of the second target point are unknown. The rendering information of the reflecting object includes the coordinates of each vertex in the reflecting object. Therefore, the GPU can determine whether the coordinates of each vertex in the reflecting object satisfy the aforementioned mathematical relationship. If the coordinates of a vertex satisfy the aforementioned mathematical relationship, then that vertex is a vertex in the target region of the reflecting object. In this way, the reflecting object can be divided into the target region and the remaining regions. Correspondingly, the rendering information of the reflecting object can also be divided into the rendering information of the target region and the rendering information of the remaining regions.

[0112] The following is combined Figure 7 The process of determining the target area of ​​a reflective object will be further described above. Figure 7 A schematic diagram of the target area provided in the embodiments of this application, such as Figure 7 As shown, suppose a 3D scene depicts a person standing on the ground, with their reflection on the ground. First, determine the maximum bounding box (MBO) of the scene; this MBO encloses the person. Based on the spatial information of the MBO and the camera's spatial information, the target area on the ground, which includes the person's reflection, can be determined.

[0113] After obtaining the rendering information of the target region of the reflecting object, the GPU can input the spatial information of the target region (e.g., the coordinates of the vertices of the reflecting object and the normals of the vertices of the reflecting object) into the vertex shader 6011. The vertex shader 6011 can perform certain calculations on this spatial information (e.g., converting 3D coordinates to 2D coordinates, etc.) and input the processed spatial information into the fragment shader 6012. In addition, the GPU will also directly input the BVH tree and the material information of the target region of the reflecting object into the fragment shader 6012. Therefore, the fragment shader 6012 can perform ray tracing operations based on the processed spatial information, the BVH tree, and the material information of the target region of the reflecting object to obtain the image of the reflection in the reflecting object, and store this image in the first frame buffer. At this point, the GPU has only successfully rendered the image of the reflection in the reflecting object. For example, Figure 7 An image of the reflection of a person in a painting.

[0114] Furthermore, the GPU sets a resolution for the off-screen rendering pipeline (the size of which can be set according to actual needs). This resolution is usually smaller than the screen resolution (for example, the size of this resolution can be 1 / n of the screen resolution, etc.). Then, the rendering pipeline 601 will render the image of the target area of ​​the reflecting object at this resolution. Since the resolution of the image to be rendered is smaller, fewer vertices of the target area need to be processed during ray tracing, and fewer rays need to be emitted. Therefore, the amount of computation for ray tracing can be reduced, thereby further saving the GPU's computing power and reducing the power consumption of image rendering.

[0115] 403. Perform rasterization based on the rendering information of the reflecting object and the reflected object respectively, and store the resulting images of the reflecting object and the reflected object in the second frame buffer. The second frame buffer is used to store the images to be displayed.

[0116] like Figure 6 As shown, the GPU sets up two rendering pipelines for the current screen rendering channel: rendering pipeline 602 and rendering pipeline 603. Rendering pipeline 602 contains a vertex shader 6021 and a fragment shader 6022, while rendering pipeline 603 contains a vertex shader 6031 and a fragment shader 6032. Both fragment shaders 6022 and 6032 are raw fragment shaders (i.e., without any internal algorithms). Therefore, both fragment shaders 6022 and 6032 can be used to perform the rasterization step. Specifically, fragment shader 6022 is used to perform rasterization of reflecting objects, and fragment shader 6032 is used to perform rasterization of reflected objects.

[0117] Therefore, after storing the image of the reflection in the reflecting object into the first frame buffer, the GPU can input the rendering information of the reflecting object into the rendering pipeline 602, so that the rendering pipeline 602 can perform rasterization based on the rendering information of the reflecting object, that is, rasterize the reflecting object to obtain the image of the reflecting object. Specifically, the GPU can input the spatial information of the reflecting object (e.g., the coordinates of the vertices of the reflecting object) into the vertex shader. The vertex shader 6021 can perform certain calculations on this spatial information and input the processed spatial information into the fragment shader 6022. In addition, the GPU will also directly input the material information of the reflecting object into the fragment shader 6022. Therefore, the fragment shader 6022 can perform rasterization based on the processed spatial information and the material information of the reflecting object to obtain the image of the reflecting object, and store this image in the second frame buffer.

[0118] The GPU can also input the rendering information of the reflected object into the rendering pipeline 603, so that the rendering pipeline 603 can perform rasterization based on the rendering information of the reflected object, that is, rasterize the reflected object to obtain an image of the reflected object. Specifically, the GPU can input the spatial information of the reflected object (e.g., the coordinates of the vertices of the reflected object) into the vertex shader 6031, which can perform certain calculations on this spatial information and input the processed spatial information into the fragment shader 6032. In addition, the GPU can also directly input the material information of the reflected object into the fragment shader 6032, so the fragment shader 6032 can perform rasterization based on the processed spatial information and the material information of the reflected object to obtain an image of the reflected object, and store this image in the second frame buffer. Thus, the GPU successfully renders the image of the reflecting object and the image of the reflected object. Figure 7 Images of the ground and images of people.

[0119] It should be understood that the rendering process of the image of the reflecting object and the rendering process of the image of the reflected object can be parallel.

[0120] 404. Obtain the image of the reflection in the reflecting object from the first frame buffer, and store the image of the reflection in the reflecting object in the second frame buffer so that the image of the reflecting object, the image of the reflected object, and the image of the reflection in the reflecting object are fused together to obtain the target image.

[0121] Since the second frame buffer stores the image that will be displayed on the screen (i.e., the image to be displayed), when the GPU stores the image of the reflecting object into the second frame buffer, the GPU must immediately retrieve the image of the reflection in the reflecting object from the first frame buffer and store the image of the reflection in the second frame buffer. In this way, the image of the reflecting object, the image of the reflected object, and the image of the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image to be displayed. For example, the target image can be presented as... Figure 7 The 3D scene shown depicts a person standing on the ground with their reflection on the ground. The CPU can then retrieve the target image from the second frame buffer and send it to the screen for the user to view.

[0122] It's important to note that the GPU also sets a resolution for the current screen rendering pipeline; this resolution is the screen resolution. Therefore, the resolution of the image of the reflecting object and the image of the object being reflected are the same as the screen resolution. Consequently, the resolution of the images of the reflecting object and the object being reflected is greater than the resolution of the image of the reflection within the reflecting object. Therefore, in the target image, the images of the reflecting object and the object being reflected will appear sharper, while the image of the reflection within the reflecting object will appear blurry.

[0123] In this embodiment, after obtaining the rendering information of the reflecting object and the reflected object, the GPU can first perform ray tracing based on the rendering information of the reflecting object and store the image of the reflection in the reflecting object in the first frame buffer. Then, the GPU performs rasterization based on the rendering information of the reflecting object and the reflected object respectively. When storing the images of the reflecting object and the reflected object in the second frame buffer, the GPU needs to retrieve the image of the reflection in the reflecting object from the first frame buffer and store it in the second frame buffer. Therefore, the images of the reflecting object, the reflected object, and the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image for display. In the aforementioned process, before obtaining the target image for display, the GPU only needs to complete the transmission of the image of the reflection in the reflecting object (that is, storing the image of the reflection in the reflecting object in the first frame buffer and retrieving the image of the reflection in the reflecting object from the first frame buffer). Since the reflection in the reflecting object is only a partial shadow or mirror image of the reflected object, the image of the reflection in the reflecting object contains less information. Therefore, the I / O overhead generated by the transmission process of the image of the reflection in the reflecting object is small, which can reduce the power consumption of image rendering.

[0124] The above is a detailed explanation of the first application scenario. The second application scenario will be introduced below. Figure 8This is another schematic diagram of the image rendering method provided in the embodiments of this application, such as... Figure 8 As shown, the method includes:

[0125] 801. Obtain the rendering information of the reflecting object and the object being reflected.

[0126] For instructions on step 801, please refer to [link / reference]. Figure 4 The relevant descriptions of step 401 in the illustrated embodiment will not be repeated here.

[0127] 802. Rasterize the images of the reflecting object and the reflected object respectively, and store the resulting images of the reflecting object and the reflected object in the second frame buffer. The second frame buffer is used to store the images to be displayed.

[0128] In this embodiment, the GPU can complete rasterization using the current screen rendering pipeline. For example... Figure 9 As shown ( Figure 9 (This is a schematic diagram of a second application scenario provided in an embodiment of this application). The GPU sets up a rendering pipeline 901 and a rendering pipeline 902 for the current screen rendering channel. Rendering pipeline 901 is used to rasterize the reflecting object, and rendering pipeline 902 is used to rasterize the reflected object. Specifically, for a description of the rasterization process in step 802, please refer to... Figure 4 The relevant descriptions of step 403 in the illustrated embodiment will not be repeated here.

[0129] 803. Perform ray tracing based on the rendering information of the reflecting object, and store the image of the reflection in the reflecting object in the second frame buffer so that the image of the reflecting object, the image of the reflected object, and the image of the reflection in the reflecting object are fused together to obtain the target image.

[0130] In this embodiment, the GPU can perform ray tracing using the current screen rendering pipeline. For example... Figure 9 As shown, the GPU also sets up a third rendering pipeline 903 for the current screen rendering channel. Rendering ray 903 is used to perform ray tracing on reflective objects. For an explanation of the ray tracing process in step 803, please refer to [link to relevant documentation]. Figure 4 The relevant descriptions of step 402 in the illustrated embodiment will not be repeated here.

[0131] It should be noted that the difference between step 803 and step 402 is that in step 803, after completing the ray tracing of the reflecting object, the image of the reflection of the reflecting object is stored in the second frame buffer. Therefore, the image of the reflecting object, the image of the reflected object, and the image of the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image.

[0132] It should be understood that the rendering process of the image of the reflecting object, the rendering process of the image of the reflected object, and the rendering process of the image of the reflection in the reflecting object can be parallel, that is, the GPU can execute steps 802 and 803 synchronously.

[0133] In this embodiment, after acquiring the rendering information of the reflecting object and the reflected object, the GPU performs rasterization based on these rendering information and stores the resulting images of the reflecting and reflected objects in the second frame buffer. Simultaneously, the GPU can also perform ray tracing based on the rendering information of the reflecting object and store the resulting image of the reflection in the reflecting object in the second frame buffer. In this way, the images of the reflecting object, the reflected object, and the reflection in the reflecting object can be fused in the second frame buffer to obtain the target image for display. In the aforementioned process, before obtaining the target image for display, the GPU does not need to store the image in the first frame buffer (used to store images that are not currently displayed on the screen) or retrieve the image from the first frame buffer, effectively reducing I / O overhead and thus lowering the power consumption of image rendering.

[0134] The above is a detailed explanation of the second application scenario. The third application scenario will be introduced below. Figure 10 This is another schematic flowchart illustrating the image rendering method provided in the embodiments of this application, as shown below. Figure 10 As shown, the method includes:

[0135] 1001. Obtain the rendering information of the first reflecting object, the rendering information of the second reflecting object, and the rendering information of the reflected object.

[0136] In this embodiment, after the CPU obtains a model file of a 3D scene, it can parse the rendering information of each object in the 3D scene. The rendering information of each object includes the object's spatial information and material information. The object's spatial information includes the object's vertex coordinates, vertex normals, and triangle indices, while the object's material information includes the object's color, metallicity, and roughness.

[0137] After obtaining the material information of an object, the CPU can determine the object's category based on its metallicity and roughness. Specifically, if the CPU determines that the metallicity of an object is less than a preset first threshold, it can determine that the object is a reflective object, such as a person or a table. If the CPU determines that the metallicity of an object is greater than or equal to the preset first threshold, it can determine that the object is a reflective object. Reflective objects include first reflective objects and second reflective objects. The reflection type of a first reflective object is diffuse reflection, such as the ground or glass. The reflection type of a second reflective object is specular reflection, such as a mirror or a lake surface. Furthermore, if the CPU determines that the metallicity of an object is greater than or equal to the preset first threshold and less than the preset second threshold, and the roughness of the object is greater than or equal to the preset third threshold, it determines that the object is a first reflective object. If the CPU determines that the metallicity of an object is greater than or equal to the preset second threshold, and the roughness of the object is less than the preset third threshold, it determines that the object is a second reflective object.

[0138] The CPU can also construct a BVH tree based on the rendering information of the reflected object and obtain the camera's spatial information. This process can be referenced. Figure 4 The relevant descriptions of step 401 in the illustrated embodiment will not be repeated here.

[0139] After the CPU obtains the rendering information of the first reflecting object, the second reflecting object, the reflected object, the BVH tree, and the spatial information of the camera, it sends this information to the GPU so that the GPU can perform image rendering based on this information.

[0140] 1002. Perform ray tracing based on the rendering information of the first reflecting object, and store the image of the reflection in the first reflecting object in the first frame buffer. The first frame buffer is used to store images that do not need to be displayed temporarily.

[0141] After obtaining rendering information for the first reflecting object, the second reflecting object, the reflected object, the BVH tree, and the camera's spatial information, the GPU can perform image rendering based on this information. For ease of understanding, the following section will combine... Figure 11 The aforementioned image rendering process will be described in detail. Figure 11 This is a schematic diagram illustrating a third application scenario provided in an embodiment of this application. For example... Figure 11 As shown, the GPU has a rendering pipeline 1101 for the off-screen rendering channel. The rendering pipeline 1101 includes a vertex shader 11011 and a fragment shader 11012. The fragment shader 11012 contains a ray tracing algorithm, so the fragment shader 11012 can be used to perform ray tracing of the first reflecting object.

[0142] The GPU can first determine the rendering information of the target region of the first reflecting object from the rendering information of the first reflecting object. Then, it performs ray tracing based on the rendering information of the target region of the first reflecting object to obtain the image of the reflection in the first reflecting object. Specifically, the GPU can determine the spatial information of the maximum bounding box from the BVH tree. Then, based on the spatial information of the maximum bounding box and the spatial information of the camera, the GPU determines the rendering information of the target region of the first reflecting object from the rendering information of the first reflecting object. Finally, the GPU inputs the rendering information of the target region of the first reflecting object into the rendering pipeline 1101, so that the rendering pipeline 1101 performs ray tracing based on the rendering information of the target region of the first reflecting object, that is, performs ray tracing on the target region of the first reflecting object to obtain the image of the reflection in the first reflecting object, and stores this image in the first frame buffer. The third target point of the target region of the first reflecting object satisfies the following condition:

[0143] The ratio between the camera's vertical height and the fourth distance is equal to the ratio between the vertical height of the first target point and the fifth distance. The first target point is any vertex in the maximum bounding box, the third target point is any vertex in the target area of ​​the first reflecting object, the fourth distance is the distance between the third target point and the camera, and the fourth distance is determined based on the coordinates of the third target point and the camera. The fifth distance is the difference between the fourth distance and the third distance, and the third distance is the distance between the first target point and the camera, and the third distance is determined based on the coordinates of the camera and the first target point.

[0144] Furthermore, the GPU also sets a resolution for the off-screen rendering pipeline (the size of which can be set according to actual needs), which is usually smaller than the screen resolution.

[0145] It should be noted that the process by which the GPU determines the target area of ​​the first reflecting object, and the process by which the GPU performs ray tracing through the rendering pipeline 1101, can be found in [reference needed]. Figure 4 The relevant descriptions of step 402 in the illustrated embodiment will not be repeated here.

[0146] 1003. Rasterize the images of the first reflecting object, the second reflecting object, and the reflected object respectively, and store the resulting images of the second reflecting object, the first reflecting object, and the reflected object in the second frame buffer. The second frame buffer is used to store the images to be displayed.

[0147] 1004. Perform ray tracing based on the rendering information of the second reflecting object, and store the image of the reflection in the second reflecting object in the second frame buffer.

[0148] like Figure 11As shown, the GPU can set up three rendering pipelines for the current screen rendering pass: rendering pipeline 1102, rendering pipeline 1103, and rendering pipeline 1104. Rendering pipeline 1102 contains a vertex shader 11021 and a fragment shader 11022; rendering pipeline 1103 contains a vertex shader 11031 and a fragment shader 11032; and rendering pipeline 1104 contains a vertex shader 11041 and a fragment shader 11042. Fragment shaders 11022 and 11032 are both raw fragment shaders (i.e., without any internal algorithms), while fragment shader 11042 has a ray tracing algorithm. Therefore, fragment shaders 11022 and 11032 can both be used to perform rasterization, and fragment shader 11042 can be used to perform ray tracing. The fragment shader 11022 is used to perform rasterization of the reflecting object (including the first reflecting object and the second reflecting object), the fragment shader 11032 is used to perform rasterization of the reflected object, and the fragment shader 11042 is used to perform ray tracing of the second reflecting object.

[0149] After storing the image of the reflection in the first reflecting object into the first frame buffer, the GPU can input the rendering information of the first reflecting object and the second reflecting object into the rendering pipeline 1102, and the rendering information of the object to be reflected into the rendering pipeline 1103. In this way, the rendering pipeline 1102 can perform rasterization (i.e., rasterize both the first and second reflecting objects) based on the rendering information of the first and second reflecting objects to obtain images of the first and second reflecting objects, and the rendering pipeline 1103 can perform rasterization (i.e., rasterize the reflected object) based on the rendering information of the object to obtain an image of the reflected object. Having obtained the images of the first reflecting object, the second reflecting object, and the reflected object, the GPU can store these images in the second frame buffer.

[0150] Simultaneously, the GPU can determine the rendering information of the target region of the second reflecting object from the rendering information of the second reflecting object, and then perform ray tracing based on the rendering information of the target region of the second reflecting object to obtain an image of the reflection in the second reflecting object. Specifically, the GPU can determine the spatial information of the maximum bounding box from the BVH tree. Then, based on the spatial information of the maximum bounding box and the spatial information of the camera, the GPU determines the rendering information of the target region of the second reflecting object from the rendering information of the second reflecting object. Finally, the GPU inputs the rendering information of the target region of the second reflecting object into the rendering pipeline 1104, so that the rendering pipeline 1104 performs ray tracing based on the rendering information of the target region of the second reflecting object, that is, performs ray tracing on the target region of the second reflecting object to obtain an image of the reflection in the second reflecting object, and stores this image in the second frame buffer. The second target point of the target region of the second reflecting object satisfies the following condition:

[0151] The ratio between the camera's vertical height and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, and the second target point is any vertex in the target area of ​​the second reflecting object. The first distance is the distance between the second target point and the camera, and is determined based on the coordinates of the second target point and the camera. The second distance is the difference between the first distance and the third distance. The third distance is the distance between the first target point and the camera, and is determined based on the coordinates of the camera and the first target point.

[0152] It should be noted that the process of rasterization performed by the GPU through rendering pipelines 1102 and 1103 can be found in [reference needed]. Figure 4 The relevant descriptions of step 403 in the illustrated embodiment will not be repeated here. Similarly, the process by which the GPU determines the target area of ​​the second reflecting object, and the process by which the GPU performs ray tracing through the rendering pipeline 1104, can be found in [reference needed]. Figure 4 The relevant descriptions of step 402 in the illustrated embodiment will not be repeated here.

[0153] It should be understood that the rendering process of the image of the first reflecting object, the rendering process of the image of the second reflecting object, the rendering process of the image of the reflected object, and the rendering process of the image of the reflection in the second reflecting object can be parallel, that is, the GPU can execute steps 1003 and 1004 synchronously.

[0154] 1005. Obtain the image of the reflection in the first reflecting object from the first frame buffer, and store the image of the reflection in the first reflecting object in the second frame buffer so that the image of the second reflecting object, the image of the reflected object, the image of the first reflecting object, the image of the reflection in the second reflecting object, and the image of the reflection in the first reflecting object are fused together to obtain the target image.

[0155] Since the second frame buffer stores the image that will be displayed on the screen (i.e., the image to be displayed), when the GPU stores the image of the first reflecting object into the second frame buffer, the GPU must immediately retrieve the image of the reflection of the first reflecting object from the first frame buffer and store the image of the reflection of the first reflecting object into the second frame buffer. In this way, the images of the second reflecting object, the reflected object, the first reflecting object, the reflection of the second reflecting object, and the reflection of the first reflecting object can be fused together in the second frame buffer to obtain the target image to be displayed. Afterward, the CPU can retrieve the target image from the second frame buffer and send it to the screen for display by the user.

[0156] It's important to note that the GPU also sets a resolution for the current screen rendering pipeline; this resolution is the screen resolution. Therefore, the resolutions of the images of the first reflecting object, the second reflecting object, the reflected object, and the reflection in the second reflecting object are all equal to the screen resolution. Consequently, the resolution of the image of the first reflecting object is greater than the resolution of its reflection. Therefore, in the target image, the images of the first reflecting object, the second reflecting object, the reflected object, and its reflection will appear relatively clear, while the reflection in the first reflecting object will appear relatively blurry.

[0157] In this embodiment, the GPU can simultaneously acquire the rendering information of the second reflecting object, the rendering information of the reflected object, and the rendering information of the first reflecting object. Generally, the reflection type of the first reflecting object is diffuse reflection, and the reflection type of the second reflecting object is specular reflection. For example, the first reflecting object is the ground, and the second reflecting object is a mirror, etc. Next, the GPU can first perform ray tracing based on the rendering information of the first reflecting object and store the image of the reflection in the first reflecting object in the first frame buffer. Then, the GPU performs rasterization based on the rendering information of the second reflecting object, the first reflecting object, and the reflected object, respectively, and performs ray tracing based on the rendering information of the second reflecting object. When storing the images of the second reflecting object, the first reflecting object, the reflected object, and the reflection in the second reflecting object in the second frame buffer, the GPU needs to retrieve the image of the reflection in the first reflecting object from the first frame buffer and store it in the second frame buffer. Therefore, the images of the second reflecting object, the first reflecting object, the reflected object, the reflection in the second reflecting object, and the reflection in the first reflecting object can be fused in the second frame buffer to obtain the target image to be displayed. For example, the target image contains images of the ground, mirrors, people, the shadows of people on the ground, and the image of people in the mirror. Therefore, the target image can present a picture of a person standing on the ground looking in a mirror.

[0158] In the above process, before obtaining the target image to be displayed, the GPU only needs to complete the transmission of the image of the reflection in the first reflecting object (that is, storing the image of the reflection in the first reflecting object into the first frame buffer and retrieving the image of the reflection in the first reflecting object from the first frame buffer). Since the reflection in the first reflecting object is only the shadow of part of the reflected object, etc., that is, the image of the reflection in the reflecting object contains less information, the I / O overhead generated by the transmission process of the image of the reflection in the first reflecting object is small, which can reduce the power consumption of image rendering.

[0159] The following section will further describe the image rendering method provided in the embodiments of this application, using a specific application example. Figure 12 This is a schematic diagram illustrating an application example of the image rendering method provided in the embodiments of this application, such as... Figure 12 As shown in this application example, an additional plugin can be added to a Unity application to achieve, for example... Figure 4 , Figure 8 and Figure 10 The method shown can achieve three rendering modes. The specific rendering mode that the plugin is used for can be configured according to actual needs; no specific restrictions are made here.

[0160] Specifically, during the initialization (awake) phase, the program can pass raw data from the 3D scene into the plugin. During the update phase of a specific frame, based on the raw data, it can update the rendering information of the reflecting objects, the reflected objects, and the camera's spatial information for that frame. It also constructs a BVH tree based on the reflected object's rendering information and updates this information into the plugin. When the program executes the on-post-render phase, it calls the plugin through its interface. The plugin passes the aforementioned information to the GPU, which uses the rendering method provided by the plugin to complete the rendering and obtain the final target image for display.

[0161] It should be understood that Figure 12 China only Figure 4 The rendering modes shown (including off-screen rendering and current-screen rendering) are illustrated for illustrative purposes and do not constitute a limitation on the rendering modes used in this application example.

[0162] Furthermore, this application also tests electronic devices under different rendering modes and obtains a superior mode based on the test results. The first mode is rasterization without ray tracing, the second mode is deferred rendering (i.e., the rendering mode mentioned in the background art), and the third mode is... Figure 8 The rendering mode shown is the fourth one. Figure 10 The rendering modes are shown in Table 1. The test results for the four rendering modes are as follows:

[0163] Table 1

[0164]

[0165]

[0166] In Table 1, difference 1 is the difference between the average value of the first mode and the average value of the second mode, difference 2 is the difference between the average value of the first mode and the average value of the third mode, and difference 3 is the difference between the average value of the first mode and the average value of the fourth mode. The test results shown in Table 1 indicate that, compared to existing rendering modes (the first two), the rendering modes provided in this application (the latter two) have lower power consumption, and the power consumption of the last rendering mode is superior to that of the third rendering mode.

[0167] The above is a detailed description of the image rendering method provided in the embodiments of this application. The electronic device provided in the embodiments of this application will be described below. Figure 13 Another structural schematic diagram of the electronic device provided in the embodiments of this application is shown below. Figure 13 As shown, the electronic device includes an acquisition module 1301 and a processing module 1302.

[0168] The acquisition module 1301 is used to acquire the rendering information of the reflecting object and the reflected object.

[0169] The processing module 1302 is used to perform ray tracing based on the rendering information of the reflecting object, and store the image of the reflection in the reflecting object in the first frame buffer. The first frame buffer is used to store images that do not need to be displayed temporarily.

[0170] The processing module 1302 is also used to perform rasterization based on the rendering information of the reflecting object and the rendering information of the reflected object, and store the resulting images of the reflecting object and the reflected object in the second frame buffer, which is used to store the images to be displayed.

[0171] The processing module 1302 is also used to obtain the image of the reflection in the reflecting object from the first frame buffer, and store the image of the reflection in the reflecting object in the second frame buffer, so that the image of the reflecting object, the image of the reflected object and the image of the reflection in the reflecting object are fused together to obtain the target image.

[0172] In one possible implementation, the processing module 1302 is specifically used to: determine the rendering information of the target area of ​​the reflecting object from the rendering information of the reflecting object, wherein the target area of ​​the reflecting object contains the reflection in the reflecting object; and perform ray tracing based on the rendering information of the target area of ​​the reflecting object.

[0173] In one possible implementation, the processing module 1302 is specifically used to: obtain the hierarchical bounding box (BVH) tree and the spatial information of the camera, wherein the BVH tree is constructed based on the rendering information of the reflected object and the camera is used to capture the target image; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target region of the reflected object from the rendering information of the reflected object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0174] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the reflecting object, the first distance is the distance between the second target point and the camera, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and the third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point.

[0175] In one possible implementation, the rendering information of the reflecting object includes the metallicity of the reflecting object, the rendering information of the reflected object includes the metallicity of the reflected object, the metallicity of the reflecting object is greater than or equal to a preset threshold, and the metallicity of the reflected object is less than the preset threshold.

[0176] In one possible implementation, the resolution of the image of the reflecting object is the same as the resolution of the image of the object being reflected, but the resolution of the image of the reflection in the reflecting object is smaller than the resolution of the image of the reflecting object.

[0177] In one possible implementation, the rendering information of the reflecting object also includes the coordinates of the vertices of the reflecting object, the normals of the vertices of the reflecting object, the triangle index of the reflecting object, the roughness of the reflecting object, and the color of the reflecting object. The rendering information of the reflected object also includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, the roughness of the reflected object, and the color of the reflected object.

[0178] Figure 14 Another structural schematic diagram of the electronic device provided in the embodiments of this application, as shown below. Figure 14 As shown, the electronic device includes an acquisition module 1401 and a processing module 1402.

[0179] The acquisition module 1401 is used to acquire the rendering information of the second reflecting object and the rendering information of the reflected object;

[0180] The processing module 1402 is used to rasterize according to the rendering information of the second reflecting object and the rendering information of the reflected object, and store the obtained images of the second reflecting object and the reflected object in the second frame buffer. The second frame buffer is used to store the images to be displayed.

[0181] The processing module 1402 is also used to perform ray tracing based on the rendering information of the second reflecting object, and store the image of the reflection in the second reflecting object in the second frame buffer, so that the image of the second reflecting object, the image of the reflected object and the image of the reflection in the second reflecting object are fused together to obtain the target image.

[0182] In one possible implementation, the processing module 1402 is specifically used to: determine the rendering information of the target area of ​​the second reflecting object from the rendering information of the second reflecting object, wherein the target area of ​​the second reflecting object contains the reflection in the second reflecting object; and perform ray tracing based on the rendering information of the target area of ​​the second reflecting object.

[0183] In one possible implementation, the processing module 1402 is specifically used to: acquire the BVH tree and the spatial information of the camera, wherein the BVH tree is constructed based on the rendering information of the reflected object and the camera is used to capture the target image; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target area of ​​the second reflecting object from the rendering information of the second reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0184] In one possible implementation, the spatial information of the maximum bounding box includes the vertical height and coordinates of the first target point, the spatial information of the camera includes the vertical height and coordinates of the camera, and the rendering information of the target region of the second reflecting object includes the coordinates of the second target point. The second target point satisfies the following conditions: the ratio between the vertical height of the camera and the first distance is equal to the ratio between the vertical height of the first target point and the second distance. The first target point is any vertex in the maximum bounding box, the second target point is any vertex in the target region of the second reflecting object, the first distance is the distance between the second target point and the camera, the first distance is determined based on the coordinates of the second target point and the coordinates of the camera, the second distance is the difference between the first distance and the third distance, and the third distance is determined based on the coordinates of the camera and the coordinates of the first target point.

[0185] In one possible implementation, the acquisition module 1401 is further configured to acquire rendering information of the first reflecting object, wherein the reflection type of the first reflecting object is different from that of the second reflecting object; the processing module 1402 is further configured to: perform ray tracing based on the rendering information of the first reflecting object, and store the image of the reflection in the first reflecting object in a first frame buffer, wherein the first frame buffer is used to store images that do not need to be displayed temporarily; perform rasterization based on the rendering information of the first reflecting object, and store the image of the first reflecting object in a second frame buffer; acquire the image of the reflection in the first reflecting object from the first frame buffer, and store the image of the reflection in the first reflecting object in the second frame buffer, so that the image of the second reflecting object, the image of the reflected object, the image of the first reflecting object, the image of the reflection in the second reflecting object, and the image of the reflection in the first reflecting object are fused together to obtain the target image.

[0186] In one possible implementation, the processing module 1402 is specifically used to: determine the rendering information of the target area of ​​the first reflecting object from the rendering information of the first reflecting object, wherein the target area of ​​the first reflecting object contains the reflection in the first reflecting object; and perform ray tracing based on the rendering information of the target area of ​​the first reflecting object.

[0187] In one possible implementation, the processing module 1402 is specifically used to: acquire the spatial information of the BVH tree and the camera; determine the spatial information of the maximum bounding box from the BVH tree; and determine the rendering information of the target area of ​​the first reflecting object from the rendering information of the first reflecting object based on the spatial information of the maximum bounding box and the spatial information of the camera.

[0188] In one possible implementation, the rendering information of the target area of ​​the first reflecting object includes the coordinates of a third target point. The third target point satisfies the following conditions: the ratio between the vertical height of the camera and the fourth distance is equal to the ratio between the vertical height of the first target point and the fifth distance. The third target point is any vertex in the target area of ​​the first reflecting object. The fourth distance is the distance between the third target point and the camera. The fourth distance is determined based on the coordinates of the third target point and the coordinates of the camera. The fifth distance is the difference between the fourth distance and the third distance.

[0189] In one possible implementation, the rendering information of the second reflecting object includes the metallicity and roughness of the second reflecting object; the rendering information of the reflected object includes the metallicity and roughness of the reflected object; and the rendering information of the first reflecting object includes the metallicity and roughness of the first reflecting object. Specifically, the metallicity of the first reflecting object is greater than or equal to a preset first threshold and less than a preset second threshold; the metallicity of the second reflecting object is greater than or equal to the preset second threshold; the metallicity of the reflected object is less than the preset first threshold; and the roughness of the second reflecting object is less than a preset third threshold, while the roughness of the first reflecting object is greater than or equal to the preset third threshold.

[0190] In one possible implementation, the resolution of the image of the second reflecting object, the resolution of the image of the reflected object, the resolution of the image of the first reflecting object, and the resolution of the image of the reflection in the second reflecting object are the same, while the resolution of the image of the reflection in the first reflecting object is less than the resolution of the image of the first reflecting object.

[0191] In one possible implementation, the rendering information of the second reflecting object also includes the coordinates of the vertices of the second reflecting object, the normals of the vertices of the second reflecting object, the triangle index of the second reflecting object, and the color of the second reflecting object. The spatial information of the reflected object includes the coordinates of the vertices of the reflected object, the normals of the vertices of the reflected object, the triangle index of the reflected object, and the color of the reflected object. The spatial information of the first reflecting object includes the coordinates of the vertices of the first reflecting object, the normals of the vertices of the first reflecting object, the triangle index of the first reflecting object, and the color of the first reflecting object.

[0192] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of this application, and the resulting technical effects are the same as those of the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in the embodiment of this application, and it will not be repeated here.

[0193] This application also relates to a computer-readable storage medium, including computer-readable instructions that, when executed on a computer, cause the computer to perform actions such as... Figure 4 , Figure 8 or Figure 10 The method described in the illustrated embodiment.

[0194] This application also relates to a computer program product, including computer-readable instructions that, when executed on a computer, cause the computer to perform actions such as... Figure 4 , Figure 8 or Figure 10 The method described in the illustrated embodiment.

[0195] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0199] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image rendering method, characterized by, The method comprises: acquiring rendering information of a reflected object and rendering information of a reflected object; performing ray tracing according to the rendering information of the reflected object, and storing an image of a reflection in the reflected object obtained in a first frame buffer, the first frame buffer being used to store images that are not required to be displayed temporarily; performing rasterization according to the rendering information of the reflected object and the rendering information of the reflected object, and storing images of the reflected object and images of the reflected object obtained in a second frame buffer, the second frame buffer being used to store images that are required to be displayed; acquiring the image of the reflection in the reflected object from the first frame buffer, and storing the image of the reflection in the reflected object in the second frame buffer, so that the images of the reflected object, the images of the reflected object and the image of the reflection in the reflected object are fused to obtain a target image; wherein the ray tracing according to the rendering information of the reflected object comprises: determining rendering information of a target region of the reflected object in the rendering information of the reflected object, the target region of the reflected object containing the reflection in the reflected object; performing ray tracing according to the rendering information of the target region of the reflected object.

2. The method of claim 1, wherein, The determination of the rendering information of the target region of the reflected object in the rendering information of the reflected object comprises: acquiring a bounding volume hierarchy (BVH) tree and spatial information of a camera, the BVH tree being constructed based on the rendering information of the reflected object, the camera being used to capture the target image; determining spatial information of a maximum bounding box from the BVH tree; determining the rendering information of the target region of the reflected object in the rendering information of the reflected object according to the spatial information of the maximum bounding box and the spatial information of the camera.

3. The method of claim 2, wherein, The spatial information of the maximum bounding box contains a vertical height of a first target point and coordinates of the first target point, the spatial information of the camera contains a vertical height of the camera and coordinates of the camera, and the rendering information of the target region of the reflected object contains coordinates of a second target point, the second target point satisfying the following condition: a ratio between the vertical height of the camera and a first distance is equal to a ratio between the vertical height of the first target point and a second distance, the first target point being any vertex in the maximum bounding box, the second target point being any vertex in the target region of the reflected object, the first distance being determined based on the coordinates of the second target point and the coordinates of the camera, and the second distance being a difference between the first distance and a third distance, the third distance being determined based on the coordinates of the camera and the coordinates of the first target point.

4. The method according to any one of claims 1 to 3, characterized in that, The rendering information of the reflected object contains a metallic degree of the reflected object, the rendering information of the reflected object contains a metallic degree of the reflected object, the metallic degree of the reflected object being greater than or equal to a preset threshold, and the metallic degree of the reflected object being less than the preset threshold.

5. The method according to any one of claims 1 to 3, characterized in that, The resolution of the image of the reflected object is the same as the resolution of the image of the reflected object, and the resolution of the image of the reflection in the reflected object is less than the resolution of the image of the reflected object.

6. The method of claim 4, wherein, The rendering information of the reflected object further includes coordinates of vertices of the reflected object, normals of the vertices of the reflected object, triangle indexes of the reflected object, roughness of the reflected object, and color of the reflected object, and the rendering information of the reflected object further includes coordinates of vertices of the reflected object, normals of the vertices of the reflected object, triangle indexes of the reflected object, roughness of the reflected object, and color of the reflected object.

7. An image rendering method characterized by, The method comprises: acquiring rendering information of a second reflected object and rendering information of a reflected object; performing rasterization according to the rendering information of the second reflected object and the rendering information of the reflected object, and storing the obtained image of the second reflected object and the image of the reflected object in a second frame buffer, the second frame buffer being used for storing images to be displayed; performing ray tracing according to the rendering information of the second reflected object, and storing the obtained image of the reflection in the second reflected object in the second frame buffer, so that the image of the second reflected object, the image of the reflected object, and the image of the reflection in the second reflected object are fused to obtain a target image; wherein the performing ray tracing according to the rendering information of the second reflected object specifically comprises: in the rendering information of the second reflected object, determining rendering information of a target region of the second reflected object, the target region of the second reflected object containing the reflection in the second reflected object; performing ray tracing according to the rendering information of the target region of the second reflected object.

8. The method of claim 7, wherein, The determining, in the rendering information of the second reflected object, of the rendering information of the target region of the second reflected object specifically comprises: acquiring a BVH tree and spatial information of a camera, the BVH tree being constructed based on the rendering information of the reflected object, and the camera being used for shooting the target image; determining spatial information of a maximum bounding box from the BVH tree; determining, according to the spatial information of the maximum bounding box and the spatial information of the camera, the rendering information of the target region of the second reflected object in the rendering information of the second reflected object.

9. The method of claim 8, wherein, The spatial information of the maximum bounding box contains a vertical height of a first target point and coordinates of the first target point, the spatial information of the camera contains a vertical height of the camera and coordinates of the camera, and the rendering information of the target region of the second reflected object contains coordinates of a second target point, the second target point satisfying the following conditions: A ratio between a vertical height of the camera and the first distance is equal to a ratio between a vertical height of the first target point and the second distance, the first target point is any one vertex in the maximum bounding box, the second target point is any one vertex in a target region of the second reflective object, the first distance is determined based on a coordinate of the second target point and a coordinate of the camera, and the second distance is a difference between the first distance and the third distance, the third distance is determined based on the coordinate of the camera and the coordinate of the first target point.

10. The method of claim 9, wherein, The method further comprises: acquiring rendering information of a first reflective object, the first reflective object having a different reflection type from the second reflective object; performing ray tracing according to the rendering information of the first reflective object, and storing an image of a reflection in the first reflective object obtained in the ray tracing in a first frame buffer, the first frame buffer being used to store images that are temporarily not required to be displayed; performing rasterization according to the rendering information of the first reflective object, and storing an image of the first reflective object obtained in the rasterization in the second frame buffer; acquiring the image of the reflection in the first reflective object from the first frame buffer, and storing the image of the reflection in the first reflective object in the second frame buffer, so that the image of the second reflective object, the image of the reflected object, the image of the first reflective object, the image of the reflection in the second reflective object, and the image of the reflection in the first reflective object are fused to obtain a target image.

11. The method of claim 10, wherein, The ray tracing according to the rendering information of the first reflective object specifically comprises: in the rendering information of the first reflective object, determining rendering information of a target region of the first reflective object, the target region of the first reflective object containing the reflection in the first reflective object; performing ray tracing according to the rendering information of the target region of the first reflective object.

12. The method of claim 11, wherein, The determination of the rendering information of the target region of the first reflective object in the rendering information of the first reflective object specifically comprises: acquiring a BVH tree and spatial information of a camera; determining spatial information of a maximum bounding box from the BVH tree; determining the rendering information of the target region of the first reflective object in the rendering information of the first reflective object according to the spatial information of the maximum bounding box and the spatial information of the camera.

13. The method of claim 12, wherein, The rendering information of the target region of the first reflective object contains a coordinate of a third target point, the third target point satisfying the following condition: a ratio between a vertical height of the camera and a fourth distance is equal to a ratio between a vertical height of the first target point and a fifth distance, the third target point being any one vertex in the target region of the first reflective object, the fourth distance being determined based on a coordinate of the third target point and a coordinate of the camera, and the fifth distance being a difference between the fourth distance and the third distance.

14. The method according to any one of claims 10 to 13, characterized in that, The rendering information of the second reflective object includes a metallic degree of the second reflective object and a roughness of the second reflective object, the rendering information of the reflected object includes a metallic degree of the reflected object and a roughness of the reflected object, and the rendering information of the first reflective object includes a metallic degree of the first reflective object and a roughness of the first reflective object; The metallic degree of the first reflective object is greater than or equal to a preset first threshold value and less than a preset second threshold value, the metallic degree of the second reflective object is greater than or equal to the preset second threshold value, and the metallic degree of the reflected object is less than the preset first threshold value. The roughness of the second reflective object is less than a preset third threshold value, and the roughness of the first reflective object is greater than or equal to the preset third threshold value.

15. The method according to any one of claims 10 to 13, characterized in that, The resolution of the image of the second reflective object, the resolution of the image of the reflected object, the resolution of the image of the first reflective object, and the resolution of the image of the reflection in the second reflective object are the same, and the resolution of the image of the reflection in the first reflective object is less than the resolution of the image of the first reflective object.

16. The method of claim 14, wherein, The rendering information of the second reflective object further includes coordinates of vertices of the second reflective object, normals of the vertices of the second reflective object, triangle indexes of the second reflective object, and colors of the second reflective object, the spatial information of the reflected object includes coordinates of vertices of the reflected object, normals of the vertices of the reflected object, triangle indexes of the reflected object, and colors of the reflected object, and the spatial information of the first reflective object includes coordinates of vertices of the first reflective object, normals of the vertices of the first reflective object, triangle indexes of the first reflective object, and colors of the first reflective object.

17. An electronic device, comprising: An electronic device including a memory and a processor; the memory stores code, and the processor is configured to execute the code, when the code is executed, the electronic device executes the method of any one of claims 1-6 or 7-16.

18. A computer-readable storage medium, characterized in that, Computer-readable instructions, when executed on a computer, cause the computer to perform the method of any one of claims 1-6 or 7-16.

19. A computer program product, characterised in that, Computer-readable instructions, when executed on a computer, cause the computer to perform the method of any one of claims 1-6 or 7-16.

Images