Specular reflection scene rendering method based on multi-space decomposition

Abstract

The present application relates to the technical field of computer vision, and provides a mirror reflection scene rendering method based on multi-space decomposition, which can perform high-quality new view rendering on a scene containing mirror reflection interference, implicitly decomposes a three-dimensional real scene into multiple parallel subspaces through a neural network, obtains a feature map through rendering, finally maps each sub-space feature map into an RGB image and a weight image through a decoding network and a gating network, and finally obtains an image of the scene at a specified view angle through weighted summation, and uses the corresponding view angle RGB image for supervision, which not only can render a higher quality image, but also avoids the multi-view inconsistency interference model fitting process caused by mirror reflection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for rendering specular reflection scenes based on multi-spatial decomposition. Background Technology

[0002] Novel perspective synthesis refers to generating an image corresponding to the target camera pose, given partial viewpoint images of an object or scene and information such as camera pose. Existing methods can be divided into novel perspective synthesis methods based on the representation methods used, namely, 3D model-based methods and deep learning-based methods.

[0003] Novel perspective synthesis is a classic problem in computer vision, computer graphics, and AR / VR research. Existing methods based on 3D models have a long history of research. These methods first estimate and generate a 3D model of the target, then obtain an image from the target's perspective through 3D transformation and projection. However, these methods rely on good 3D reconstruction algorithms, which cannot achieve good reconstruction in cases of sparse perspectives or inconsistencies between multiple perspectives. Furthermore, these algorithms can produce problems such as holes and blurring in areas not visible from known perspectives. Deep learning-based methods have a shorter history of research. These methods define this task as an optimization problem, utilizing the fitting ability of neural networks to store scene or object information in the neural network parameters. When rendering a new perspective, the relevant information is retrieved through a query. These methods have achieved state-of-the-art rendering results since Neural Radiance Fields (NeRF). However, these methods are highly dependent on the consistency of the training data across multiple perspectives. When there are interferences such as specular reflections in the scene, severe blurring occurs when rendering a new perspective.

[0004] While existing methods can achieve high-quality new perspective rendering results for normal scenes (without significant specular reflections), they fail to achieve high-quality new perspective rendering under significant specular reflection interference. Therefore, both academia and industry urgently need a method that can achieve high-quality new perspective rendering under significant specular reflection interference. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a method for rendering specular reflection scenes based on multi-spatial decomposition, comprising the following steps:

[0006] Obtain parameter information of a three-dimensional physical space, wherein the parameter information of the three-dimensional physical space includes information of multiple three-dimensional space sampling points;

[0007] The three-dimensional physical space is divided into subspaces to obtain multiple parallel subspaces;

[0008] Based on multiple parallel subspaces, the information of multiple three-dimensional space sampling points is mapped to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors.

[0009] Multiple feature maps are obtained by rendering multiple sets of parallel density values ​​and multiple sets of parallel feature vectors using a volume rendering formula.

[0010] The multiple feature maps are decoded to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map.

[0011] Based on the weight map corresponding to each feature map, the RGB image corresponding to each feature map is weighted and summed to obtain the target scene image.

[0012] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of obtaining parameter information of the three-dimensional physical space, comprising:

[0013] Multiple three-dimensional physical space images are obtained by acquiring images of the three-dimensional physical space using an image acquisition device.

[0014] Parameter analysis is performed on multiple three-dimensional physical space images to determine the pose and intrinsic parameters of the image acquisition device;

[0015] The three-dimensional physical space is sampled using the pose and intrinsic parameters of the image acquisition device to obtain information on multiple three-dimensional spatial sampling points, and this information is used as the parameter information of the three-dimensional physical space.

[0016] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of performing three-dimensional spatial sampling on the three-dimensional physical space using the device parameter information to obtain multiple three-dimensional spatial sampling point information, and using the multiple three-dimensional spatial sampling point information as the three-dimensional physical space parameter information, comprising:

[0017] Pixel extraction is performed on multiple three-dimensional physical space images to determine multiple pixels;

[0018] Based on the pose and intrinsic parameters of the image acquisition device and the multiple pixels, the image acquisition device is subjected to light direction analysis to determine multiple light directions;

[0019] Based on the multiple light directions, the three-dimensional physical space is sampled in three dimensions to obtain multiple three-dimensional spatial sampling point information, and the multiple three-dimensional spatial sampling point information is used as the three-dimensional physical space parameter information.

[0020] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of mapping multiple three-dimensional spatial sampling point information based on multiple parallel subspaces to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors, comprising:

[0021] Based on multiple parallel subspaces, the information of multiple three-dimensional spatial sampling points is implicitly mapped through a preset neural network model to obtain multiple sets of parallel density values.

[0022] Based on multiple parallel subspaces, multiple three-dimensional spatial sampling points and multiple light directions are implicitly mapped through a preset neural network model to obtain multiple sets of parallel feature vectors.

[0023] According to the present invention, a method for rendering a specular reflection scene based on multi-spatial decomposition is provided, wherein the volume rendering formula is:

[0024]

[0025] in, Representation of feature map Medium beam The feature value of the corresponding pixel;

[0026] ;

[0027] Indicates light rays In the Reaching the three-dimensional space sampling point in each subspace The probability that the previous color was not yet applied;

[0028] Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in each subspace;

[0029] Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in each subspace;

[0030] Indicates light rays Three-dimensional spatial sampling points and The distance between;

[0031] Indicates light rays Three-dimensional spatial sampling points and The distance between;

[0032] Indicates light rays Three-dimensional spatial sampling points In the Feature vectors in each subspace;

[0033] ;

[0034] Indicates the number of sampling points in three-dimensional space;

[0035] Indicates the number of subspaces.

[0036] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition, wherein the method for decoding multiple feature maps to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map is as follows:

[0037] Will The feature maps of each subspace are mapped to RGB images and weight maps, respectively, and the calculation formulas are as follows:

[0038]

[0039] in, Representation domain;

[0040] For decoding networks; For gating networks;

[0041] For feature map The corresponding RGB image;

[0042] For feature map The corresponding RGB image;

[0043] ; Indicates the number of subspaces.

[0044] According to the present invention, a method for rendering a specular reflection scene based on multi-spatial decomposition includes the step of performing a weighted summation of the RGB images corresponding to each feature map based on the weight map corresponding to each feature map to obtain the target scene image, comprising:

[0045] Based on the weight map corresponding to each feature map, the RGB images corresponding to each feature map are weighted and summed using a weighted summation formula to obtain the target scene image. The weighted summation formula is as follows:

[0046]

[0047] in, To obtain the target scene image;

[0048] For feature map The corresponding RGB image;

[0049] For feature map The corresponding RGB image;

[0050] ;

[0051] Indicates the number of subspaces.

[0052] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0053] This invention provides a method for rendering specular reflection scenes based on multi-space decomposition, which can perform high-quality new perspective rendering of scenes containing specular reflection interference. By implicitly decomposing the three-dimensional physical space into multiple parallel subspaces through a neural network, the method avoids the interference of multi-view inconsistency caused by specular reflection on the model fitting process, thereby achieving high-quality new perspective rendering results for scenes containing specular reflections.

[0054] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0056] Figure 1 This is a flowchart of a specular reflection scene rendering method based on multi-space decomposition provided by the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0058] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0059] The following is combined Figure 1 The present invention describes a method for rendering specular reflection scenes based on multi-space decomposition:

[0060] This invention provides a method for rendering a specular reflection scene based on multi-spatial decomposition, comprising the following steps:

[0061] Obtain parameter information of a three-dimensional physical space, wherein the parameter information of the three-dimensional physical space includes information of multiple three-dimensional space sampling points;

[0062] The three-dimensional physical space is divided into subspaces to obtain multiple parallel subspaces;

[0063] Based on multiple parallel subspaces, the information of multiple three-dimensional space sampling points is mapped to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors.

[0064] Multiple feature maps are obtained by rendering multiple sets of parallel density values ​​and multiple sets of parallel feature vectors using a volume rendering formula.

[0065] The multiple feature maps are decoded to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map.

[0066] Based on the weight map corresponding to each feature map, the RGB image corresponding to each feature map is weighted and summed to obtain the target scene image.

[0067] In this process, the three-dimensional physical space is divided into multiple parallel subspaces. Furthermore, an object is placed in the three-dimensional physical space, and multiple virtual images are generated by reflecting the object through multiple mirrors. Each virtual image is dispersed in different subspaces, and each subspace follows multi-view consistency.

[0068] Furthermore, in the multi-space decomposition-based specular reflection scene rendering method, the number of subspaces is not necessarily equal to the number of actual virtual images.

[0069] Furthermore, multi-view consistency refers to measuring the differences between key points in two images, ensuring that key points can track the same parts of an object from different viewpoints. Specifically, the coordinates of 3D key points in the first image should be consistent with the coordinates of the corresponding key points in the second image.

[0070] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of obtaining parameter information of the three-dimensional physical space, comprising:

[0071] Multiple three-dimensional physical space images are obtained by acquiring images of the three-dimensional physical space using an image acquisition device.

[0072] Parameter analysis is performed on multiple three-dimensional physical space images to determine the pose and intrinsic parameters of the image acquisition device;

[0073] The three-dimensional physical space is sampled using the pose and intrinsic parameters of the image acquisition device to obtain information on multiple three-dimensional spatial sampling points, and this information is used as the parameter information of the three-dimensional physical space.

[0074] In one embodiment, multiple mirror objects are placed in the scene to form a scene with complex mirror reflections; the physical space is rendered using specified camera intrinsics and poses to obtain image results from multiple viewpoints, forming a dataset.

[0075] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of performing three-dimensional spatial sampling on the three-dimensional physical space using the device parameter information to obtain multiple three-dimensional spatial sampling point information, and using the multiple three-dimensional spatial sampling point information as the three-dimensional physical space parameter information, comprising:

[0076] Pixel extraction is performed on multiple three-dimensional physical space images to determine multiple pixels;

[0077] Based on the pose and intrinsic parameters of the image acquisition device and the multiple pixels, the image acquisition device is subjected to light direction analysis to determine multiple light directions;

[0078] Based on the multiple light directions, the three-dimensional physical space is sampled in three dimensions to obtain multiple three-dimensional spatial sampling point information, and the multiple three-dimensional spatial sampling point information is used as the three-dimensional physical space parameter information.

[0079] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition includes the step of mapping multiple three-dimensional spatial sampling point information based on multiple parallel subspaces to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors, comprising:

[0080] Based on multiple parallel subspaces, the information of multiple three-dimensional spatial sampling points is implicitly mapped through a preset neural network model to obtain multiple sets of parallel density values.

[0081] Based on multiple parallel subspaces, multiple three-dimensional spatial sampling points and multiple light directions are implicitly mapped through a preset neural network model to obtain multiple sets of parallel feature vectors.

[0082] Before implicitly mapping the information of multiple three-dimensional spatial sampling points, it is necessary to encode the obtained three-dimensional spatial sampling points and their light directions, and the encoding method can be arbitrary.

[0083] Among them, the neural network model is an arbitrary fully connected network. The fully connected network is essentially a multilayer perceptron that connects all inputs and outputs, and has the characteristics of high throughput, high reliability and low latency.

[0084] According to the present invention, a method for rendering a specular reflection scene based on multi-spatial decomposition is provided, wherein the volume rendering formula is:

[0085]

[0086] in, Representation of feature map Medium beam The feature value of the corresponding pixel;

[0087] ;

[0088] Indicates light rays In the Reaching the three-dimensional space sampling point in each subspace The probability that the previous color was not yet applied;

[0089] Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in each subspace;

[0090] Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in each subspace;

[0091] Indicates light rays Three-dimensional spatial sampling points and The distance between;

[0092] Indicates light rays Three-dimensional spatial sampling points and The distance between;

[0093] Indicates light rays Three-dimensional spatial sampling points In the Feature vectors in each subspace;

[0094] ;

[0095] Indicates the number of sampling points in three-dimensional space;

[0096] Indicates the number of subspaces.

[0097] According to the present invention, a method for rendering a specular reflection scene based on multi-space decomposition, wherein the method for decoding multiple feature maps to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map is as follows:

[0098] Will The feature maps of each subspace are mapped to RGB images and weight maps, respectively, and the calculation formulas are as follows:

[0099]

[0100] in, Representation domain;

[0101] For decoding networks; For gating networks;

[0102] For feature map The corresponding RGB image;

[0103] For feature map The corresponding RGB image;

[0104] ; Indicates the number of subspaces.

[0105] According to the present invention, a method for rendering a specular reflection scene based on multi-spatial decomposition includes the step of performing a weighted summation of the RGB images corresponding to each feature map based on the weight map corresponding to each feature map to obtain the target scene image, comprising:

[0106] Based on the weight map corresponding to each feature map, the RGB images corresponding to each feature map are weighted and summed using a weighted summation formula to obtain the target scene image. The weighted summation formula is as follows:

[0107]

[0108] in, To obtain the target scene image;

[0109] For feature map The corresponding RGB image;

[0110] For feature map The corresponding RGB image;

[0111] ;

[0112] Indicates the number of subspaces.

[0113] It should be noted that the multi-spatial decomposition-based specular reflection scene rendering method is a specular reflection rendering method based on multi-spatial implicit neural representation. It does not require additional supervision and can directly obtain high-quality new perspective rendering results by training directly using images with complex specular reflection interference.

[0114] Furthermore, the specular reflection scene rendering method based on multi-space decomposition can also render high-quality new perspective images in scenes without specular reflection, with performance similar to the existing best method.

[0115] Furthermore, the specular reflection scene rendering method based on multi-space decomposition can be applied to new perspective rendering, depth estimation, and virtual image ray path length estimation tasks, including but not limited to scenes with and without specular reflection.

[0116] It should be further explained that a dataset needs to be established to test the specular reflection scene rendering method based on multi-space decomposition provided by this invention. The dataset includes 25 synthetic scenes and 7 real scenes. All scenes contain a 360-degree surround view. The synthetic dataset provides 100 training views, 10 verification views and 10 test views for each scene. The real dataset contains 62-118 views for each scene.

[0117] Furthermore, the synthetic dataset is constructed based on the open-source software Blender. The 3D model uses various common 3D models of objects publicly available in existing technologies, placed in the center of a preset room scene, and mirrors are randomly added in the room or on the walls to form the data scene. During rendering, the camera is made to uniformly surround the room, with the camera always pointing towards the object in the center of the room. 120 images with a resolution of 800×800 are rendered, and 100 images are randomly selected as training data, 10 as validation data, and 10 as test data.

[0118] In one embodiment, the method for capturing real data is to use a camera to take pictures. Common objects such as tables, chairs, books, and rubber duck toys are used as regular objects, and a number of mirrors, glass balls, and other objects with mirror reflections are placed in them. The camera is randomly surrounded during the shooting process to obtain 62-118 images. After sorting them in lexicographical order, one image is taken out of every eight images as test data.

[0119] To demonstrate the effectiveness of the specular reflection scene rendering method based on multi-spatial decomposition provided by this invention, the method of this invention is compared with current advanced image rendering methods on both a self-created synthetic dataset and a real-world dataset. These advanced methods include NeRF (Neural Radiation Field), Mip-NeRF (an extension of NeRF that replaces the original spatial point sampling method with a three-dimensional spatial integral region, which can combat aliasing problems in multi-scale images), Ref-NeRF (which replaces the original NeRF shading method with a Phong lighting model, which can better render material surfaces such as specular highlights and reflections), and Mip-NeRF360 (an extension of Mip-NeRF that uses a newly proposed spatial parameterization method to render boundless large-scale three-dimensional scenes in a NeRF-like manner).

[0120] The metrics used for comparison include:

[0121] PSNR stands for Peak Signal-to-Noise Ratio, an objective standard for evaluating images. It has limitations and is generally used as an engineering measure between the maximum signal and background noise. The higher the value, the less image distortion.

[0122] SSIM: Structural Similarity, is a metric that measures the similarity between two images. SSIM uses two images: one uncompressed and undistorted, and the other distorted. The SIM value ranges from [0, 1], with a higher value indicating greater structural similarity. If two images are completely identical, the SSIM value is 1.

[0123] LPIPS: This metric measures the difference between two images. It learns an inverse mapping from the generated image to the ground truth, forces the generator to learn an inverse mapping to reconstruct the real image from the fake image, and prioritizes perceptual similarity between them. LPIPS is more in line with human perception than traditional methods. A lower LPIPS value indicates that the two images are more similar, and vice versa.

[0124] Table 1 shows the results of the multi-space decomposition-based specular reflection scene rendering method provided by this invention on the same synthetic dataset as other methods:

[0125] Table 1. Comparison of our method with existing state-of-the-art methods on synthetic datasets.

[0126]

[0127] The higher the PSNR value, the less image distortion. As shown in Table 1, the PSNR value obtained by the specular reflection scene rendering method based on multi-space decomposition provided by this invention is significantly higher than other existing advanced methods, indicating that the image rendered by the specular reflection scene rendering method based on multi-space decomposition provided by this invention has the least image distortion.

[0128] The higher the SSIM value, the more similar the image structure. Table 1 shows that the SSIM value obtained by the specular reflection scene rendering method based on multi-space decomposition provided by this invention is significantly higher than other existing advanced methods, indicating that the image structure rendered by this method has the highest similarity to the real image.

[0129] The lower the LPIPS value, the higher the perceptual similarity of the images. As shown in Table 1, the LPIPS value obtained by the specular reflection scene rendering method based on multi-space decomposition provided by this invention is significantly lower than that of other existing advanced methods, indicating that the images rendered by the specular reflection scene rendering method based on multi-space decomposition provided by this invention have the highest perceptual similarity.

[0130] As shown in Table 1, the rendering effects of Mip-NeRF 360 and the specular reflection scene rendering method based on multi-spatial decomposition provided by this invention are significantly better than other advanced existing methods. Therefore, Table 2 confirms the comparison results between the specular reflection scene rendering method based on multi-spatial decomposition provided by this invention and Mip-NeRF 360 on the same real-world dataset:

[0131] Table 2. Comparison results of this method and Mip-NeRF360 on real-world datasets.

[0132]

[0133] The higher the PSNR value, the less image distortion. Table 2 shows that the PSNR value tested using the specular reflection scene rendering method based on multi-space decomposition provided by this invention is higher than Mip-NeRF 360, indicating that the image rendered by the specular reflection scene rendering method based on multi-space decomposition provided by this invention has the least image distortion.

[0134] The higher the SSIM value, the more similar the image structures. Table 2 shows that the SSIM value obtained using the multi-spatial decomposition-based specular reflection scene rendering method provided by this invention is higher than Mip-NeRF 360, indicating that the images rendered using this multi-spatial decomposition-based specular reflection scene rendering method have better structural similarity.

[0135] The lower the LPIPS value, the higher the perceptual similarity of the images. As shown in Table 2, the LPIPS value tested using the multi-spatial decomposition-based specular reflection scene rendering method provided by this invention is slightly lower than Mip-NeRF 360.

[0136] In summary, based on the indicator data, the specular reflection scene rendering method based on multi-space decomposition provided by this invention has a significant improvement over various advanced methods in the prior art. Therefore, in scenes containing specular reflection, this method will not cause blurring or distortion when rendering the reflection area, and will not cause a reduction in rendering effect in other areas.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rendering a specular reflection scene based on multi-spatial decomposition, characterized in that, Includes the following steps: Obtain parameter information of a three-dimensional physical space, wherein the parameter information of the three-dimensional physical space includes information of multiple three-dimensional space sampling points; The three-dimensional physical space is divided into subspaces to obtain multiple parallel subspaces; Based on multiple parallel subspaces, the information of multiple three-dimensional space sampling points is mapped to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors. Multiple feature maps are obtained by rendering multiple sets of parallel density values ​​and multiple sets of parallel feature vectors using a volume rendering formula. The volume rendering formula is: in, Representation of feature map Medium beam The feature value of the corresponding pixel; ； Indicates light rays In the Reaching the three-dimensional space sampling point in a parallel subspace The probability that the previous color was not yet applied; Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in parallel subspaces; Indicates light rays Three-dimensional spatial sampling points In the Density values ​​in parallel subspaces; Indicates light rays Three-dimensional spatial sampling points and The distance between; Indicates light rays Three-dimensional spatial sampling points and The distance between; Indicates light rays Three-dimensional spatial sampling points In the Feature vectors in parallel subspaces; ； Indicates the number of sampling points in three-dimensional space; Indicates the number of parallel subspaces; The multiple feature maps are decoded to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map. The method for decoding multiple feature maps to obtain an RGB image corresponding to each feature map and a weight map corresponding to each feature map is as follows: Will The feature maps of each parallel subspace are mapped to RGB images and weight maps, respectively, and the calculation formulas are as follows: in, Representation domain; For decoding networks; For gating networks; For feature map The corresponding RGB image; For feature map The corresponding weighted graph; ; Indicates the number of parallel subspaces; Based on the weight map corresponding to each feature map, the RGB image corresponding to each feature map is weighted and summed to obtain the target scene image.

2. The method for rendering a specular reflection scene based on multi-space decomposition according to claim 1, characterized in that, The step of obtaining parameter information of the three-dimensional physical space includes: Multiple three-dimensional physical space images are obtained by acquiring images of the three-dimensional physical space using an image acquisition device. Parameter analysis is performed on multiple three-dimensional physical space images to determine the pose and intrinsic parameters of the image acquisition device; The three-dimensional physical space is sampled using the pose and intrinsic parameters of the image acquisition device to obtain information on multiple three-dimensional spatial sampling points, and this information is used as the parameter information of the three-dimensional physical space.

3. The method for rendering a specular reflection scene based on multi-space decomposition according to claim 2, characterized in that, The steps include: performing three-dimensional spatial sampling on the three-dimensional physical space using device parameter information to obtain multiple three-dimensional spatial sampling point information, and using the multiple three-dimensional spatial sampling point information as the three-dimensional physical space parameter information; Pixel extraction is performed on multiple three-dimensional physical space images to determine multiple pixels; Based on the pose and intrinsic parameters of the image acquisition device and the multiple pixels, the image acquisition device is subjected to light direction analysis to determine multiple light directions; Based on the multiple light directions, the three-dimensional physical space is sampled in three dimensions to obtain multiple three-dimensional spatial sampling point information, and the multiple three-dimensional spatial sampling point information is used as the three-dimensional physical space parameter information.

4. The method for rendering a specular reflection scene based on multi-space decomposition according to claim 3, characterized in that, The step of mapping multiple sets of three-dimensional space sampling point information based on multiple parallel subspaces to obtain multiple sets of parallel density values ​​and multiple sets of parallel feature vectors includes: Based on multiple parallel subspaces, the information of multiple three-dimensional spatial sampling points is implicitly mapped through a preset neural network model to obtain multiple sets of parallel density values. Based on multiple parallel subspaces, multiple three-dimensional spatial sampling points and multiple light directions are implicitly mapped through a preset neural network model to obtain multiple sets of parallel feature vectors.

5. The method for rendering a specular reflection scene based on multi-space decomposition according to claim 1, characterized in that, The step of performing a weighted summation of the RGB images corresponding to each feature map based on the weight map corresponding to each feature map to obtain the target scene image includes: Based on the weight map corresponding to each feature map, the RGB images corresponding to each feature map are weighted and summed using a weighted summation formula to obtain the target scene image. The weighted summation formula is as follows: in, To obtain the target scene image; For feature map The corresponding RGB image; For feature map The corresponding weighted graph; ； This indicates the number of parallel subspaces.

Images