Panoramic image new viewpoint synthesis method based on geometric appearance decoupling multi-spherical image
Through the geometric appearance decoupling method, the color channel and alpha channel learning are separated, the number of network output channels is reduced, and the consistency loss of visibility masks and masks is introduced, which solves the problems of rendering quality and generalization performance in the multispherical image method, and realizes high-quality new panoramic image synthesis.
Patent Information
- Application Number
- CN202510283618.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-11
- Publication Date
- 2025-07-29
AI Technical Summary
The existing method based on multispherical images has increased learning difficulty due to the excessive number of output channels, which affects the generalization performance of the network and the rendering quality in new scenarios. It also lacks a correct understanding of the 360° scene, especially the occlusion part, resulting in a decrease in rendering quality.
Through the geometric appearance decoupling method, the learning process of color channels and alpha channels is separated, the number of network output channels is reduced, and the consistency loss of visibility masks and masks is introduced to improve rendering quality and generalization capabilities.
It realizes high-quality multi-spherical image generation with strong generalization capabilities, improving the adaptability of multi-scene joint optimization and rendering quality for unknown scenes.
Smart Images
Figure CN120387940A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and particularly relates to a novel view synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images. Background Art
[0002] Panoramic images are widely used in visualizing 360° scenes and have various applications in fields such as virtual reality, virtual tourism, robot navigation, and autonomous driving. However, providing an immersive experience for users requires a large number of high-resolution panoramic images, which greatly increases the required storage space and transmission bandwidth. Therefore, synthesizing new views from a limited number of panoramic images is crucial for providing a real-time and smooth viewing experience.
[0003] The method based on multi-spherical images represents a 360° scene by generating a set of discrete spherical image layers centered at the source viewpoint at predefined depths (each layer consists of color channels and channels), and obtains the rendering result at the new viewpoint through sampling and synthesis. The discrete and finite spherical image layers representing the 360° scene ensure the rendering efficiency of the method, but also limit the final rendering quality. To ensure the rendering quality, scene details can be captured by generating a large number of depth layers or using high-dimensional features instead of three-dimensional color channels. However, this will result in too many output channels, high model complexity, easy overfitting of the network, and poor generalization ability. Therefore, the network structure of the existing methods based on multi-spherical images cannot take into account both the rendering quality and the network generalization performance of the method.
[0004] MatryODS obtains two sets of panoramic image scans through the homography matrix at different depths for a pair of panoramic images under adjacent viewpoints, and uses them as the input of the neural network. Through the encoder-decoder, a set of fusion weight layers and a set of layers are obtained, and then the color layer (i.e., appearance representation) of the multi-spherical image is obtained by weighted summation of the two sets of panoramic image scans through the fusion weight layers. Using one-dimensional weights instead of three-dimensional color layers as the output of the network reduces the number of output channels of the network and also ensures the generalization ability of the network for unknown scenes. However, due to the lack of a correct understanding of the 360° scene, especially for the occluded parts in the input panoramic images, the overall rendering quality of the network is reduced.
[0005] SOMSI proposes using high-dimensional feature vector layers instead of three-dimensional color layers in multi-spherical images to obtain a higher-quality rendering effect through the final decoder decoding process. Although in order to reduce the network learning difficulty, SOMSI uses only 3 high-dimensional feature vector layers and performs weighted summation on the features through a generated three-dimensional mask layer corresponding to the depth, but ultimately increases the number of output channels of the network, affecting the generalization performance of the network. Summary of the Invention
[0006] The purpose of the present invention is to address the shortcomings of the existing technology, avoid the problem that the network learning difficulty is increased due to the excessive number of output channels in the method network based on multi-spherical images, which affects the generalization performance of the network and the rendering quality in new scenes, achieve high-quality panoramic image rendering, improve the adaptability of multi-scene joint optimization and the generalization ability for unknown scenes, and provide a new viewpoint synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images.
[0007] The present invention is specifically as follows: Step (1). Perform data augmentation on the training dataset: The training dataset includes panoramic images for training, and the corresponding position of each panoramic image and the camera pose matrix ; The panoramic image includes a source viewpoint panoramic image and as a reference image Panoramic image of target viewpoints .
[0008] Each target viewpoint panoramic image is passed through Rotate and reproject to obtain the enhanced panoramic image of the target viewpoint , and the corresponding position and the camera pose matrix , to enhance the training dataset, .
[0009] Step (2). Panoramic image of the source viewpoint conduct Sub-bilinear downsampling, we get Panoramic image of the sampled source viewpoints : , , Indicates bilinear downsampling with a downsampling factor of 2; original-size source viewpoint panoramic image and sampled source viewpoint panoramic image composition Input panoramic images of different sizes , , ; Combine the input panoramic image with the spherical coordinate code of the corresponding size to obtain Splicing results of different sizes as output.
[0010] The plane coordinates of the input panoramic image are The spherical coordinates of the point on the unit sphere are , represents the pixel point of the input panoramic image mapped to the azimuth angle on the unit sphere, represents the pixel point of the input panoramic image mapped to the elevation angle on the unit sphere: , , and represents the pixel point of the input panoramic image mapped to the width and height coordinates on the plane, and represent the width and height of the input panoramic image respectively.
[0011] Step (3). Input the stitching result into the pyramid encoder to extract multi-scale features as the output.
[0012] Step (4). Input the extracted multi-scale features into the appearance prediction network to decode and obtain layers of color layers , and layers of visibility masks .
[0013] Step (5). Use the last multi-scale feature in the multi-scale features extracted by the pyramid encoder and the visibility mask as the input of the appearance feature decoder, and execute times through the gated convolutional network with parameter sharing to predict each layer corresponding layer .
[0014] Step (6). According to the color layers of the normal ratio and channels , obtain the complete multi-spherical image representation ; sample at the position of the target viewpoint, and then render the panoramic image of the target viewpoint to obtain the panoramic image of the target viewpoint .
[0015] Step (7). Compare the prediction with the real image and train the multi-spherical image generation network through the loss function. The loss function includes the L2 loss function , the visibility mask consistency loss function and the color gradient loss function . The total loss function , and To balance the importance weights of different loss terms.
[0016] Furthermore, step (2) inputs the panoramic image The spherical coordinate encoding calculation process is as follows: Azimuth Encoding , pitch angle Encoding , Indicates the order; Get spherical coordinate encoding , , is a real number domain; input panoramic image The corresponding spherical coordinate encoding Stitching along the third dimension, the stitching result , , Indicates stitching along the third dimension.
[0017] Furthermore, step (3) is specifically: (3-1) The splicing results The input is sequentially fed into the pyramid encoder in descending order; the pyramid encoder consists of Residual Coding Module Each residual coding module consists of Convolution structure, batch normalization and activation function are connected after each level of convolution; the first splicing result Enter the first residual module , get the first eigenvector ; Represents feature extraction; (3-2) Starting from the second residual coding module, The input is from the previous level Output features The result of performing bilinear downsampling And the splicing results of this level composition, ; Output ;final Multi-scale features extracted by a pyramid encoder composed of residual coding modules .
[0018] Furthermore, step (4) is specifically: (4-1) Yes Perform non-uniform compression; multi-scale features exist The result after performing non-uniform compression , Indicates the output In the feature map The eigenvector of the position, Indicates vertical coordinates The absolute value of the pitch angle on the corresponding unit sphere; Greater than The area where the is compressed is compressed, and the other areas remain unchanged; Indicates compression, The first step is to input features Split along the first dimension to get two The vector is then spliced along the third dimension to obtain , and then through a The convolution module maps the output features to a uniform number of channels Up, output : ;right Perform non-uniform compression on all feature vectors in to obtain multi-scale features after non-uniform compression ; (4-2) The multi-scale features after non-uniform compression Input to the pyramid decoder , decoding multi-scale features into color layers ,in is the number of layers of the multi-spherical image; (4-3) Use the reverse compression strategy to convert the color layer Restore the color layer to normal proportions : First pass The convolutional layer will The number of channels of the compressed part is expanded to twice the original number of channels, and then the extra channel information expanded is rearranged into the vertical space by rearranging the pixels in the vertical direction to obtain ; (4-4) Using a decoder module consisting of two layers of multi-layer perceptrons predict Corresponding visibility mask ;in, represents the linear transformation layer of the multilayer perceptron, Represents the activation function, which is generated through two layers of linear transformation layers. The mask corresponding to the layer color layer ;right application Activation function gets visibility mask , so that the sum of the visibility weights of each pixel on all MSI layers is .
[0019] Furthermore, step (5) is specifically: (5-1) Calculate the Visibility mask of the layer The sum of all visibility masks of the current layer and subsequent layers, and then the last multi-scale feature Multiply them together to get Visible features of layers , ;in, represents element-wise multiplication operation, is the current layer number; (5-2) Visible features of layers and the visibility mask of the layer As the input of the gated convolutional network, Layer Channel prediction, get the ; (5-3) Repeat (5-1) and (5-2) times, and obtain the multi-spherical image aisle .
[0020] Furthermore, step (6) is specifically: (6-1) At the target viewpoint The generated multi-layer spherical image is sampled at The pixel at which the target is located generates a line from the target viewpoint to the pixel. The light of departure , calculate the intersection of the ray with the spherical image at each depth , record each intersection Corresponding color value and value ,get color values and A collection of values : ;in, Representative The spherical radius of the layer, Indicates the position from the target viewpoint The light of departure The unit vector of (6-2) Pair Set Use opacity synthesis algorithm to calculate the target panoramic image Located in The pixel color value at : ; Among them, the cumulative transmittance ; Calculate the color value of each position to get a panoramic image Prediction .
[0021] Furthermore, in step (7): L2 loss function: ;in, 、 Respectively 、 exist The color value at Consistency loss of visibility masks :Compute appearance prediction network based on real images Predicted The visibility mask at and The visibility mask at The Euclidean distance between And feed back to the multi-spherical image generation network, For the homography matrix operation, the source viewpoint panoramic image Location The mask is mapped to the position of the target viewpoint , to align the mask information; Color gradient loss :This loss is based on the gradient information of the image and constrains the network's learning of edge structure by calculating the second-order derivative of the color distribution: ;in, Represents the Laplace filter operator, which can calculate the second-order derivative of the RGB color value of the image. represents the L2 norm.
[0022] Starting from the network structure based on the multi-spherical image method, the present invention utilizes geometric appearance decoupling to separate the learning process of color channel and alpha channel, reduces the number of network output channels, improves the rendering quality by adding visibility mask guidance and mask consistency loss, and realizes the generation of high-quality and generalized multi-spherical images. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 It is a simplified flow chart of the present invention. DETAILED DESCRIPTION
[0024] like Figure 1 As shown in FIG, a new viewpoint synthesis method for panoramic images based on geometric appearance decoupling of multi-spherical images is as follows: Step (1). Perform data augmentation on the training dataset; The training data set includes A panoramic image for training, and the corresponding position of each panoramic image and the camera pose matrix ; The panoramic image includes a source viewpoint panoramic image and a target viewpoint panoramic image as a reference image ; Each target viewpoint panoramic image is rotated and reprojected times to obtain an enhanced target viewpoint panoramic image , and the corresponding position and the camera pose matrix , so as to enhance the training data set.
[0025] Step (2). Perform bilinear downsampling on the source viewpoint panoramic image to obtain times sampled source viewpoint panoramic images : , ; Among them, represents bilinear downsampling, and the downsampling factor is 2. For the input image , the size of the downsampled image is , , respectively represent the width and height of the input image, represents rounding down, and the value of the pixel point of the sampled image ; The original size source viewpoint panoramic image and the sampled source viewpoint panoramic image form different sizes of input panoramic images , , ; The input panoramic image is superimposed with the spherical coordinate encoding of the corresponding size as the input; The calculation process of the spherical coordinate encoding of the input panoramic image is as follows: The plane coordinates of the input panoramic image are The spherical coordinates of the point corresponding to the unit sphere are , represents the pixel point of the input panoramic image mapped to the azimuth angle on the unit sphere, represents the pixel point of the input panoramic image mapped to the pitch angle on the unit sphere: , , and represent the pixel point The width and height coordinates on the plane, and Represent the width and height of the input panoramic image respectively; Azimuth Encoding , pitch angle Encoding , Indicates the order; Get spherical coordinate encoding , , is a real number domain; input panoramic image The corresponding spherical coordinate encoding Stitching along the third dimension, the stitching result , , Represents splicing along the third dimension, resulting in Splicing results of different sizes as output.
[0026] Step (3). Splice the result Input to the pyramid encoder to extract feature outputs at different scales; (3-1) The splicing results The input is sequentially fed into the pyramid encoder in descending order; the pyramid encoder consists of Residual Coding Module Each residual coding module consists of Convolution structure, batch normalization and activation function are connected after each level of convolution; the first splicing result Enter the first residual module , get the first eigenvector . Represents feature extraction; (3-2) Starting from the second residual coding module, The input is from the previous level Output features The result of performing bilinear downsampling And the splicing results of this level composition, ; Output ;final Multi-scale features extracted by a pyramid encoder composed of residual coding modules .
[0027] Step (4). Extract the multi-scale features Input to the appearance prediction network and decoded to get Layer Color Layer ,as well as Layer visibility mask .
[0028] (4-1) To reduce the network parameters of this module, perform non-uniform compression on ; perform non-uniform compression on the multi-scale features at the result after non-uniform compression , denote the feature vector at the th feature map at the position, denote the absolute value of the pitch angle on the unit sphere corresponding to the vertical coordinate ; perform compression on the region greater than and keep other regions unchanged; denote compression, the first step of is to split the input feature along the first dimension to obtain two vectors of size , and then splice them along the third dimension to obtain . Without loss of information, compress the vector of size with the number of channels into a feature vector of size with the number of channels and , and then map the output feature to a unified number of channels through a convolutional module, and output : ; perform non-uniform compression on all feature vectors in to obtain the multi-scale features after non-uniform compression .
[0029] (4-2) Input the multi-scale features after non-uniform compression into the pyramid decoder to decode the multi-scale features into the color layer ; (4-3) Use the reverse compression strategy to restore the color layer to the color layer with normal proportion : First, expand the number of channels of the compressed part in by two times the original number of channels through the convolutional layer, and then through pixel rearrangement, rearrange the additional channel information expanded in the vertical direction into the vertical space to obtain ; (4-4) Use a decoder module composed of two multi-layer perceptrons Predict The corresponding visibility mask ; where Represents the linear transformation layer of the multi-layer perceptron, Represents the activation function. Through two linear transformation layers, generate The mask corresponding to the layer color layer For Apply The activation function to obtain the visibility mask , such that the sum of the visibility weights of each pixel on all MSI layers is , ensuring that the sum of the contributions of different layers is ; , Represents The visibility mask value at the activated position in the th layer, And Represents the exponential function with the real number as the base.
[0030] Step (5). Use the last multi-scale feature in the multi-scale features extracted by the pyramid encoder And the visibility mask As the input of the appearance feature decoder, and perform times through the parameter-sharing gated convolutional network to predict the corresponding to each layer layer ; (5-1) Calculate the visibility mask of the th layer and the sum of all visibility masks of its current layer and subsequent layers, and then multiply it by the last multi-scale feature to obtain the visible feature of the th layer, ; where Represents the element-wise multiplication operation, Is the current layer number. Calculating the sum of the visibility masks of all layers from the th layer to the th layer is to consider the occlusion relationship between different layers. When a pixel is occluded in the current layer, its subsequent layer (deeper layer) may provide the visibility information of this pixel, so as to more accurately predict the occluded area. Then through element-wise multiplication, when a pixel is occluded in the current layer, the information of its subsequent layer can make up for the occlusion, thus improving the occlusion inference ability and accuracy of the network.
[0031] (5-2) Visible features of layers and the visibility mask of the layer As the input of the gated convolutional network, Layer Channel prediction, get the .
[0032] (5-3) Repeat (5-1) and (5-2) times, and obtain the multi-spherical image aisle .
[0033] Step (6). Color layer according to normal proportions and aisle , and get the complete multi-sphere image representation At the target viewpoint Sampling is performed, and then a panoramic image of the target viewpoint is rendered to obtain a panoramic image of the target viewpoint .
[0034] (6-1) At the target viewpoint The generated multi-layer spherical image is sampled at The pixel at which the target is located generates a line from the target viewpoint to the pixel. The light of departure , calculate the intersection of the ray with the spherical image at each depth , record each intersection Corresponding color value and value ,get color values and A collection of values : ;in, Representative The spherical radius of the layer, Indicates the position from the target viewpoint The light of departure The unit vector of .
[0035] (6-2) Pair Set Use opacity synthesis algorithm to calculate the target panoramic image Located in The pixel color value at : ; Among them, the cumulative transmittance . Calculate the color value of each position to get a panoramic image Prediction .
[0036] Step (7). Compare the prediction with the real image to train the multi-spherical image generation network through a loss function, where the loss function includes an L2 loss function, a visibility mask consistency loss function, and a color gradient loss function.
[0037] ; where , respectively represent , at the color values.
[0038] Consistency loss of the visibility mask : Calculate the Euclidean distance between the visibility mask predicted by the appearance prediction network according to the real image at and the visibility mask at and feedback it to the multi-spherical image generation network to ensure that the network learns consistent visibility information at different depth layers, avoid unreasonable transparency estimation, and improve the network's adaptability to complex scenes. is a homography matrix operation that maps the mask at the position of the source viewpoint panoramic image to the position of the target viewpoint to align the mask information..
[0039] is the visibility mask predicted by the network with the panoramic image at the source viewpoint position as the input in the decoder module . It is the visibility mask describing the source viewpoint position, so it needs to go through the homography matrix to obtain the target viewpoint position to align the mask information.
[0040] Color gradient loss : This loss is based on the gradient information of the image and constrains the network's learning of edge structures by calculating the second-order derivative of the color distribution: ; where represents the Laplacian filter operator, which can calculate the second-order derivative of the RGB color values of the image, represents the L2 norm. Since the Laplacian filter can highlight the edge regions of the image, this loss term can effectively enhance the network's learning ability of image details and texture information, reduce blurring, and improve the rendering quality.
[0041] Total loss function , and To balance the importance weights of different loss terms and ensure that the network achieves the best optimization effect among geometric structure, color consistency, and texture details, and .
Claims
1. A novel viewpoint synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images, characterized in that: Step (1). Perform data augmentation on the training data set; The described training dataset includes panoramic images for training, and the corresponding positions and camera pose matrices for each panoramic image; the panoramic images include a source viewpoint panoramic image and target viewpoint panoramic images as reference images; Each target viewpoint panoramic image is passed through times of rotational reprojection to obtain the enhanced target viewpoint panoramic image , as well as the corresponding position and the camera pose matrix to enhance the training dataset, ; Step (2). Perform bilinear downsampling on the source viewpoint panoramic image for times to obtain sampled source viewpoint panoramic images : , , denotes bilinear downsampling with a downsampling factor of 2; Original-size source-viewpoint panoramic image and the sampled source-viewpoint panoramic image constitute input panoramic images of different sizes , , ; The input panoramic images are spliced with the spherical coordinate encodings of the corresponding sizes to obtain splicing results of different sizes as the output; The planar coordinates of the input panoramic image are The spherical coordinates on the unit sphere corresponding to the points are , denotes the pixel point of the input panoramic image The azimuth angle mapped onto the unit sphere, denotes the pixel point of the input panoramic image The elevation angle mapped onto the unit sphere: , , and denotes the pixel point of the input panoramic image The width and height coordinates on the plane, and denote the width and height of the input panoramic image respectively; Step (3). Input the splicing result into the pyramid encoder to extract multi-scale features as the output; Step (4). The extracted multi-scale features are input into the appearance prediction network, and after decoding, the color layer is obtained, as well as the visibility mask ; Step (5). The last multi-scale feature among the multi-scale features extracted by the pyramid encoder and the visibility mask are used as the input to the appearance feature decoder and are executed through a gated convolutional network with parameter sharing times to predict the corresponding layer for each layer ; Step (6). Based on the color layer with normal proportion and channel , obtain a complete multi-spherical image representation ; sample at the position of the target viewpoint , and then render the panoramic image of the target viewpoint to obtain the panoramic image of the target viewpoint ; Step (7). Compare the prediction with the real image and train the multi-spherical image generation network through a loss function, where the loss function includes the L2 loss function , the visibility mask consistency loss function and the color gradient loss function ; Total loss function , and are the importance weights for balancing different loss terms.
2. The novel viewpoint synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images according to claim 1, characterized in that: Step (2) Input panoramic image The spherical coordinate encoding calculation process is as follows: Azimuth angle Encoding of , Elevation angle Encoding of , indicating the order; Obtain spherical coordinate encoding , , is the real number field; input panoramic image and the corresponding spherical coordinate encoding are spliced along the third dimension, and the splicing result , , indicates splicing along the third dimension.
3. The novel viewpoint synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images according to claim 2, characterized in that, Specifically, step (3) is: (3-1) Input the splicing results into the pyramid encoder in descending order; the pyramid encoder is composed of residual coding modules connected in series. Each residual coding module consists of convolutions. After each level of convolution, batch normalization and activation functions are connected; input the first splicing result into the first residual module to obtain the first feature vector ; represents feature extraction; (3-2) Starting from the second residual encoding module, the input is composed of the output features from the previous stage after performing bilinear downsampling and the concatenation result at this stage ; ; the output of ; Finally, the multi-scale features extracted by the pyramid encoder composed of 4. The novel view synthesis method of panoramic image based on geometric appearance decoupled multi-spherical images according to claim 3, characterized in that, Specifically, step (4) is: (4-1) Pair Perform non-uniform compression; for multi-scale features At The result after performing non-uniform compression , Indicates the feature vector at the th position in the output feature map, Indicates the absolute value of the pitch angle on the unit sphere corresponding to the vertical coordinate ; For Regions greater than Perform compression, and keep other regions unchanged; Indicates compression, The first step is to split the input features along the first dimension to obtain two vectors of size , and then concatenate them along the third dimension to obtain , and then pass through a convolution module to map the output features to a unified number of channels , and output : ; Perform non-uniform compression on all feature vectors in to obtain multi-scale features after non-uniform compression (4-2) Input the multi-scale features after non-uniform compression into the pyramid decoder , and decode the multi-scale features into color layers , where is the number of layers of the multi-spherical image; (4-3) Use the reverse compression strategy to restore the color layer to the color layer with normal proportion : First, expand the number of channels of the compressed part in to twice the original number of channels through the convolutional layer, and then through pixel rearrangement, rearrange the additional channel information expanded in the vertical direction into the vertical space to obtain ; ; (4-4) Use a decoder module composed of two layers of multi-layer perceptrons Predict The corresponding visibility mask ; where Represents the linear transformation layer of the multi-layer perceptron, Represents the activation function. Through two layers of linear transformation layers, generate The mask corresponding to the layer color layer ; For Apply The activation function to obtain the visibility mask , so that the sum of the visibility weights of each pixel on all MSI layers is .
5. The novel view synthesis method of panoramic image based on geometric appearance decoupled multi-spherical images according to claim 4, characterized in that, Specifically, step (5) is: (5-1) Calculate the visibility mask of the layer, the sum of all visibility masks of its current layer and subsequent layers, and then multiply it by the last multi-scale feature to obtain the visibility feature of the layer, ; where represents an element-wise multiplication operation, is the current layer number; (5-2) Use the visible features of the layer and the visibility mask of this layer as the input of the gated convolutional network to predict the channels of the layer, and obtain the ; ; (5-3) Repeat the execution of (5-1) and (5-2) for times to obtain the channels of the multi-spherical image.
6. The novel viewpoint synthesis method for panoramic images based on geometric appearance decoupled multi-spherical images according to claim 5, characterized in that Specifically, step (6) is: At the target viewpoint sample the generated multi-layer spherical image. For a pixel located at in the target panoramic image, generate a ray starting from the position of the target viewpoint , calculate the intersection points of the ray with the spherical images at each depth , record each intersection point corresponding color value and value , and obtain a set composed of color values and values: ; where represents the spherical radius of the th layer, and denotes the unit vector of the ray starting from the position of the target viewpoint ; (6-2) Pair of sets Using the opacity synthesis algorithm, calculate the pixel color value at in the target panoramic image : ; where the cumulative transmittance ; calculate the color value at each position to obtain the prediction of the panoramic image ; .
7. The novel view synthesis method of panoramic image based on geometric appearance decoupled multi-spherical images according to claim 6, characterized in that In step (7): L2 loss function: ; where , respectively represent , at the color values; Consistency Loss of Visibility Mask : Calculate the visibility mask predicted by the appearance prediction network based on the real image at and the visibility mask at and calculate the Euclidean distance between them, and feedback it to the spherical image generation network at most For the homography matrix operation, map the mask at the position of the source viewpoint panoramic image to the position of the target viewpoint to align the mask information; The mask of Color gradient loss : This loss is based on the gradient information of the image, and constrains the network's learning of edge structures by calculating the second-order derivative of the color distribution: ; where represents the Laplace filter operator, which can calculate the second-order derivative of the RGB color values of the image, represents the L2 norm.