Panoramic image editing method and system for spherical geometry

Abstract

The invention relates to the field of computer vision and image processing, in particular to a panoramic image editing method and system for spherical geometry, and the method comprises the steps: obtaining a to-be-edited panoramic image and an editing instruction of a user, drawing a mask of a user editing dragging region on the panoramic image, and marking a control point and a target point; rotating an area edited by the user to the center of the spherical image; image editing is carried out on the panoramic image after rotation transformation by using a trained point editing diffusion model, and the point editing diffusion model adaptively adjusts an optimization area and an editing track according to spherical latitude information of a control point; and outputting the edited panoramic image. Compared with the prior art, the method has the advantages that the distortion area of the panoramic image is effectively corrected, editing path planning with higher consistency in the panoramic space is realized, and the problem that an existing editing method is difficult to apply to the spherical image is solved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and image processing, and in particular to a panoramic image editing method and system oriented to spherical geometry. Background Art

[0002] In the field of image point editing, the earliest representative work is DragGAN, which uses generative adversarial networks (GANs) to construct a user-interactive image structure manipulation framework, allowing users to customize the transformation of local image structures by dragging key points. However, due to certain limitations in the diversity and stability of GAN-generated images, subsequent research has shifted to diffusion models, such as the publicly available solutions in Chinese patent applications CN202411495543.5 and CN202411993796.5. DragDiffusion is the first study to introduce point editing into the field of diffusion models. By optimizing latent variables, it achieves more stable and high-fidelity image editing effects. However, existing image point generation methods generally target two-dimensional images and are not suitable for spherical panoramic images. Due to differences in projection scale, potential optimization inconsistencies exist in different latitudes, affecting editing accuracy and controllability. In polar regions, there is severe spherical distortion, and the trajectory of editing points is inaccurate, causing structural damage.

[0003] Diffusion models have also made significant progress in panoramic image generation, as exemplified by the proposal in Chinese patent application CN202310603253.7. Representative work, such as SphereDiffusion, employs spherical geometry modeling or cubic surface expansion to address distortion in panoramic image generation. However, existing methods primarily focus on improving the quality and structural consistency of 360-degree image generation and do not address interactive editing of existing images.

[0004] In summary, although there are related studies on image point editing technology and panoramic image generation technology, there is no method that can simultaneously solve the challenges brought by point editing and spherical geometry perception in panoramic image editing. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defect of the above-mentioned existing panoramic image editing technology that is unable to simultaneously solve point editing and spherical geometry perception, and to provide a panoramic image editing method and system for spherical geometry, which is suitable for interactive editing tasks of 360-degree panoramic images in technical scenarios such as virtual reality, autonomous driving, and panoramic image editing.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] As one aspect of the present invention, a method for editing a panoramic image based on spherical geometry is provided, comprising the following steps:

[0008] Obtain the panoramic image to be edited and the user's editing instructions, draw a mask of the user's editing drag area on the panoramic image, and mark the control points and target points;

[0009] Rotate the user edited area to the center of the spherical image;

[0010] For the panoramic image after rotation transformation, the trained point editing diffusion model is used to adaptively adjust the optimization area and edit the trajectory according to the spherical latitude information of the control points, and the edited panoramic image is output.

[0011] As a preferred technical solution, the user-edited area is rotated and transformed to the center of the spherical image as follows:

[0012] Calculate the midpoint of the control point and the target point and convert it into three-dimensional coordinates on the 3D sphere;

[0013] Generate a rotation matrix based on the three-dimensional coordinates of the midpoint between the obtained control point and the target point;

[0014] Use the rotation matrix to uniformly rotate the input panorama, mask, control points, and target points to the center area of the sphere to complete the image reprojection.

[0015] As a preferred technical solution, the point editing diffusion model is specifically as follows:

[0016] The denoising diffusion implicit model (DDIM) is used to invert and extract the latent variable sequence of multiple time steps corresponding to the panoramic image.

[0017] Select the latent variable at the nth time step as the target latent variable and set it as the initial latent variable;

[0018] A feature map is extracted from the initial latent variables, and K iterative optimizations are performed based on the feature map. In each iteration, the system adaptively adjusts the size of the optimization area and the direction of the editing trajectory according to the location of the node.

[0019] As a preferred technical solution, the size of the optimization area is adaptively adjusted and the trajectory is edited as follows:

[0020] During the kth iteration, the editing trajectory direction of the current control point is set along the great circle direction of the sphere:

[0021]

[0022] Where d represents the editing trajectory direction between two iterations; Indicates the editing trajectory direction calculation function; h krepresents the current control node; h represents the initially set control point; t represents the initially set target point;

[0023] Based on the latitude of the control point, calculate the motion feature acquisition area:

[0024] Calculate the acquisition width:

[0025]

[0026] Where r(φ) represents the acquisition width; r b is a hyperparameter, indicating the basic width; φ is the current latitude value;

[0027] Set the acquisition area according to the acquisition width:

[0028]

[0029]

[0030] Where h k Indicates the current control node, h k +d means the control node will be moved according to the editing trajectory direction; Ω(h k ,r b ) represents the current control node h k The collection area; q represents the point in the collection area;

[0031] Based on the editing trajectory direction and the acquisition area, the point editing diffusion model is used to move the feature map along the predetermined route through the motion loss function;

[0032] After the kth iteration, the control points are re-established and the control point tracking formula is:

[0033]

[0034] Where, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; F h (·) represents the diffusion model and the feature map extracted from the acquisition area based on the initial control points; represents the latent variable of the k+1th iteration; w 0 represents the latent variable that was not modified.

[0035] As a preferred technical solution, the loss function of the point editing diffusion model is set as follows:

[0036]

[0037] Where, represents the latent variable at the kth iteration; Represents the diffusion model based on the moved control point h k +d feature map extracted from the acquisition area, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; sg(·) indicates stopping the gradient calculation; Indicates the initial latent variables in the kth iteration The intermediate representation after a DDIM reverse denoising inference; represents the n-1th step latent variable obtained by reverse denoising inference of the original image through DDIM. It serves as a static reference item and does not participate in training; M represents the mask value set by the user; ||·||1 represents the L1 norm; and λ represents the weight coefficient.

[0038] As an optimal technical solution, the point editing diffusion model uses LoRA for fine-tuning training during the training phase, and the LoRA weights obtained from the training fine-tuning are saved for use in the subsequent image editing reasoning process.

[0039] As one aspect of the present invention, a panoramic image editing system for spherical geometry is provided. The system executes the above-mentioned panoramic image editing method for spherical geometry, specifically comprising:

[0040] The user input module receives the panoramic image and the user's editing instructions, draws a mask of the user's editing and dragging area on the panoramic image, and marks the control points and target points;

[0041] The area adaptive reprojection module moves the user edit point area to the center of the sphere by rotating the sphere;

[0042] a point editing module that performs image editing on the rotationally transformed panoramic image based on a diffusion model;

[0043] The latitude adaptive operation module adaptively adjusts the optimization area and edits the trajectory according to the spherical latitude information of the control point.

[0044] As a preferred technical solution, the region-adaptive reprojection module is specifically as follows:

[0045] Calculate the midpoint of the control point and the target point and convert it into three-dimensional coordinates on the 3D sphere;

[0046] Generate a rotation matrix based on the three-dimensional coordinates of the midpoint between the obtained control point and the target point;

[0047] Use the rotation matrix R to uniformly rotate the input panorama, mask, control points and target points to the center area of the sphere to complete the image reprojection.

[0048] As a preferred technical solution, the latitude adaptive operation module adaptively adjusts the size of the optimization area and edits the trajectory, as follows:

[0049] During the kth iteration, the editing trajectory direction of the current control point is set along the great circle direction of the sphere:

[0050]

[0051] Where d represents the editing trajectory direction between two iterations; Indicates the editing trajectory direction calculation function; h k represents the current control node; h represents the initially set control point; t represents the initially set target point;

[0052] Based on the latitude of the control point, calculate the motion feature acquisition area:

[0053] Calculate the acquisition width:

[0054]

[0055] Where r(φ) represents the acquisition width; r b is a hyperparameter, indicating the basic width; φ is the current latitude value;

[0056] Set the acquisition area according to the acquisition width:

[0057]

[0058]

[0059] Where h k Indicates the current control node, h k +d means the control node will be moved according to the editing trajectory direction; Ω(h k ,r b ) represents the current control node h k The collection area; q represents the point in the collection area;

[0060] Based on the editing trajectory direction and the acquisition area, the point editing diffusion model is made to move the feature map along the established route through the motion loss function. The loss function of the point editing diffusion model is set as follows:

[0061]

[0062] Where, represents the latent variable at the kth iteration; Represents the diffusion model based on the moved control point h k +d feature map extracted from the acquisition area, Indicates that the diffusion model is based on the current control point h kThe feature map extracted from the acquisition area; sg(·) indicates stopping the gradient calculation; Indicates the initial latent variables in the kth iteration The intermediate representation after a DDIM reverse denoising inference; represents the n-1th step latent variable obtained by DDIM reverse denoising inference of the original image. It serves as a static reference and does not participate in training; M represents the mask value set by the user; ||·||1 represents the L1 norm; and λ represents the weight coefficient. After the kth iteration, the control points are re-established. The control point tracking formula is:

[0063]

[0064] Where, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; F h (·) represents the diffusion model and the feature map extracted from the acquisition area based on the initial control points; represents the latent variable of the k+1th iteration; w 0 represents the latent variable that was not modified.

[0065] Finally, by combining motion supervision and point tracking, we successfully generate high-quality panoramic images with precise and controllable editing.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] 1) This paper proposes a panoramic image editing method for spherical geometry. This method introduces a spherical projection association module, integrates knowledge of spherical geometry, adjusts the feature extraction region based on spherical latitude, and sets the motion direction to a great circle during path optimization, overcoming the limitations of planar optimization methods and making the optimization more consistent with spherical motion. A diffusion editing model uses a motion loss function for motion supervision and tracking of control points to effectively correct distorted areas in panoramic images and achieve more consistent editing path planning in panoramic space, thus addressing the difficulty of applying existing editing methods to spherical images.

[0068] 2) The regional adaptive reprojection module proposed in the present invention moves the user editing point area to the center of the sphere through spherical rotation, reducing the distortion interference of the image polar region.

[0069] 3) This paper proposes a latitude-adaptive latent variable manipulation module. This module selects target latent variables from the diffusion model's latent space and adaptively adjusts and optimizes the acquisition area based on the spherical latitude information of the current control point. This optimization is guided by a great-circle path. This overcomes the limitations of planar optimization methods and makes the dragging process more consistent with the geometric characteristics of spherical images.

[0070] 4) The solution of the present invention is easy to implement and reproduce in a modular manner; it can be directly deployed on an image editing platform based on StableDiffusion; and it can be widely used in industrial scenarios such as virtual reality, panoramic live broadcast, and autonomous driving image perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a schematic diagram of the structure of the spherical panoramic image point drag editing model proposed by the present invention;

[0072] Figure 2 This is a flow chart of the inference phase of the spherical panoramic image point drag editing model of the present invention;

[0073] Figure 3 Comparison chart of the visualization generation results of the solution of the present invention and the existing image editing baseline model, a) user drag editing instruction, b) DragDiffusion generation result, c) DragNoise generation result, d) StableDrag generation result, e) generation result of the solution of the present invention. DETAILED DESCRIPTION

[0074] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0075] Example 1

[0076] This paper proposes a spherical image editing method based on the DragDiffusion framework. By introducing a region-adaptive reprojection module and a latitude-adaptive optimization module, it enables high-precision dragging and editing of arbitrary regions in a panoramic image. The entire method is divided into two phases: training and inference.

[0077] During the training phase, this example uses LoRA (Low-Rank Adaptation) technology for fine-tuning based on the existing Stable Diffusion V1.5 diffusion model. By introducing a small number of training parameters, LoRA can adapt the model to the dragging task without modifying the original model's large number of parameters. After training, the fine-tuned LoRA weights are saved and used in subsequent image editing inference processes.

[0078] In the inference phase, the user provides a 2:1 scale panoramic image, draws a mask of the drag area on the image, and marks a control point (original position) and a target point (the position to be moved to). The mask is used to indicate the area in the image that is allowed to be modified by the model; the non-masked area will be strictly kept unchanged to ensure the locality and consistency of the editing. After receiving these input information, the image editing process is automatically executed in combination with the trained LoRA weights. In order to achieve more natural and accurate editing effects, such as Figure 1 As shown, the present invention designs and introduces two key modules during the inference phase: a region-adaptive reprojection module, which rotates the user's edited area to the center of the spherical image to reduce image distortion. A latitude-adaptive operation module, which adaptively adjusts the optimized area and editing trajectory based on the spherical latitude information of the control points, ensuring that the dragging process better conforms to the geometric characteristics of the spherical image. This ultimately outputs a high-fidelity panoramic image that meets the user's intended editing intent.

[0079] like Figure 2 As shown in Figure 2, the image point generation steps in the inference phase are as follows:

[0080] Step 1: First, obtain the panoramic image provided by the user, draw a mask of the user edited and dragged area on the panoramic image, and mark the control points and target points.

[0081] Step 2. Set up the region-adaptive reprojection module: rotate the user-edited region to the center of the spherical image to reduce image distortion interference.

[0082] 2.1.1) First calculate the midpoint of the control point and the target point and convert it into three-dimensional coordinates on the 3D sphere:

[0083]

[0084] Where, T 2d23d (·) indicates converting a two-dimensional coordinate point into a three-dimensional coordinate point on a sphere; i m ,j m Indicates the midpoint coordinates of the control point and the target point; The 3D coordinates representing the midpoint between the control point and the target point.

[0085] 2.1.2) Generate the rotation matrix R based on the 3D coordinates of the midpoint between the obtained control point and the target point:

[0086]

[0087] Where R align (·) represents the alignment matrix calculation formula.

[0088] 2.1.3) Use the rotation matrix R to uniformly rotate the input panorama Image, mask Mask, control points, and target point Point to the center of the sphere to complete the image reprojection:

[0089] [I′,M′,P′]=T R (I,M,P).

[0090] Step 3: The trained Stable Diffusion model performs image editing based on the center of the spherical image after rotation transformation.

[0091] Step 4. During the optimization of the latent variables, the latent variable sequence of 50 time steps corresponding to the image is first extracted through inversion using the Denoising Diffusion Implicit Model (DDIM). In this embodiment, the latent variable of the 35th time step is selected as the target latent variable, which is called the initial latent variable. A feature map is extracted from the initial latent variable, and K iterative optimizations are performed based on it, where K is the total number of iterative optimizations. In each iteration, the system adaptively adjusts the size of the optimization area and the direction of the editing trajectory based on the location of the node, thereby ensuring that the dragging operation conforms to the geometric properties of the spherical image and achieves a geometrically consistent editing effect.

[0092] 4.1) During the kth iteration, the editing trajectory direction of the current control point is set along the great circle direction of the sphere, which is defined by the following formula:

[0093]

[0094] Where d represents the editing trajectory direction between two iterations; Indicates the editing trajectory direction calculation function; h k represents the current control node; h represents the initially set control point; t represents the initially set target point.

[0095] 4.2) After determining the direction of movement, in order to more accurately capture the changes in node feature areas at different latitudes, a latitude-adaptive feature area size acquisition mechanism is introduced.

[0096] Calculate the acquisition width:

[0097]

[0098] Where r(φ) represents the acquisition width obtained by combining the latitude information; r b is a hyperparameter representing the base width; φ is the node's current latitude. The higher the latitude, the larger the sampling area needs to be in spherical projection to maintain a consistent sampling density in real space.

[0099] Set the acquisition area based on width:

[0100] Ω(h k ,r b )={q=(x,y):|x-(h k ) x |≤r b ,|y-(h k ) y |≤r(φ hk )}

[0101]

[0102] Where h k Indicates the current control node, h k +d means the control node will be moved according to the editing trajectory direction; Ω(h k ,r b ) represents the current control node h k The acquisition area, Ω(h k +d,r b ) indicates the control node after moving (h k +d) of the collection area; q represents a point in the collection area.

[0103] 4.3) After setting the motion path and acquisition area, we design a motion loss function to make the model move features along the established route. The loss function is as follows:

[0104]

[0105] In the formula, the left side of the plus sign is to ensure that the control node h k Move to h k +d, the plus sign on the right ensures that the feature map of the area outside the mask remains unchanged. represents the latent variable at the kth iteration, Represents the diffusion model according to the control point h after movement k +d feature map extracted from the acquisition area, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area, sg(·) stops the gradient calculation; Indicates that in the kth iteration, from the current initial latent variable The intermediate representation after a DDIM reverse denoising inference (i.e., the 34th step latent variable); represents the 34th step latent variable obtained by the DDIM reverse denoising inference of the original image. It serves as a static reference and does not participate in training. M represents the value of the mask set by the user, which is used to control the mask area to remain unchanged. ||·||1 represents the L1 norm, which is used to measure the difference between two feature maps. λ represents the weight coefficient, which is used to adjust the influence of the constraint of the area outside the mask on the total loss.

[0106] 4.4) After the kth iteration, the control points need to be re-established. The control point tracking formula is:

[0107]

[0108] Where, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; F h (·) represents the diffusion model and the feature map extracted from the acquisition area based on the initial control points; represents the latent variable of the k+1th iteration; w 0 represents the latent variable that was not modified.

[0109] Step 5: After the editing is completed, a panoramic image with high fidelity that meets the user's expected editing intention is output.

[0110] The spherical image editing method based on the DragDiffusion framework proposed in the present invention realizes high-precision dragging and editing of any area in the panoramic image by introducing a regional adaptive reprojection module and a latitude adaptive optimization module. The image editing scheme of the present invention is compared with the existing baseline model. In order to ensure the consistency and fairness of the evaluation, all experiments are based on the Stable Diffusion v1.5 pre-trained model and trained using a unified LoRA fine-tuning method. The specific settings are as follows: the learning rate is uniformly set to 0.01, the maximum number of optimization iterations is 80, and the image resolution is fixed to 1024×512. In the inference stage, the trained LoRA weights are loaded, and the panoramic image, mask, control points and target points input by the user are combined to complete the automatic editing and generation of the image. All experiments are run on a server equipped with an L40S GPU. The rest of the training and inference processes are consistent with DragDiffusion to ensure the objectivity and repeatability of the comparison results.

[0111] Experimental results at different field of view angles show that this method maintains stable performance advantages across low, medium, and high viewing angles. Compared to other image editing algorithms (such as StableDrag and DragNoise), the proposed panoramic image point editing solution performs better overall. Specific experimental comparison results can be found in Table 1.

[0112] Table 1 Quantitative analysis of the model

[0113]

[0114] The visual analysis further verifies the advantages of the proposed solution in actual editing effects. Figure 3 As shown, in the dragging example with a 90° field of view: the first row of examples demonstrates that the solution of the present invention can effectively solve the serious artifact problem produced by the traditional method at the projection stitching with the help of the latitude adaptive optimization module, while the baseline model DragDiffusion has obvious edge misalignment under the same setting; the second row of examples shows that this method achieves a more natural and smooth drag deformation propagation effect, while other competing methods have structural breaks and confusion; the chair editing example in the third row shows that this method can reconstruct the user-specified target shape with high fidelity, while StableDrag and DragNoise both have structural ambiguity problems in this scenario. Overall, the solution of the present invention is superior to the traditional planar dragging method in key indicators such as dragging accuracy, shape restoration and structural fidelity, reflecting the significant performance improvement of the present invention in spherical image editing tasks.

[0115] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A panoramic image editing method based on spherical geometry, characterized in that the steps include: Obtain the panoramic image to be edited and the user's editing instructions, draw a mask of the user's editing drag area on the panoramic image, and mark the control points and target points; Rotate the user edited area to the center of the spherical image; For the panoramic image after rotation transformation, the trained point editing diffusion model is used to adaptively adjust the optimization area and edit the trajectory according to the spherical latitude information of the control points, and the edited panoramic image is output.

2. The spherical geometry-oriented panoramic image editing method according to claim 1, characterized in that: The rotation transformation of the user edited area to the center of the spherical image is as follows: Calculate the midpoint of the control point and the target point and convert it into three-dimensional coordinates on the 3D sphere; Generate a rotation matrix based on the three-dimensional coordinates of the midpoint between the obtained control point and the target point; Use the rotation matrix to uniformly rotate the input panorama, mask, control points, and target points to the center area of the sphere to complete the image reprojection.

3. The method for editing panoramic images based on spherical geometry according to claim 1, characterized in that: The point editing diffusion model is as follows: The denoising diffusion implicit model (DDIM) is used to invert and extract the latent variable sequence of multiple time steps corresponding to the panoramic image. Select the latent variable at the nth time step as the target latent variable and set it as the initial latent variable; A feature map is extracted from the initial latent variables, and K iterative optimizations are performed based on the feature map. In each iteration, the system adaptively adjusts the size of the optimization area and the direction of the editing trajectory according to the location of the node.

4. The method for editing panoramic images based on spherical geometry according to claim 3, characterized in that: The adaptive adjustment of the optimization area size and editing of the trajectory are as follows: During the kth iteration, the editing trajectory direction of the current control point is set along the great circle direction of the sphere: Where d represents the editing trajectory direction between two iterations; Indicates the editing trajectory direction calculation function; h k represents the current control node; h represents the initially set control point; t represents the initially set target point; Based on the latitude of the control point, calculate the motion feature acquisition area: Calculate the acquisition width: Where r(φ) represents the acquisition width; r b is a hyperparameter, indicating the basic width; φ is the current latitude value; Set the acquisition area according to the acquisition width: Where h k Indicates the current control node, h k +d means the control node will be moved according to the editing trajectory direction; Ω(h k ,r b ) represents the current control node h k The collection area; q represents the point in the collection area; Based on the editing trajectory direction and the acquisition area, the point editing diffusion model is used to move the feature map along the predetermined route through the motion loss function; After the kth iteration, the control points are re-established and the control point tracking formula is: Where, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; F h (·) represents the diffusion model and the feature map extracted from the acquisition area based on the initial control points; represents the latent variable of the k+1th iteration; w 0 represents the latent variable that was not modified.

5. The method for editing panoramic images based on spherical geometry according to claim 4, characterized in that: The loss function of the point editing diffusion model is set as follows: Where, represents the latent variable at the kth iteration; Represents the diffusion model based on the moved control point h k +d feature map extracted from the acquisition area, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; sg(·) indicates stopping the gradient calculation; Indicates the initial latent variables in the kth iteration The intermediate representation after a DDIM reverse denoising inference; represents the n-1th step latent variable obtained by reverse denoising inference of the original image through DDIM. It serves as a static reference item and does not participate in training; M represents the mask value set by the user; ||·||1 represents the L1 norm; and λ represents the weight coefficient.

6. The method for editing panoramic images based on spherical geometry according to claim 1, characterized in that: The point editing diffusion model is fine-tuned using LoRA during the training phase, and the LoRA weights obtained from the training fine-tuning are saved for use in the subsequent image editing reasoning process.

7. A panoramic image editing system for spherical geometry, characterized in that: The system executes the spherical geometry-oriented panoramic image editing method according to any one of claims 1 to 6, specifically comprising: The user input module receives the panoramic image and the user's editing instructions, draws a mask of the user's editing and dragging area on the panoramic image, and marks the control points and target points; The area adaptive reprojection module moves the user edit point area to the center of the sphere by rotating the sphere; a point editing module that performs image editing on the rotationally transformed panoramic image based on a diffusion model; The latitude adaptive operation module adaptively adjusts the optimization area and edits the trajectory according to the spherical latitude information of the control point.

8. The spherical geometry-oriented panoramic image editing system according to claim 7, characterized in that: The region-adaptive reprojection module is specifically as follows: Calculate the midpoint of the control point and the target point and convert it into three-dimensional coordinates on the 3D sphere; Generate a rotation matrix based on the three-dimensional coordinates of the midpoint between the obtained control point and the target point; Use the rotation matrix R to uniformly rotate the input panorama, mask, control points and target points to the center area of the sphere to complete the image reprojection.

9. The spherical geometry-oriented panoramic image editing system according to claim 7, characterized in that: The latitude adaptive operation module adaptively adjusts the size of the optimization area and edits the trajectory as follows: During the kth iteration, the editing trajectory direction of the current control point is set along the great circle direction of the sphere: Where d represents the editing trajectory direction between two iterations; Indicates the editing trajectory direction calculation function; h k represents the current control node; h represents the initially set control point; t represents the initially set target point; Based on the latitude of the control point, calculate the motion feature acquisition area: Calculate the acquisition width: Where r(φ) represents the acquisition width; r b is a hyperparameter, indicating the basic width; φ is the current latitude value; Set the acquisition area according to the acquisition width: Where h k Indicates the current control node, h k +d means the control node will be moved according to the editing trajectory direction; Ω(h k ,r b ) represents the current control node h k The collection area; q represents the point in the collection area; Based on the editing trajectory direction and the acquisition area, the point editing diffusion model is used to move the feature map along the predetermined route through the motion loss function; After the kth iteration, the control points are re-established and the control point tracking formula is: Where, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; F h (·) represents the diffusion model and the feature map extracted from the acquisition area based on the initial control points; represents the latent variable of the k+1th iteration; w 0 represents the latent variable that was not modified.

10. The spherical geometry-oriented panoramic image editing system according to claim 9, characterized in that: The motion loss function of the point editing diffusion model is set as follows: Where, represents the latent variable at the kth iteration; Represents the diffusion model according to the control point h after movement k +d feature map extracted from the acquisition area, Indicates that the diffusion model is based on the current control point h k The feature map extracted from the acquisition area; sg(·) indicates stopping the gradient calculation; Indicates the initial latent variables in the kth iteration The intermediate representation after a DDIM reverse denoising inference; represents the n-1th step latent variable obtained by reverse denoising inference of the original image through DDIM. It serves as a static reference item and does not participate in training; M represents the mask value set by the user; ||·||1 represents the L1 norm; and λ represents the weight coefficient.

Images