Method and apparatus for splicing irregular fragments based on an end-to-end diffusion model

Abstract

This application belongs to the field of computer vision technology, specifically disclosing a method and apparatus for irregular fragment stitching based on an end-to-end diffusion model. The method includes: randomly cutting multiple images into several irregular fragments, approximating the edges of the irregular fragments as polygons to obtain a dataset; training an end-to-end diffusion model based on the dataset to obtain a trained model; and inputting the fragments of the image to be stitched into the trained model to obtain a complete stitched image. This method can improve the fragment stitching speed and accuracy.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and more specifically, relates to a method and apparatus for stitching irregular fragments based on an end-to-end diffusion model. Background Technology

[0002] Existing fragment stitching methods can be divided into two categories: heuristic-based methods and deep neural network-based methods, and end-to-end diffusion model-based methods, among which:

[0003] Heuristic methods primarily learn the edge and color features of fragments and search the solution space for the stitching method that achieves the highest global matching score. These methods include fragment stitching based on genetic algorithms, simulated annealing, and discrete artificial bee colony algorithms. The main drawbacks of these methods are high computational complexity, slow computation speed, and low stitching accuracy.

[0004] Deep neural network-based methods improve the accuracy of fragment stitching by learning more complex image features, including convolutional neural network-based and residual network-based methods. However, these methods have weak geometric inference capabilities, are limited to stitching rectangular fragments, and have lower accuracy in stitching irregular fragments.

[0005] Therefore, existing technologies cannot simultaneously achieve both high accuracy and fast splicing speed in fragment assembly. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this application is to provide an irregular fragment splicing method and apparatus based on an end-to-end diffusion model, which aims to solve the problem that existing fragment splicing schemes cannot simultaneously achieve both fragment splicing accuracy and splicing speed.

[0007] To achieve the above objectives, in a first aspect, this application provides a method for splicing irregular fragments based on an end-to-end diffusion model, comprising:

[0008] Multiple images are randomly cut into several irregular fragments, and the edges of the irregular fragments are approximated as polygons to obtain a dataset;

[0009] The end-to-end diffusion model is trained based on the dataset to obtain the trained model.

[0010] The fragments of the image to be stitched are input into the trained model to obtain the complete image after stitching.

[0011] This application creates an image fragment dataset and uses it to train an end-to-end diffusion model, which is used to stitch irregular fragments into a complete image. Using the diffusion model can improve the fragment stitching speed and stitching accuracy.

[0012] According to the irregular fragment stitching method based on an end-to-end diffusion model provided in this application, the dataset includes one or more of the following:

[0013] Fragment datasets of cross-sectioned natural images, fragment datasets of face images after Vino cuts, and Magic Plan datasets of room layouts.

[0014] This application uses a fragment dataset of transverse natural images, a fragment dataset of face images after Vino cuts, and a Magic Plan dataset of room layouts to train the diffusion model, which can improve the fragment stitching speed and stitching accuracy of the diffusion model.

[0015] According to the irregular fragment stitching method based on an end-to-end diffusion model provided in this application, the step of training the end-to-end diffusion model based on the dataset to obtain the trained model includes:

[0016] Based on the displacement distance and rotation angle of the fragments in the dataset, determine the state vector of the fragments;

[0017] Noise is gradually added to the state vector to make the fragments become disordered.

[0018] Piecing together disordered fragments into a complete image.

[0019] This application applies a diffusion model to the fragment stitching process by using image features and shape encoding, which can improve the accuracy and consistency of fragment stitching, as well as increase the stitching speed.

[0020] According to the irregular fragment stitching method based on an end-to-end diffusion model provided in this application, the step of stitching disordered fragments into a complete image includes:

[0021] Using linear layers and multilayer perceptrons, feature encoding is initialized for fragments in an unordered state to obtain initialized feature encodings;

[0022] The feature encoding is processed based on polygon self-attention mechanism and global self-attention mechanism;

[0023] The processed feature encoding is converted into a 4-dimensional state vector and then denoised.

[0024] This application employs two self-attention mechanisms during training: polygonal self-attention and global self-attention. These mechanisms can improve the accuracy and consistency of fragment stitching, while also increasing the stitching speed.

[0025] According to the irregular fragment stitching method based on an end-to-end diffusion model provided in this application, the method uses linear layers and multilayer perceptrons to initialize the feature encoding of the disordered fragments, obtaining the initialized feature encoding, including:

[0026] A linear layer is used to transform the 4-dimensional state vector of each vertex of the fragment into a 256-dimensional feature vector;

[0027] Use a linear layer to transform the conditional vector into a 256-dimensional vector;

[0028] A two-layer multilayer perceptron is used to convert time into a 256-dimensional vector.

[0029] Image features of fragments are extracted using a linear layer, and the image features are converted into 256-dimensional vectors.

[0030] According to the irregular fragment stitching method based on an end-to-end diffusion model provided in this application, the step of training the end-to-end diffusion model based on the dataset includes:

[0031] Based on the dataset and loss function, the end-to-end diffusion model is trained;

[0032] The loss function is a weighted sum of standard noise regression loss, matching loss, and overlap loss.

[0033] This application introduces three loss functions in the reverse process, which can improve the accuracy and consistency of fragment splicing, and also increase the splicing speed.

[0034] Secondly, this application provides an irregular fragment splicing device based on an end-to-end diffusion model, comprising:

[0035] The cutting module is used to randomly cut multiple images into several irregular fragments, and approximate the edges of the irregular fragments as polygons to obtain a dataset;

[0036] The training module is used to train the end-to-end diffusion model based on the dataset to obtain the trained model.

[0037] The stitching module is used to input fragments of the image to be stitched into the trained model to obtain a complete image after stitching.

[0038] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the irregular fragment splicing method based on an end-to-end diffusion model described in the first aspect or any possible implementation thereof.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the irregular fragment splicing method based on an end-to-end diffusion model as described in the first aspect or any possible implementation of the first aspect.

[0040] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to execute the irregular fragment splicing method based on an end-to-end diffusion model described in the first aspect or any possible implementation of the first aspect.

[0041] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0042] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0043] (1) By creating an image fragment dataset and using it to train an end-to-end diffusion model, which is used to stitch irregular fragments into a complete image, the speed and accuracy of fragment stitching can be improved.

[0044] (2) The diffusion model is applied to the fragment splicing process by means of image features and shape encoding. Two self-attention mechanisms, polygon self-attention and global self-attention, are proposed. Three loss functions are introduced in the reverse process, which can improve the accuracy and consistency of fragment splicing, and also improve the splicing speed. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the irregular fragment splicing method based on an end-to-end diffusion model provided in an embodiment of this application.

[0047] Figure 2 This is a schematic diagram of the dataset provided in the embodiments of this application;

[0048] Figure 3 This is a schematic diagram of the irregular fragment splicing device based on an end-to-end diffusion model provided in an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0052] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0053] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0054] Next, combined Figures 1-2 This application introduces a method for splicing irregular fragments based on an end-to-end diffusion model, as provided in the embodiments of this application.

[0055] Figure 1 This is a flowchart illustrating the irregular fragment stitching method based on an end-to-end diffusion model provided in this application embodiment, as shown below. Figure 1 As shown, the method includes the following steps:

[0056] Step 100: Randomly cut multiple images into several irregular fragments, and approximate the edges of the irregular fragments as polygons to obtain the dataset;

[0057] First, collect various images and randomly cut them into several irregular fragments. Approximate the edges of the irregular fragments as polygons and save the coordinate information of the polygon vertices.

[0058] Alternatively, a dynamic programming (DP) algorithm can be used to approximate the edges of the fragments as polygons.

[0059] The irregular fragments whose edges are approximated as polygons are used as the dataset.

[0060] Step 110: Train the end-to-end diffusion model based on the dataset to obtain the trained model;

[0061] End-to-end diffusion models are a type of model used in the fields of deep learning and neural networks to generate data. They simulate the physical diffusion process of data generation by reversing the diffusion process to generate new data samples.

[0062] After obtaining the dataset, the end-to-end diffusion model is trained based on the dataset to obtain the trained model, which is then used to piece together irregular fragments.

[0063] Step 120: Input the fragments of the image to be stitched into the trained model to obtain the complete image after stitching.

[0064] After the model is trained, the fragments of the image to be stitched are input into the model to output the stitched complete image.

[0065] This application provides an irregular fragment stitching method based on an end-to-end diffusion model. By creating an image fragment dataset and using it to train an end-to-end diffusion model, the diffusion model is used to stitch irregular fragments into a complete image. Using the diffusion model can improve the fragment stitching speed and stitching accuracy.

[0066] In some embodiments, the dataset includes one or more of the following:

[0067] Fragment datasets of cross-sectioned natural images, fragment datasets of face images after Vino cuts, and Magic Plan datasets of room layouts.

[0068] Figure 2 This is a schematic diagram of the dataset provided in an embodiment of this application, such as... Figure 2As shown, three datasets with different distribution characteristics can be selected, including the ImageNet 256×256 fragment dataset (Cross-cut) which cross-cuts natural images, the FFHQ 256×256 fragment dataset (Voronoi) which cross-cuts face images, and the Magic Plan dataset (RPLAN) which shows room layouts. All three datasets contain training and testing datasets.

[0069] In one embodiment of this application, the transverse fragment dataset is obtained as follows: ImageNet 256×256 natural images are randomly cut, with each cutting line traversing the entire image to ensure the sum of the angles between the fragments is π. Each image is cut into approximately 20 fragments. To simulate potential damage or deformation of fragments in real-world scenarios, Gaussian noise with variances of 0, 1, and 2 is added to the vertex coordinates of the fragments. A total of 100,000 images were created for training, and 1,000 images were used for testing.

[0070] In one embodiment of this application, the fragment dataset obtained after Vino segmentation of face images is obtained as follows: A Vino segmentation is performed on an FFHQ 256×256 face image. 10-20 points are randomly selected in the middle of each image, and the perpendicular bisector of the line connecting any two points is used as the segmentation line to divide the image into fragments. Similar to the transverse segment dataset, Gaussian noise with variances of 0, 1, and 2 is added to the vertex coordinates of the fragments to simulate real-world scenarios. A total of 200,000 images were created for training, and 1,000 images were used for testing.

[0071] In one embodiment of this application, the Magic Plan dataset for room layouts is internal data from a construction company, containing the room shapes and layouts of 100,000 single-story houses. Each room's plan shape is treated as a fragment, and the overall floor plan of the entire house is considered a complete image. Different rooms are distinguished by different colors. The final dataset has 3-10 fragments, each fragment being a 3-7-sided polygon. 80,000 data points are used for training, and 20,000 data points are used for testing.

[0072] In some embodiments, step 110 specifically includes:

[0073] Step 1101: Determine the state vector of the fragment based on the displacement distance and rotation angle of the fragment in the dataset;

[0074] Step 1102: Gradually add noise to the state vector to make the fragments become disordered.

[0075] Step 1103: Piece together the disordered fragments into a complete image.

[0076] The basic principle of diffusion models can be divided into two main processes: forward diffusion and reverse diffusion.

[0077] During the forward diffusion process, let the state vector of vertex i of fragment r at time t be denoted as . , represented as . The two-dimensional displacement vector representing the center of the fragment consists of displacements in the x and y directions. Represents the rotation vector, that is, the vector obtained from the rotation matrix, including... A two-dimensional vector.

[0078] Noise is gradually added to the state vector, causing the fragments to eventually become disordered. The noise sequence forms a Markov chain and satisfies the following expression:

[0079]

[0080] in, This represents the noise at time t. This represents the mean of a standard Gaussian distribution. This represents the variance of the standard Gaussian distribution.

[0081] noise It is standard Gaussian noise. and standard cosine noise A linear combination of .

[0082] Specifically, , , .

[0083] The reverse diffusion process is the inverse of the forward diffusion process; it's the process of piecing together disordered fragments with added noise into a complete image. and Under the condition of deriving This is equivalent to the noise added during the prediction at time t. .

[0084] In the known Predicting noise under the given conditions Obtained by removing noise The noise reduction process is repeated continuously until the final result is obtained. This refers to the final stitched image.

[0085] In some embodiments, step 1103 specifically includes:

[0086] Step 11031: Use a linear layer and a multilayer perceptron to initialize the feature encoding of the unordered fragments and obtain the initialized feature encoding.

[0087] Step 11032: Process the feature encoding based on polygon self-attention mechanism and global self-attention mechanism;

[0088] Step 11033: Convert the processed feature encoding into a 4-dimensional state vector and then denoise it.

[0089] The back-diffusion process is based on the transformer architecture. The vertices of each approximate polygon of each fragment are used as nodes of the transformer architecture. The feature encoding is initialized using linear layers and multilayer perceptrons to obtain the initialized feature encoding.

[0090] After initialization, feature encoding requires six self-attention modules, which include two self-attention mechanisms: polygonal self-attention and global self-attention.

[0091] Among them, polygonal self-attention is similar to sparse self-attention, which only restricts the changes in the state vectors between vertices within the same fragment. This is beneficial for generating consistent displacements and rotations between different vertices within the same fragment.

[0092] Alternatively, to reduce computation and memory consumption, polygon self-attention can adopt a similar approach to sparse self-attention, calculating the similarity of vertices within the same approximate polygon only.

[0093] Global self-attention is a standard self-attention method that considers all vertices of all approximate polygons and takes into account the variations between different fragments, thus facilitating the generation of globally optimal translations and rotations between different fragments.

[0094] After the self-attention module, a linear layer is used to transform the 256-dimensional vector back into a 4-dimensional state vector. Then, a noise reduction step is performed, and the specific formula is as follows:

[0095]

[0096] Among them, when , ,otherwise .

[0097] According to the recursive formula, it can be calculated that That is, the displacement vector and rotation vector of the i-th vertex of the r-th approximate polygon.

[0098] exist When the reverse process ends, for the r-th approximate polygon, the average of the calculation results of all vertices can be taken as the final translation vector and rotation vector to reduce the error.

[0099] In some embodiments, step 11031 specifically includes:

[0100] Step 110311: Use a linear layer to convert the 4-dimensional state vector of each vertex of the fragment into a 256-dimensional feature vector;

[0101] Step 11032: Use a linear layer to convert the conditional vector into a 256-dimensional vector;

[0102] Step 11033: Use a 2-layer multilayer perceptron to convert time into a 256-dimensional vector;

[0103] Step 11034: Use a linear layer to extract image features of the fragments and convert the image features into a 256-dimensional vector.

[0104] Feature encoding initialized to .

[0105] in, Use a linear layer to process the 4D state vector of each vertex. Convert to a 256-dimensional feature vector. A linear layer is also used to convert the conditional vector (composed of fragment shape, fragment index, and vertex index) into a 256-dimensional vector; A two-layer multilayer perceptron is used to convert time t into a 256-dimensional vector. Similarly, a linear layer is used to compress the image features of the fragments into a 256-dimensional vector.

[0106] In some embodiments, step 110 specifically includes:

[0107] The end-to-end diffusion model is trained based on the dataset and loss function.

[0108] The loss function is a weighted sum of the standard noise regression loss, the matching loss, and the overlap loss.

[0109] The loss function in the reverse process is a weighted sum of the standard noise regression loss, the matching loss, and the overlap loss, as shown in the following equation:

[0110]

[0111] in, It is a hyperparameter used to balance the order of magnitude between the standard noise regression loss, the matching loss, and the overlap loss.

[0112] Standard noise regression loss is about noise The square function, .

[0113] To enhance the quality of supervision, a loss function specifically for the vertices of matched edges is proposed, namely the matching loss.

[0114] Assume the first The first approximate polygon The vertex and the first The first polygon If two vertices match, the matching loss between them is expressed as their Euclidean distance, which measures the stitching effect between the matched fragments. The smaller the value, the better the stitching effect. The specific expression is: .

[0115] To reduce the overlapping area after piecing together the fragments, the ratio of the overlapping area to the total area is used as the overlap loss to measure the degree of overlap between the fragments after piecing together. The smaller the value, the less overlap there is, which means a better piecing effect. The specific formula is as follows:

[0116]

[0117] Figure 3 This is a schematic diagram of the irregular fragment splicing device based on an end-to-end diffusion model provided in an embodiment of this application, as shown below. Figure 3 As shown, the device includes a cutting module 310, a training module 320, and a splicing module 330, wherein:

[0118] The cutting module 310 is used to randomly cut multiple images into several irregular fragments, and approximate the edges of the irregular fragments as polygons to obtain a dataset;

[0119] Training module 320 is used to train the end-to-end diffusion model based on the dataset to obtain the trained model;

[0120] The stitching module 330 is used to input fragments of the image to be stitched into the trained model to obtain the complete image after stitching.

[0121] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0122] Based on the methods in the above embodiments, Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown in the illustration, this application provides an electronic device that may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions stored in the memory 430 to execute the irregular fragment stitching method based on an end-to-end diffusion model described in the above embodiment.

[0123] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the irregular fragment splicing method based on the end-to-end diffusion model described in the various embodiments of this application.

[0124] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the irregular fragment splicing method based on an end-to-end diffusion model in the above embodiments.

[0125] Based on the methods in the above embodiments, this application provides a computer program product that, when running on a processor, causes the processor to execute the irregular fragment splicing method based on an end-to-end diffusion model in the above embodiments.

[0126] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0127] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0128] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0129] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0130] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for piecing together irregular fragments based on an end-to-end diffusion model, characterized in that, include: Multiple images are randomly cut into several irregular fragments, and the edges of the irregular fragments are approximated as polygons to obtain a dataset; The end-to-end diffusion model is trained based on the dataset to obtain the trained model. The fragments of the image to be stitched are input into the trained model to obtain the complete image after stitching. The step of training an end-to-end diffusion model based on the dataset to obtain the trained model includes: Based on the displacement distance and rotation angle of the fragments in the dataset, determine the state vector of the fragments; Noise is gradually added to the state vector to make the fragments become disordered. Piecing together disordered fragments into a complete image; The process of piecing together disordered fragments into a complete image includes: Using linear layers and multilayer perceptrons, feature encoding is initialized for fragments in an unordered state to obtain initialized feature encodings; The feature encoding is processed based on polygon self-attention mechanism and global self-attention mechanism; The processed feature encoding is converted into a 4-dimensional state vector and then denoised. The training of the end-to-end diffusion model based on the dataset includes: Based on the dataset and loss function, the end-to-end diffusion model is trained; The loss function is a weighted sum of standard noise regression loss, matching loss, and overlap loss.

2. The irregular fragment stitching method based on an end-to-end diffusion model according to claim 1, characterized in that, The dataset includes one or more of the following: Fragment datasets of cross-sectioned natural images, fragment datasets of face images after Vino cuts, and Magic Plan datasets of room layouts.

3. The irregular fragment stitching method based on an end-to-end diffusion model according to claim 1, characterized in that, The method uses linear layers and multilayer perceptrons to initialize feature encoding for fragments in an unordered state, obtaining initialized feature encodings, including: A linear layer is used to transform the 4-dimensional state vector of each vertex of the fragment into a 256-dimensional feature vector; Use a linear layer to convert the conditional vector into a 256-dimensional vector; A two-layer multilayer perceptron is used to convert time into a 256-dimensional vector. Image features of fragments are extracted using a linear layer, and the image features are converted into 256-dimensional vectors.

4. An irregular fragment splicing device based on an end-to-end diffusion model, characterized in that, include: The cutting module is used to randomly cut multiple images into several irregular fragments, and approximate the edges of the irregular fragments as polygons to obtain a dataset; The training module is used to train the end-to-end diffusion model based on the dataset to obtain the trained model. The stitching module is used to input fragments of the image to be stitched into the trained model to obtain the complete image after stitching. The training module is specifically used for: Based on the displacement distance and rotation angle of the fragments in the dataset, determine the state vector of the fragments; Noise is gradually added to the state vector to make the fragments become disordered. Piecing together disordered fragments into a complete image; The process of piecing together disordered fragments into a complete image includes: Using linear layers and multilayer perceptrons, feature encoding is initialized for fragments in an unordered state to obtain initialized feature encodings; The feature encoding is processed based on polygon self-attention mechanism and global self-attention mechanism; The processed feature encoding is converted into a 4-dimensional state vector and then denoised. The training module is specifically used for: Based on the dataset and loss function, the end-to-end diffusion model is trained; The loss function is a weighted sum of standard noise regression loss, matching loss, and overlap loss.

5. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the irregular fragment stitching method based on an end-to-end diffusion model as described in any one of claims 1-3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the irregular fragment stitching method based on an end-to-end diffusion model as described in any one of claims 1-3.

7. A computer program product, characterized in that, When the computer program product is run on a processor, the processor performs the irregular fragment stitching method based on an end-to-end diffusion model as described in any one of claims 1-3.

Images