A method and system for detecting heterogeneous image splicing

Abstract

This invention presents a method and system for detecting heterogeneous image stitching, comprising: constructing a dataset including a foreground dataset and a background dataset; randomly combining the foreground and background datasets to form a heterogeneous image stitching dataset; performing Gaussian blurring and random quality transformation on the heterogeneous image stitching dataset to randomly generate training, validation, and test sets; constructing a deep semantic segmentation model that combines a self-attention mechanism with a locally recursive UNET; training the deep semantic segmentation model using the training set, wherein the cost function is the sum of Tversky Loss and Focal Loss; fusing the final model results using a simple averaging method after training; and using the final model results for heterogeneous image stitching detection. This heterogeneous image stitching detection method has the advantages of significantly improving the recognition rate, high accuracy, and intuitive results.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for detecting heterogeneous image stitching. Background Technology

[0002] As the technical barriers to Photoshop skills have decreased, the misuse of image stitching techniques has become a significant negative factor in internet content security. From content pranks, personal insults, and portrait rights violations to transaction fraud, media manipulation, and various rumors, each instance of technological misuse profoundly impacts normal social production and life. Timely and accurate detection of fabricated information permeating the internet has become an urgent need for related industries, leading to the emergence of Photoshop digital detection technology. Early detection methods relied entirely on traditional image processing techniques, resulting in low accuracy, slow speed, and extremely poor generalization. Recently, with the disruptive impact of deep learning on computer vision and graphics, deep Photoshop detection has begun to demonstrate its power, showing remarkable effectiveness in terms of accuracy, generalization, and ease of use.

[0003] Chinese patent CN112508849A discloses a digital image stitching detection method and apparatus. It describes a traversal discrimination method based on image patches as detection units. Specifically, patches are sequentially fed into a convolutional neural network, and the possibility of stitching is determined. Finally, a voting mechanism is used to determine the authenticity of the entire image. Its shortcomings are quite obvious: the technology is outdated and violates the current concept of using a hierarchical integrated feature map of convolutional neural networks for feature discrimination. It is extremely inefficient, requiring inference as many times as there are patches, and is extremely time-consuming when dealing with large images.

[0004] Chinese patent CN110647948A discloses a method and system for image stitching detection based on neural networks. It performs detection in two stages: first, a first neural network model identifies the image of the subject to be detected and removes the background; then, a second neural network detects the patches. Essentially, it still relies on the idea of ​​traversing and judging patches. While the first neural network reduces the scope of judgment, it consumes inference time, and overall, there is no significant improvement. Summary of the Invention

[0005] To address the technical problems of low efficiency and the fact that existing image stitching detection methods still rely on patch block traversal and discrimination, this invention proposes a method and system for detecting heterogeneous image stitching to solve these problems.

[0006] According to a first aspect of the present invention, a method for detecting heterogeneous image splicing is proposed, comprising:

[0007] S1: Construct a dataset, which includes a foreground dataset and a background dataset. Randomly combine the foreground dataset and the background dataset to create a heterogeneous image stitching dataset.

[0008] S2: Perform Gaussian blurring and random quality transformation on the heterogeneous image stitching dataset to randomly generate training, validation and test sets;

[0009] S3: Construct a deep semantic segmentation model. The deep semantic segmentation model combines a self-attention mechanism with a locally recursive UNET. The deep semantic segmentation model is trained using the training set. The cost function is the sum of Tversky Loss and Focal Loss. After training, the final model results are fused using a simple averaging method.

[0010] S4: Use the final model results to perform heterogeneous image stitching detection.

[0011] In some specific embodiments, S1, which involves randomly combining the foreground dataset and the background dataset to stitch the images together, specifically includes: randomly pasting elements from the foreground dataset into any valid position of elements in the background dataset, with the pasting method being overlay.

[0012] In some specific embodiments, the ratio of the training set, validation set, and test set in S2 is 8:1:1, and the quality random transformation adopts a pseudo-random method with a normal distribution.

[0013] According to the heterogeneous image stitching detection method of claim 2, before S3, it further includes: constructing groundtruth labels based on the foreground overlay position, specifically in that: the foreground pixels of each image in the heterogeneous image stitching dataset are 255, and the background pixels are 0.

[0014] In some specific embodiments, the structure of the deep semantic segmentation model, from low to high, includes LayerNorm, divergence-regularized multi-head self-attention, MLP, and GELU; the formal definition of the deep semantic segmentation model is as follows: π =SMSA(layernorm(z) l-1 ))+z l-1 Among them, integrated information z l =mlp(layernorm(z) π ))+z π MLP is a fully connected layer used to transform dimensions, and the divergence regularized multi-head self-attention (SMSA) adds a subspace divergence regularization strategy to the standard MSA paradigm.

[0015] In some specific embodiments, the deep semantic segmentation model also includes a recurrent transformer block component, which is formally defined as follows: F r= loop(trf_block_A(trf_block_A(x)+x0,t), where t is the number of loops and trf_block_A is the transformer block. The image of the input x after one mapping is summed with itself and then fed into the mapping function for inference. After several loops, important signals are focused on and strengthened.

[0016] In some specific embodiments, the Tversky Loss formula in S3 is: Where A and B are the actual ground truth and the predicted pixel region, respectively, and λ and μ are set to 0.2 and 0.8, respectively; Focal Loss is formally defined as follows: Where α is set to 0.65, γ is set to 2, p is the prediction probability, and y is the label value.

[0017] In some specific embodiments, the AdamW optimizer combined with a cosine annealing strategy is used for 64 training epochs. A validation is performed after each training epoch, using the Dice coefficient as the validation formula. Training continues until the Dice coefficient increases to a stable state. The Dice coefficient is defined as follows: Where |X| represents the true result, |Y| represents the predicted result, and |X∩Y| represents the correctly predicted result.

[0018] In some specific embodiments, S3 further includes: repeating the training and validation process, iteratively increasing the proportion of difficult samples, completing the training and retaining the last 5 rounds of training model, and using a simple averaging method to fuse the final model results.

[0019] According to a second aspect of the invention, a computer-readable storage medium is provided on which one or more computer programs are stored, which, when executed by a computer processor, implement any of the methods described above.

[0020] According to a third aspect of the present invention, a heterogeneous image stitching detection system is proposed, the system comprising:

[0021] Dataset construction unit: Configured to construct a dataset, which includes a foreground dataset and a background dataset, and randomly combines heterogeneous image stitching datasets based on the foreground and background datasets;

[0022] Dataset processing unit: configured to perform Gaussian blurring and random quality transformation on heterogeneous image stitching datasets, randomly generating training, validation, and test sets;

[0023] Model building unit: Configured to construct a deep semantic segmentation model. The deep semantic segmentation model combines a self-attention mechanism with a locally recursive UNET. The deep semantic segmentation model is trained using the training set. The cost function is the sum of Tversky Loss and Focal Loss. After training, the final model results are fused using a simple averaging method.

[0024] Detection Unit: Configured for detecting heterogeneous image stitching using the final model results.

[0025] In some specific embodiments, the random combination of the foreground dataset and the background dataset to create a composite dataset specifically includes: randomly pasting elements from the foreground dataset into any valid position of elements in the background dataset, with the pasting method being overlay.

[0026] In some specific embodiments, the ratio of training set, validation set and test set is 8:1:1, and the quality random transformation adopts a pseudo-random method with normal distribution; the ground truth label is constructed based on the foreground overlay position, specifically: the foreground pixels of each image in the heterogeneous image stitching dataset are 255, and the background pixels are 0.

[0027] In some specific embodiments, the structure of the deep semantic segmentation model, from low to high, includes LayerNorm, divergence-regularized multi-head self-attention, MLP, and GELU; the formal definition of the deep semantic segmentation model is as follows: π =SMSA(layernorm(z) l-1 ))+z l-1 Among them, integrated information z l =mlp(layernorm(z) π 00+z π MLP is a fully connected layer used to transform dimensions, and the divergence regularized multi-head self-attention (SMSA) adds a subspace divergence regularization strategy to the standard MSA paradigm. The deep semantic segmentation model also includes a recurrent transformer block component, formally defined as follows: F r = loop(trf_block_A(trf_block_A(x)+x),t), where t is the number of loops and trf_block_A is the transformer block. The image of the input x after one mapping is summed with itself and then fed into the mapping function for inference. After several loops, important signals are focused on and strengthened.

[0028] In some specific embodiments, the Tversky Loss formula in the model building unit is: Where A and B are the actual ground truth and the predicted pixel region, respectively, and λ and μ are set to 0.2 and 0.8, respectively; Focal Loss is formally defined as follows: Where α is set to 0.65, γ to 2, p is the prediction probability, and y is the label value; the AdamW optimizer with cosine annealing is used, and the training is performed for 64 epochs. Validation is performed after each training epoch, using the Dice coefficient as the validation formula. Training continues until the Dice coefficient increases to a stable state. The Dice coefficient is defined as follows: Where |X| represents the true result, |Y| represents the predicted result, and |X∩Y| represents the correctly predicted result; the training and validation process is repeated, the proportion of difficult samples is increased iteratively, the training is completed and the last 5 rounds of training model are retained, and the final model result is fused using a simple averaging method.

[0029] This invention proposes a method and system for detecting heterogeneous image splicing. In terms of dataset construction, it randomly superimposes elements from the foreground dataset onto arbitrary valid positions of elements in the background dataset through a random combination of foreground and background datasets, significantly improving the recognition rate. Furthermore, it proposes an advanced transformer application concept, namely a multi-head attention mechanism, and introduces divergence constraints and iterative reasoning to locate the splicing region in one step from a semantic segmentation perspective, achieving high accuracy and intuitive results. This invention will have a positive impact on the development of related technologies and products in the fields of AI security, electronic data forensics, and artificial intelligence. Attached Figure Description

[0030] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Other features, objects, and advantages of this application will become more apparent from reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0031] Figure 1 This is a flowchart of an embodiment of an image splicing detection method according to this application;

[0032] Figure 2 This is a structural framework diagram of R2TransUnet, a deep semantic segmentation model of a specific embodiment of this application;

[0033] Figure 3 This is a framework diagram of an image stitching detection system according to an embodiment of this application;

[0034] Figure 4This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0035] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0036] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0037] A method for detecting heterogeneous image splicing according to an embodiment of this application. Figure 1 A flowchart of an image stitching detection method according to an embodiment of this application is shown. Figure 1 As shown, the method includes:

[0038] S101: Construct a dataset, which includes a foreground dataset and a background dataset, and randomly combine heterogeneous images to create a dataset based on the foreground dataset and the background dataset.

[0039] In a specific embodiment, a foreground dataset F and a background dataset B are constructed. The foreground dataset F contains n images of common objects, such as plants and animals, household items, artistic fonts, faces, and human bodies. The background dataset B contains m different background images, including common indoor and outdoor scenes, landscape photos, news photos, etc. Based on the constructed B and foreground dataset F, a heterogeneous image stitching dataset C is randomly combined. The method is to randomly paste elements from the foreground dataset F into any valid position of elements in the background dataset B. The pasting method is overlay. Due to the huge number of mapping relationships, the upper limit capacity of the heterogeneous image stitching dataset C is set to a constant of 200,000 images.

[0040] S102: Perform Gaussian blurring and quality randomization on the heterogeneous image stitching dataset to randomly generate training, validation and test sets.

[0041] In a specific embodiment, the data in the heterogeneous image stitching dataset C is subjected to Gaussian blurring and quality randomization transformation, and then training set D, validation set Dv, and test set Dt are randomly formed in an 8:1:1 ratio. The formal definition of Gaussian blurring is as follows: σ is the standard deviation of the normal distribution, u 2 +v 2 It equals the square of the fuzzy radius.

[0042] The random transformation of quality also adopts a pseudo-random method based on a normal distribution, with a quality range of 30-95, which will not be elaborated here.

[0043] In a specific embodiment, groundtruth labels are constructed based on the foreground overlay position obtained in the aforementioned steps. Specifically, in the heterogeneous image stitching dataset C, the foreground pixels of each image are 255, and the background pixels are 0.

[0044] S103: Construct a deep semantic segmentation model. The deep semantic segmentation model combines a self-attention mechanism with a locally recursive UNET. The deep semantic segmentation model is trained using a training set. The cost function is the sum of Tversky Loss and Focal Loss. After training, the final model results are fused using a simple averaging method.

[0045] In a specific embodiment, Figure 2 A structural framework diagram of the R2TransUnet deep semantic segmentation model of a specific embodiment of this application is shown, as follows: Figure 2 As shown, the model structure includes:

[0046] Based on adding regularization constraints to the multi-head structure, a divergent self-attention module is proposed. From low to high, it consists of LayerNorm, divergent regularized multi-head self-attention, MLP, and GELU, forming a residual structure within the module. Its formal definition is as follows: z π =SMSA(layernorm(z) l-1 ))+z l-1 The layer normal is used to address the issue of small batch size, while in the following formula, MLP is a fully connected layer used to transform dimensions and integrate information: z l =mlp(layernorm(z) π ))+z π The above process can be repeated within the module to enhance the depth of integration.

[0047] z ∴ =SMSA(layernorm(z) l ))+z l ;

[0048] z l+1 =mlp(layernorm(z) ∴ ))+z ∴ ;

[0049] Where z ∴ For the SMSA self-loop output result, z l+1 For the information integration result, l is the module index number.

[0050] The divergence-regularized multi-head self-attention (SMSA) model adds a subspace divergence regularization strategy to the standard MSA paradigm, resulting in the following optimization problem expression:

[0051] J(θ) = argmax θ {L(y|z;θ)+α*D(a|x,y;θ)}

[0052] Where L is the loss function and D is the regularization term. D can be further expressed as:

[0053]

[0054] Where y|x is the formal expression of the posterior probability, θ represents the neural network parameters of the module, and v i v j The parameter unit of a neural network, such as a convolution kernel.

[0055] In a specific implementation, a looping transformer block component is proposed, whose formal definition is as follows:

[0056] F r =loop(trf_block_A(trf_block_A(x)+x),t)

[0057] Where t is the number of iterations, trf_block_A is the transformer block, the image of input x after one mapping is summed with itself and then fed into the mapping function for inference, and after several iterations, important signals are focused on and strengthened.

[0058] In a specific embodiment, the R2TransUnet trained in the above steps is trained using the training set, where the cost function is designed as Tversky Loss + Focal Loss, and the Tversky Loss formula is as follows:

[0059]

[0060] Where A and B are the actual ground truth and the predicted pixel region, respectively, and λ and μ are set to 0.2 and 0.8, respectively; FocalLoss is formally defined as follows:

[0061]

[0062] Where α is set to 0.65, γ to 2, p is the prediction probability, and y is the label value. The initial learning rate is 0.0001, the weight decay is set to 0.0001, and the AdamW optimizer with cosine annealing is used for 64 training epochs. Validation is performed after each training epoch, using the Dice coefficient, defined as follows: Where |X| represents the true result, |U| represents the predicted result, and |X∩Y| represents the correctly predicted result.

[0063] Training continues until the Dice coefficient increases to a stable state, meaning it no longer changes significantly.

[0064] Repeat the above training and validation process, iteratively increasing the proportion of difficult samples, and finally complete the training. Keep the last 5 rounds of training models and use a simple averaging method to fuse the final model results.

[0065] S104: Use the final model results to perform heterogeneous image stitching detection.

[0066] This application presents a method for detecting heterogeneous images (PS) based on deep image semantic segmentation technology. It demonstrates significant innovation in dataset construction, data augmentation, and neural network structure design. The unique dataset construction method significantly improves the recognition rate. It proposes an advanced transformer application concept, namely a multi-head attention mechanism, and introduces divergence constraints and loop reasoning to locate the stitched region in one go from a semantic segmentation perspective, achieving high accuracy and intuitive results. This method can have a positive impact on the development of related technologies and products in the fields of AI security, electronic data forensics, and artificial intelligence.

[0067] Continue to refer to Figure 3 , Figure 3 A framework diagram of an image stitching detection system according to an embodiment of this application is shown. The system specifically includes a dataset construction unit 301, a dataset processing unit 302, a model unit 303, and a detection unit 304. The dataset construction unit 301 is configured to construct a dataset, which includes a foreground dataset and a background dataset, and randomly combines the foreground and background datasets to form an image stitching dataset. The dataset processing unit 302 is configured to perform Gaussian blurring and random quality transformation on the image stitching dataset, randomly forming a training set, a validation set, and a test set. The model construction unit 303 is configured to construct a deep semantic segmentation model, which combines a self-attention mechanism and a locally recursive UNET. The deep semantic segmentation model is trained using the training set, where the cost function is the sum of Tversky Loss and Focal Loss. After training, the final model results are fused using a simple averaging method. The detection unit 304 is configured to perform image stitching detection using the final model results.

[0068] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system 400 suitable for implementing electronic devices according to embodiments of the present application. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0069] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage section 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the system 400. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0070] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a liquid crystal display (LCD) and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card and a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0071] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0072] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0074] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0075] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: construct a dataset, the dataset including a foreground dataset and a background dataset, and randomly combine the foreground dataset and the background dataset into a heterogeneous image stitching dataset; perform Gaussian blurring and random quality transformation on the heterogeneous image stitching dataset to randomly form a training set, a validation set, and a test set; construct a deep semantic segmentation model, the deep semantic segmentation model combining a self-attention mechanism and a locally recursive UNET, train the deep semantic segmentation model using the training set, wherein the cost function is the sum of Tversky Loss and Focal Loss, and after training, fuse the final model results using a simple averaging method; and use the final model results to perform heterogeneous image stitching detection.

[0076] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method of detecting an anamorphic mosaic, characterized by, The method comprises the following steps: S1: constructing a data set, the data set comprising a foreground data set and a background data set, and combining a heterogeneous image stitching data set based on the foreground data set and the background data set randomly; S2: performing Gaussian blur and quality random transformation processing on the heterogeneous image stitching data set to randomly form a training set, a verification set and a test set; S3: constructing a deep semantic segmentation model, the deep semantic segmentation model combining a self-attention mechanism and a local recursive UNET, training the deep semantic segmentation model by using the training set, wherein a cost function is the sum of Tversky Loss and Focal Loss, and after training, the final model result is fused by using a simple average method; S4: using the final model result to perform heterogeneous image stitching detection. The structure of the deep semantic segmentation model from low to high respectively comprises LayerNorm, divergence regularization multi-head self-attention, MLP and GELU; the formal definition of the deep semantic segmentation model is as follows: Wherein, the integrated information , the mlp is a full connection layer used to transform dimensions, the divergence regularization multi-head self-attention SMSA is a sub-space divergence regularization strategy added on the basis of the standard MSA paradigm, is a module index number; The deep semantic segmentation model further comprises a recurrent transformer block component, which is defined as follows: where t is the number of cycles, is a transformer block, input The image after one mapping is summed with itself and then sent into the mapping function again for inference. After several cycles, important signals are paid attention to and strengthened.

2. The anamorphic stitch detection method of claim 1, wherein, In the S1, the foreground data set and the background data set are combined to form the heterogeneous image stitching data set, specifically, elements in the foreground data set are randomly pasted to any valid position of elements in the background data set, and the pasting method is superposition.

3. The anamorphic stitch detection method of claim 1, wherein, In the S2, the ratio of the training set, the verification set and the test set is 8:1:1, and the quality random transformation adopts a normal distribution pseudo-random method.

4. The anamorphic stitch detection method of claim 2, wherein, Before the S3, a groundtruth label is constructed based on the foreground superposition position, specifically, the pixels in the foreground of each picture in the heterogeneous image stitching data set are 255, and the rest of the background are 0.

5. The anamorphic stitch detection method of claim 4, wherein, The Tversky Loss formula in S3 is wherein A and B are actual ground truth and predicted pixel regions, λ and are respectively set to 0.2 and 0.8; the Focal Loss is defined as follows: wherein α is set to 0.65, γ is set to 2, is a prediction probability, is a label value.

6. The anamorphic stitch detection method of claim 5, wherein, The AdamW optimizer + cosine annealing strategy is adopted, and the model is trained for 64 rounds. After each round of training, the model is verified once, the verification formula adopts the Dice coefficient, and the training is stopped until the Dice coefficient increases to a stable state. The Dice coefficient is defined as follows: wherein, represents the true result, represents the predicted result, represents the result of correct prediction.

7. The anamorphic stitch detection method of claim 6, wherein, The S3 further comprises: repeating the training and verification process, iteratively increasing the proportion of difficult samples, completing the training and retaining the last 5 rounds of training models, and using the simple average method to fuse the final model result.

8. A computer readable storage medium having stored thereon one or more computer programs. The one or more computer programs are executed by a computer processor to implement the method of any one of claims 1-7.

9. A misregistration detection system characterized by, The system comprises: a data set construction unit configured to construct a data set, the data set comprising a foreground data set and a background data set, and combining a heterogeneous image stitching data set based on the foreground data set and the background data set randomly; a data set processing unit configured to perform Gaussian blur and quality random transformation processing on the heterogeneous image stitching data set to randomly form a training set, a verification set and a test set; a model construction unit configured to construct a deep semantic segmentation model, the deep semantic segmentation model combining a self-attention mechanism and a local recursive UNET, training the deep semantic segmentation model by using the training set, wherein a cost function is the sum of Tversky Loss and Focal Loss, and after training, the final model result is fused by using a simple average method; a detection unit configured to use the final model result to perform heterogeneous image stitching detection. The structure of the deep semantic segmentation model from low to high respectively comprises LayerNorm, divergence regularization multi-head self-attention, MLP and GELU; the formal definition of the deep semantic segmentation model is as follows: Wherein, the integrated information , mlp is a full connection layer for transforming dimensions, divergence regularization multi-head self-attention SMSA is a subspace divergence regularization strategy added on the basis of the standard MSA paradigm; the deep semantic segmentation model further comprises a recurrent transformer block component, and the formal definition thereof is as follows: Wherein, t is the number of cycles, is a transformer block, input The image after one mapping is summed with itself and then sent into the mapping function again for inference, and after a plurality of cycles, important signals are paid attention to and strengthened, is a module index number.

10. The anamorphic mosaic detection system of claim 9, wherein, The foreground data set and the background data set are combined to form the heterogeneous image stitching data set, specifically, elements in the foreground data set are randomly pasted to any valid position of elements in the background data set, and the pasting method is superposition.

11. The misgraph detection system of claim 9, wherein, The proportion of the training set, the validation set and the test set is 8:1:1, and the quality random transformation adopts a pseudo-random method of normal distribution; the groundtruth label is constructed based on the foreground superposition position, and the specific manner is that the pixels at the foreground of each picture in the heterogeneous image stitching data set are 255, and the background is 0.

12. The misgraph detection system of claim 9, wherein, The Tversky Loss formula in the model construction unit is Wherein, A, B are actual ground truth and predicted pixel area, λ and 0.2 and 0.8 respectively; the Focal Loss is defined as follows: Wherein, α is 0.65, γ is 2, is the prediction probability, is the label value; the AdamW optimizer + cosine annealing strategy is adopted, and 64 rounds of training are performed, and once every round of training, verification is performed, wherein the verification formula adopts the Dice coefficient, and the training is performed until the Dice coefficient increases to a stable state, and the Dice coefficient is defined as follows: Wherein, represents the true result, represents the predicted result, represents the correct prediction result; the training and verification process is repeated, the difficulty sample ratio is iteratively increased, the training model of the last 5 rounds is reserved, and the simple average method is used to fuse the final model result.

Images