Multi-temporal monitoring image anti-interference anomaly detection model training method, device and detection method

Abstract

The present invention discloses a training method, device and detection method for an anti-interference anomaly detection model of multi-phase monitoring images. The present invention performs scene conversion based on the image to be tested, and generates an interference image with seasonal and weather changes with the reference image. A triple twin network is used to extract features from the reference image, the image to be tested and the interference image, and at the same time, spatial consistency constraints are constructed at the feature level to resist the interference of seasonal and weather changes. After feature fusion, a rectangular frame is used to locate the abnormal area. The present invention uses the reference image + the image to be tested + the interference image as the network input, and can effectively extract robust difference features in combination with the spatial consistency constraints, thereby improving the accuracy and anti-interference ability of the multi-phase monitoring image anomaly detection model. The present invention can be used for automated anomaly detection in fixed monitoring scenes, and the anomaly detection model has strong adaptability and anti-interference ability.

Description

Technical Field

[0001] The present invention relates to intelligent anti-interference anomaly detection in industrial scenarios, and in particular to a multi-phase monitoring image anti-interference anomaly detection model training method, device and detection method. Background Art

[0002] In the daily operation of industrial scenes such as power plants and oil depots, due to the complex working environment and the large number of personnel and equipment, it is often easy to have abnormal equipment status, such as the opening of the electrical box door, the opening of the valve, the on of the indicator light, etc., as well as abnormal foreign intrusion, such as the intrusion of personnel, vehicles, and animals. In order to maintain a safe and stable operating environment, these abnormalities need to be sensed and handled in a timely manner. At present, the main method is to manually view videos or conduct regular inspections, which is inefficient and has high labor costs.

[0003] In recent years, with the development of computer vision technology and monitoring technology, it has become possible to realize intelligent monitoring of industrial production. In actual industrial sites, there are many scenes and most of the scenes are in a static and normal state, and the cost of real-time monitoring is huge. Therefore, the method of multi-temporal monitoring images is often used for detection and maintenance. The monitoring image of a certain scene at a certain time is used as the reference image, and the monitoring image of another different time is used as the image to be tested. After comparing and analyzing the differences between the two, anomaly detection is achieved. The advantage of this is that it can deal with the problems of many scenes, many types of anomalies and few abnormal samples, and all anomalies are classified as differences, which is conducive to improving the accuracy and generalization of the anomaly detection model. However, at the same time, most industrial scenes are outdoor operations. Due to the certain time interval between the previous and next monitoring images, there are certain interferences such as weather changes, lighting changes, and seasonal changes between the reference image and the image to be tested, which will affect the effect of difference detection. How to ignore the interference of these pseudo-differences and improve the accuracy and robustness of anomaly detection in multi-temporal monitoring images is a direction worth studying.

[0004] During the training of deep learning models, the general method to improve the accuracy and robustness of the model is to collect more samples, or to perform some data enhancement methods such as cropping and noise to expand the samples. On the one hand, due to the uncontrollable weather and the large seasonal time span, it is difficult to collect a large number of samples with weather changes and seasonal changes in actual operation; on the other hand, it is difficult to simulate difficult samples with high-level semantics such as weather changes and seasonal changes using conventional data enhancement methods. At present, the style transfer network based on deep learning is very popular. By generating adversarial networks to learn the mapping relationship between different style domains on open source datasets, scene conversion between different style domains can be achieved, which can be used as a method for generating difficult samples such as weather changes and seasonal changes.

[0005] How to combine the style transfer network to generate effective difficult interference samples, and how to combine the difficult interference samples to build a network structure, extract robust and anti-interference difference features, and obtain anomaly detection models with high robustness and high precision for anti-interference anomaly detection in multi-temporal surveillance images is a huge challenge. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-phase monitoring image anti-interference anomaly detection model training method, device and detection method, which solves the problem of multi-phase monitoring image anti-interference anomaly detection.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A method for training a multi-temporal surveillance image anti-interference anomaly detection model, specifically:

[0009] Construct an image anomaly detection dataset based on multi-temporal monitoring images, wherein each sample in the image anomaly detection dataset is a triplet (In, Id, Ii), In is a reference image, Id is a test image, Ii is an interference image corresponding to the test image, and the interference image has different weather and / or season from the test image;

[0010] Construct an anti-interference anomaly detection model, which includes a triple twin network, a difference feature fusion module and a multi-scale prediction network, wherein:

[0011] The triple twin network is three feature extraction networks with the same structure and shared weights, which are used to extract features N of different scales of the reference image, the image to be tested, and the interference image. l ,D l ,I l , l=1,2,3…M;N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l represents the feature of the lth scale extracted by the interference graph, and M is the number of scale types;

[0012] The difference feature fusion module is used to calculate the detection difference feature H of each scale based on the features of different scales obtained. Dl =|N l -D l |、Interference difference feature H Il =|N l -I l | And calculate the fusion difference feature H of the detection difference feature and the interference difference feature DIl ;

[0013] The multi-scale prediction network is used to fusion difference features H based on each scaleDIl , predict the predicted value R of each scale l ∈(S l *S l *K l *4); the number of division units in the lth scale is S l , the number of anchors is K l ;

[0014] Each sample in the image anomaly detection dataset is used as the input of the anti-interference anomaly detection model, and the anti-interference anomaly detection model is trained with the goal of minimizing the loss function to obtain a trained anti-interference anomaly detection model; the loss function includes the predicted value R l and the localization loss of the true label.

[0015] The trained anti-interference anomaly detection model can be used for anti-interference anomaly detection in multi-temporal surveillance images.

[0016] Furthermore, the interference image is obtained by migrating the image to be tested based on a trained style transfer model.

[0017] Furthermore, the loss function also includes a loss function according to the detection difference feature H Dl and interference difference feature H Il Computed spatial consistency loss

[0018]

[0019] where λ l is the difference tolerance threshold of different scales, the average() function represents the average, and the max() function represents the maximum value, l=1,2,3…,M.

[0020] Furthermore, λ1>λ2>λ3…λ M-1 >λ M >0, corresponding to the difference tolerance threshold from large to small according to the scale from large to small.

[0021] A multi-temporal monitoring image anti-interference anomaly detection method, comprising:

[0022] The image to be tested and the reference image are input into the anti-interference anomaly detection model obtained by training in claims 1-4 to obtain the prediction values ​​R of different scales. l ;

[0023] The predicted value R l Perform post-processing, set the rectangular frame overlap threshold r_t, use NMS non-maximum suppression to filter out rectangular frames with high overlap, and output the final anti-interference anomaly detection result.

[0024] Furthermore, in the anti-interference anomaly detection model, the input of the feature extraction network branch corresponding to the interference graph is replaced by the graph to be tested, so that the interference difference feature H Il And detect the difference feature H Dl equal.

[0025] A multi-temporal monitoring image anti-interference anomaly detection model training device, comprising:

[0026] A data acquisition module is used to construct an image anomaly detection data set based on multi-phase monitoring images, wherein each sample in the image anomaly detection data set is a triplet (In, Id, Ii), In is a reference image, Id is a test image, Ii is an interference image corresponding to the test image, and the interference image has different weather and / or season from the test image;

[0027] A training module is used to take each sample in the image anomaly detection data set as the input of the anti-interference anomaly detection model, train the anti-interference anomaly detection model with the goal of minimizing the loss function, and obtain a trained anti-interference anomaly detection model;

[0028] Construct an anti-interference anomaly detection model, which includes a triple twin network, a difference feature fusion module and a multi-scale prediction network, wherein:

[0029] The triple twin network is three feature extraction networks with the same structure and shared weights, which are used to extract features N of different scales of the reference image, the image to be tested, and the interference image. l ,D l ,I l , l=1,2,3…M;N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l represents the feature of the lth scale extracted by the interference graph, and M is the number of scale types;

[0030] The difference feature fusion module is used to calculate the detection difference feature H of each scale based on the features of different scales obtained. Dl =|N l -D l |、Interference difference feature H Il =|N l -I l | And calculate the fusion difference feature H of the detection difference feature and the interference difference feature DIl ;

[0031] The multi-scale prediction network is used to fusion difference features H based on each scale DIl , predict the predicted value R of each scale l ∈(Sl *S l *K l *4); the number of division units in each scale is S l , the number of anchors is K l ;

[0032] The loss function contains the predicted value R l and the localization loss of the true label.

[0033] Furthermore, it also includes:

[0034] The style transfer module is used to transfer the image to be tested based on a trained style transfer model to obtain an interference map.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. The present invention adopts the reference image + the image to be tested + the interference image as the network input, and combined with the spatial consistency constraint, it can effectively extract the robust difference features, thereby improving the accuracy and anti-interference ability of the multi-phase monitoring image anomaly detection model; 2. The present invention can be used for automated anomaly detection in fixed monitoring scenes, and the anomaly detection model has strong adaptability and anti-interference ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall structure of this method;

[0037] Figure 2 This is a schematic diagram of the style transfer module structure of this method;

[0038] Figure 3 It is a schematic diagram of the structure of the anti-interference anomaly detection module of the present method;

[0039] Figure 4 It is a flowchart of the training process of the style transfer module of this method;

[0040] Figure 5 is a flow chart of the training process of the anti-interference anomaly detection module of the present method;

[0041] Figure 6 : is a schematic diagram of the experimental results of this method, where a is the reference image, b is the image to be tested, and c is the interference image generated by the style transfer module. A rectangular box is used in the figure to show the abnormal area predicted by the method. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0043] This embodiment takes several fixed monitoring video data of a factory as an example to construct a multi-temporal monitoring image dataset I to explain in detail the anti-interference anomaly detection model training and anti-interference anomaly detection method of the present invention. Each sample in I contains a reference image and a test image, which correspond to monitoring images taken at different times of the same scene, and there is a certain amount of weather and seasonal interference between the two. The multi-temporal monitoring image anti-interference anomaly detection task requires that the anomaly detection accuracy mAP remain above 90%.

[0044] like Figure 1 As shown, the multi-temporal monitoring image anti-interference anomaly detection model training of the present invention constructs an interference map corresponding to the test image, and adopts the reference image + the test image + the interference map as the network input, which can effectively extract the robustness difference features, thereby improving the accuracy and anti-interference ability of the image anomaly detection model, wherein the interference map is constructed based on the actual weather and seasonal change interference simulation in the monitoring scene, that is: the weather and / or season of the interference map are different from those of the test image; the interference map can be obtained by collection, simulation, migration, etc., and is preferably constructed by migration of the style migration model.

[0045] Figure 2 and Figure 4 The following is an example of a style transfer model structure and training process diagram that does not require paired data. The specific training method is as follows:

[0046] (1) Based on the open source dataset Summer2Winter, a style transfer dataset T is constructed. T contains two image sets Ts and Tw. Ts contains several Summer style images of arbitrary scenes, and Tw contains several Winter style images of arbitrary scenes. At the same time, the image size in T is normalized to 256×256×3.

[0047] (2) Build a style transfer model. This step is implemented through the following sub-steps:

[0048] (2.1) Build encoder E1 and encoder E2. Encoder E1 and encoder E2 have the same structure, consisting of multiple residual blocks, convolutional layers and activation function Leaky ReLU.

[0049] (2.2) Build decoder G1 and decoder G2. Decoder G1 and decoder G2 have the same structure, consisting of multiple residual blocks, deconvolution layers and activation function ReLU.

[0050] (2.3) Build discriminator D1 and discriminator D2. Discriminator D1 and discriminator D2 have the same structure, consisting of multiple residual blocks, convolutional layers, activation function Leaky ReLU, and fully connected layers.

[0051] (2.4) Connect the encoder E1, decoder G1 and discriminator D1 in sequence, and connect the encoder E2, decoder G2 and discriminator D2 in sequence to form a style transfer module.

[0052] (3) Training the style transfer model. This step is implemented through the following sub-steps:

[0053] (3.1) Use the style transfer dataset T to train the style transfer model and set hyperparameters such as the number of training rounds and learning rate.

[0054] (3.2) Randomly sample from the Ts and Tw image sets to obtain the binary samples (p, q) for this round of training.

[0055] (3.3) Input the two-tuple samples (p, q) into the style transfer model respectively, and the transfer graphs can be obtained in turn and the reconstructed graph p rec =G2(E2(G1(E1(p)))),q rec =G1(E1(G2(E2(q)))), calculate the weighted loss The specific sub-steps include the following:

[0056] (3.3.1) According to the reconstructed graph p rec ,q rec And the two-tuple sample (p, q) is used to calculate the cycle consistency loss

[0057]

[0058] ||*||1 means calculating the minus norm.

[0059] (3.3.2) According to the migration diagram And the discriminant loss is calculated for the two-tuple sample (p, q)

[0060]

[0061] Where D1(p), D2(q), Corresponding to p, q, The probability values ​​output after inputting into the discriminators D1 and D2 are both in the range of 0-1.

[0062] (3.3.3) According to the cycle consistency loss and the discriminative loss Calculate weighted loss

[0063]

[0064] where w c and w d is the weighted coefficient, w c ,w d ∈[0,1] and w d +w c =1.

[0065] (3.4) According to step (3.3), we get the weighted loss of the style transfer model. The SGD optimizer is used to update the network weights of the style transfer model.

[0066] (3.5) Determine whether the set number of iterations has been reached. If not, return to step (3.2) for the next iteration; if so, save the parameters in the style transfer model and the trained network weights.

[0067] like Figure 3 and Figure 5 As shown in the figure, the structure and training flow chart of the anti-interference anomaly detection model, the steps are as follows:

[0068] (1) Data preprocessing: This step is implemented through the following sub-steps:

[0069] (1.1) Based on fixed surveillance videos in actual industrial scenarios, we use images of the same scene at different times to construct a multi-temporal surveillance image anomaly detection dataset I. Each sample in I contains a reference image In and a test image Id.

[0070] (1.2) The reference image In and the image to be tested Id are pre-processed for image registration based on ORB features so that the two images are aligned, which can cope with the monitoring jitter interference caused by complex environments.

[0071] (1.3) Label the registered samples and use the VOC rectangular box format to mark the abnormal area between the reference image In and the image to be tested Id, with the category of defeat.

[0072] (1.4) The image to be tested Id is input into the trained style transfer module for testing to generate the interference image Ii, forming a multi-temporal surveillance image anti-interference anomaly detection dataset II, where each sample is a (In, Id, Ii) triple.

[0073] (1.5) Normalize the image size in Dataset II to 416×416×3.

[0074] (2) Build an anti-interference anomaly detection model ( Figure 3 ), which is implemented by the following sub-steps:

[0075] (2.1) Build a triple twin network. The triple twin network is composed of three feature extraction networks with the same structure and shared weights. The feature extraction network consists of multiple residual blocks, convolutional layers, BN layers and ReLU activation functions. The reference image, the image to be tested and the interference image are used as the input of the triple twin network. They are downsampled multiple times to obtain features N of different scales. l ,D l ,I l , l=1,2,3…,M,N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l It represents the feature of the lth scale extracted by the interference graph, M is the number of scale types, and M=3 in this embodiment.

[0076] (2.2) Perform the difference feature fusion module to calculate the detection difference feature H Dl =|N l -D l |, and the interference difference feature H Il =|N l -I l |, according to the detection difference feature H Dl and interference difference feature H Il , constructing spatial consistency constraint loss at the feature level It is used to extract more robust difference features. Further, the fusion difference features of the detection difference features and the interference difference features are calculated. In this embodiment, H DIl =H Il *H Dl , l=1,2,3…,M.

[0077] (2.3) Build a multi-scale prediction network. The multi-scale prediction network consists of cascaded multi-layer convolutional layers, BN layers, and ReLU activation functions at M scales. The fusion difference feature H DI1 , H DI2 , H DI3 …H DIM Corresponding to the input multi-scale prediction network, the predicted value R of the rectangular box is output at M scales l ∈(S l *S l *K l *4). The number of division units in each scale is S l , the number of anchors is K l , l=1,2,3…,M.

[0078] (3) Training the anti-interference anomaly detection model. This step is implemented through the following sub-steps:

[0079] (3.1) Use the training set data of Multi-temporal Monitoring Image Anti-interference Anomaly Detection Dataset II to train the anti-interference anomaly detection module and set hyperparameters such as training rounds, learning rate, and batch size;

[0080] (3.2) Randomly sample from the training set of dataset II to obtain sample m for this round of training.

[0081] (3.3) The same random mirroring, random cutting, random noise adding and other methods are used to perform data enhancement processing on the reference image In, the test image Id and the interference image Ii.

[0082] (3.4) Input m into the anti-interference anomaly detection module to obtain the detection difference feature H Dl , interference difference feature H Il , and the predicted value R l , l=1,2,3…,M.

[0083] (3.4.1) According to the detection difference feature H Dl and interference difference feature H Il , calculate the spatial consistency loss

[0084]

[0085] where λ l is the difference tolerance threshold of different scales, the average() function represents the average value, and the max() function represents the maximum value, l=1,2,3…,M. At the same time, λ1>λ2>λ3…λ M-1 >λ M >0, corresponding to the difference tolerance threshold from large to small according to the scale from large to small.

[0086] (3.4.2) According to the predicted value R l Calculate the positioning loss with the true label

[0087]

[0088] Among them, for the jth anchor of the ith unit of the lth scale, the anti-interference anomaly detection model predicts the central horizontal and vertical coordinates, width and length of the rectangular box, that is, the output prediction value R l Contains x ijl ,y ijl ,w ijl ,h ijl , the corresponding true label is

[0089] (3.4.3) According to the spatial consistency loss and positioning loss Calculate weighted loss

[0090]

[0091] where w s and w o is the weighted coefficient, w s ,w o ∈[0,1] and w s +w o =1.

[0092] (3.5) According to step (3.4), the weighted loss of the anti-interference anomaly detection model is obtained The Adam optimizer is used to update the network weights of the anti-interference anomaly detection model.

[0093] (3.5) Determine whether the set number of iterations has been reached. If not, return to step (3.2) for the next iteration. If yes, save the parameters in the anti-interference anomaly detection model and the trained network weights to obtain the trained anti-interference anomaly detection model.

[0094] Furthermore, the performance of the trained anti-interference anomaly detection model was tested as follows:

[0095] (4.1) Load the anti-interference anomaly detection model and the trained network weights, use the (In, Id, Ii) triples in the test set data of the multi-temporal surveillance image anti-interference anomaly detection dataset II as input, and generate the prediction result R i , i=1,2,3,M.

[0096] (4.2) For the prediction result R i After post-processing, the rectangular frame overlap threshold r_t=0.4 is set, and the rectangular frames with high overlap are filtered out using NMS non-maximum suppression to output the final anti-interference anomaly detection result.

[0097] The multi-temporal surveillance image anti-interference anomaly detection dataset has a total of 964 samples, including 438 abnormal samples and 526 non-abnormal samples. Each sample contains a reference image and a test image, both of which are taken at different times of the same scene, and there are certain interference factors such as weather, light and season. The dataset is randomly divided into training set, validation set and test set in a ratio of 7:2:1.

[0098] Table 1 lists the detection results of the multi-temporal surveillance image anti-interference anomaly detection method based on deep learning and the related ablation model on the test set. N represents the reference image, I represents the interference image, and D represents the image to be tested. Four training methods were used to compare the experimental results during training: 1. Using the same feature extraction network in the double twin network and triple twin network, directly splicing two pictures from the channel dimension as input; 2. Using the double twin network, the input is a binary of two pictures; 3. Using the triple twin network, the input is a triple of three pictures; 4. Using the four methods of triple twin network + spatial consistency constraint, the input is a triple of three pictures. During the test evaluation, the evaluation indicator used was mAP.

[0099] Table 1 Comparative performance (mAP) of anti-interference anomaly detection methods based on spatial consistency constraints

[0100]

[0101] Figure 6 It is a schematic diagram of the experimental results of this method. According to Table 1, when the feature extraction network or the double twin network method is used and only the original samples are used for training, the model's detection ability for difficult samples with interference is significantly reduced; after adding the interference samples generated by the style transfer network during training, the model has a certain improvement in the test accuracy of the interference samples and the overall test accuracy, which shows the effectiveness of the style transfer module. The first two methods only directly add difficult interference samples to the training samples to expand the data set; the latter two methods use difficult interference samples for joint training. When the triple twin network + spatial consistency constraint method is used, constraints at the feature level can extract more robust difference features, which can further improve the anomaly detection accuracy of multi-phase monitoring images, which shows the effectiveness of the anti-interference anomaly detection module.

[0102] Corresponding to the aforementioned multi-temporal monitoring image anti-interference anomaly detection model training method, the present invention also provides a multi-temporal monitoring image anti-interference anomaly detection model training device, comprising:

[0103] A data acquisition module is used to acquire an image anomaly detection data set based on multi-phase monitoring images, wherein each sample in the image anomaly detection data set is a triplet (In, Id, Ii), In is a reference image, Id is a test image, Ii is an interference image corresponding to the test image, and the interference image has different weather and / or season from the test image;

[0104] A training module is used to take each sample in the image anomaly detection data set as the input of the anti-interference anomaly detection model, train the anti-interference anomaly detection model with the goal of minimizing the loss function, and obtain a trained anti-interference anomaly detection model;

[0105] Construct an anti-interference anomaly detection model, which includes a triple twin network, a difference feature fusion module and a multi-scale prediction network, wherein:

[0106] The triple twin network is three feature extraction networks with the same structure and shared weights, which are used to extract features N of different scales of the reference image, the image to be tested, and the interference image. l ,D l ,I l , l=1,2,3…M;N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l represents the feature of the lth scale extracted by the interference graph, and M is the number of scale types;

[0107] The difference feature fusion module is used to calculate the detection difference feature H of each scale based on the features of different scales obtained. Dl =|N l -D l |、Interference difference feature H Il =|N l -I l | And calculate the fusion difference feature H of the detection difference feature and the interference difference feature DIl ;

[0108] The multi-scale prediction network is used to fusion difference features H based on each scale DIl , predict the predicted value R of each scale l ∈(S l *S l *K l *4); the number of division units in each scale is S l , the number of anchors is k l ;

[0109] The loss function contains the predicted value R l and the localization loss of the true label.

[0110] Furthermore, the device also includes:

[0111] The style transfer module is used to transfer the image to be tested based on a trained style transfer model to obtain an interference map.

[0112] The embodiment of the anti-interference anomaly detection model training device of the present invention can be applied to any device with data processing capability, and the any device with data processing capability can be a device or apparatus such as a computer.

[0113] The device embodiments may be implemented through software, hardware, or a combination of software and hardware.

[0114] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0115] In addition, the present invention also provides a multi-phase monitoring image anti-interference anomaly detection method, comprising:

[0116] The acquired multi-temporal monitoring images are used as the test images and the reference images to be input into the anti-interference anomaly detection model obtained by the above training to obtain the prediction values ​​R at different scales. l ;

[0117] The predicted value R l Perform post-processing, set the rectangular frame overlap threshold r_t, use NMS non-maximum suppression to filter out rectangular frames with high overlap, and output the final anti-interference anomaly detection result;

[0118] Among them, in the anti-interference anomaly detection model obtained by the above training, the input of the feature extraction network branch corresponding to the interference graph is replaced by the graph to be tested, so that the interference difference feature H Il and detect difference feature D Hl equal.

[0119] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A multi-temporal monitoring image anti-interference anomaly detection model training method, characterized in that: Specifically: Construct an image anomaly detection dataset based on multi-temporal monitoring images, wherein each sample in the image anomaly detection dataset is a triplet (In, Id, Ii), In is a reference image, Id is a test image, Ii is an interference image corresponding to the test image, and the interference image has different weather and / or season from the test image; Construct an anti-interference anomaly detection model, which includes a triple twin network, a difference feature fusion module and a multi-scale prediction network, wherein: The triple twin network is three feature extraction networks with the same structure and shared weights, which are used to extract features N of different scales of the reference image, the image to be tested, and the interference image. l ,D l ,I l , l=1,2,3…M;N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l represents the feature of the lth scale extracted by the interference graph, and M is the number of scale types; The difference feature fusion module is used to calculate the detection difference feature H of each scale based on the features of different scales obtained. Dl =|N l -D l |、Interference difference feature H Il =|N l -I l | And calculate the fusion difference feature H of the detection difference feature and the interference difference feature DIl ; The multi-scale prediction network is used to fusion difference features H based on each scale Dll , predict the predicted value R of each scale l ∈(S l *S l *K l *4); the number of division units in the lth scale is S l , the number of anchors is K l ; Each sample in the image anomaly detection dataset is used as the input of the anti-interference anomaly detection model, and the anti-interference anomaly detection model is trained with the goal of minimizing the loss function to obtain a trained anti-interference anomaly detection model; the loss function includes the predicted value R l and the localization loss of the true label.

2. The method according to claim 1, characterized in that The interference image is obtained by migrating the image to be tested based on a trained style transfer model.

3. The method according to claim 1, characterized in that The loss function also includes detecting the difference feature H Dl and interference difference feature H Il Computed spatial consistency loss where λ l is the difference tolerance threshold of different scales, the average() function represents the average, and the max() function represents the maximum value, l=1,2,3…,M.

4. The method according to claim 3, characterized in that: λ1>λ2>λ3…λ M-1 >λ M >0, corresponding to the difference tolerance threshold from large to small according to the scale from large to small.

5. A multi-temporal monitoring image anti-interference anomaly detection method, characterized in that: include: The acquired multi-phase monitoring images are used as the test images and the reference images to be input into the anti-interference anomaly detection model obtained by training in claims 1-4 to obtain the prediction values ​​R at different scales. l ; The predicted value R l Perform post-processing, set the rectangular frame overlap threshold r_t, use NMS non-maximum suppression to filter out rectangular frames with high overlap, and output the final anti-interference anomaly detection result.

6. The method according to claim 5, characterized in that In the anti-interference anomaly detection model obtained by training according to claims 1-4, the input of the feature extraction network branch corresponding to the interference graph is replaced by the graph to be tested, so that the interference difference feature H Il And detect the difference feature H Dl equal.

7. A multi-temporal monitoring image anti-interference anomaly detection model training device, characterized in that: include: A data acquisition module is used for monitoring anomaly detection data sets based on multi-temporal images, wherein each sample in the image anomaly detection data set is a triplet (In, Id, Ii), In is a reference image, Id is a test image, Ii is an interference image corresponding to the test image, and the interference image has different weather and / or season from the test image; A training module is used to take each sample in the image anomaly detection data set as the input of the anti-interference anomaly detection model, train the anti-interference anomaly detection model with the goal of minimizing the loss function, and obtain a trained anti-interference anomaly detection model; Construct an anti-interference anomaly detection model, which includes a triple twin network, a difference feature fusion module and a multi-scale prediction network, wherein: The triple twin network is three feature extraction networks with the same structure and shared weights, which are used to extract features N of different scales of the reference image, the image to be tested, and the interference image. l ,D l ,I l , l=1,2,3…M;N l represents the l-th scale feature extracted from the benchmark image, D l Indicates the l-th scale feature extracted from the image to be tested, I l represents the feature of the lth scale extracted by the interference graph, and M is the number of scale types; The difference feature fusion module is used to calculate the detection difference feature H of each scale based on the features of different scales obtained. Dl =|N l -D l |、Interference difference feature H Il =|N l -I l | And calculate the fusion difference feature H of the detection difference feature and the interference difference feature DIl ; The multi-scale prediction network is used to fusion difference features H based on each scale DIl , predict the predicted value R of each scale l ∈(S l *S l *K l *4); the number of division units in each scale is S l , the number of anchors is K l ; The loss function contains the predicted value R l and the localization loss of the true label.

8. The device according to claim 7, characterized in that Also includes: The style transfer module is used to transfer the image to be tested based on a trained style transfer model to obtain an interference map.

Images