End-to-end anomaly detection method and system based on positive and negative sample fusion, and medium

Abstract

The invention discloses an end-to-end anomaly detection method and system based on positive and negative sample fusion and a medium. Relates to the technical field of anomaly detection. According to the scheme, the method is improved on the basis of a traditional detection technology, the advantages of positive and negative samples are fully utilized, and the accuracy and recall rate of anomaly detection are improved; constructing and generating an abnormal data set, a real abnormal data set and a real abnormal image-text description pair, correspondingly constructing a training set A, a training set B and a training set C, and training the end-to-end anomaly detection network in stages by using the training set A, the training set B and the training set C; it is ensured that an end-to-end anomaly detection network can effectively learn anomaly features, and false alarms are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection, and in particular to an end-to-end anomaly detection method, system and medium based on positive and negative sample fusion. Background Art

[0002] With the development of deep learning technology, anomaly detection is being used more and more widely in various fields, especially in anomaly detection of substation inspection. At present, in the process of substation inspection, positive sample anomaly detection and negative sample anomaly detection usually adopt different technical routes in substation inspection anomaly detection. Positive sample anomaly detection usually adopts a comparative learning route, which has a high detection rate, but more false alarms, and no classification of detected anomalies. Negative sample anomaly detection adopts a target detection route, which has fewer false alarms, and detected anomalies have classification, but a low detection rate. This difference leads to the fact that after the two algorithms are run independently, it will be difficult to truly improve recall and reduce false alarms by fusing positive and negative samples.

[0003] At present, there are two common solutions for the fusion of positive and negative samples in the process of substation inspection. The first is to perform limited merging and filtering on the output results of the two methods after both positive and negative sample anomaly detection are executed. This method can effectively reduce false positives, but abnormal recall cannot be guaranteed. The second solution is to use the output of positive samples as the prior knowledge of negative samples and divide the fusion of positive and negative samples into two stages. However, since positive sample anomaly detection outputs more false positives, using the results of more false positives as the prior knowledge of negative sample detection will have a greater impact on the detection accuracy of negative samples.

[0004] As mentioned above, in the current substation inspection process, the positive and negative sample fusion method does not fully utilize the advantages of positive sample anomaly detection and negative sample anomaly detection. It only fuses the two anomaly detection algorithms at the surface level, but does not fuse the two anomaly detection algorithms at a deeper level. This makes it difficult to reduce the false positives of anomaly detection while maintaining a high recall of anomalies. Summary of the invention

[0005] The technical problem to be solved by the present invention is: the current problem of insufficient fusion of positive and negative samples for anomaly detection in the substation inspection process leads to low detection accuracy; the purpose of the present invention is to provide an end-to-end anomaly detection method, system and medium based on the fusion of positive and negative samples, improve the method on the basis of traditional detection technology, make full use of the advantages of positive and negative samples, and improve the accuracy and recall rate of anomaly detection.

[0006] The present invention is achieved through the following technical solutions: This solution provides an end-to-end anomaly detection method based on positive and negative sample fusion, including: Construct an end-to-end anomaly detection network based on positive and negative sample fusion; Constructing an abnormal data set, a real abnormal data set and a real abnormal image-text description pair, and correspondingly constructing a training set A, a training set B and a training set C; the training set A and the training set B both include a test image set and a base image set; The end-to-end anomaly detection network is trained in stages using training set A, training set B, and training set C respectively; Anomaly detection is performed based on the trained end-to-end anomaly detection network.

[0007] Working principle of this scheme: The current problem of insufficient fusion of positive and negative samples for anomaly detection in the substation inspection process leads to low detection accuracy; the purpose of the present invention is to provide an end-to-end anomaly detection method, system and medium based on the fusion of positive and negative samples, improve the method on the basis of traditional detection technology, make full use of the advantages of positive and negative samples, and improve the accuracy and recall rate of anomaly detection; by constructing a generated anomaly data set, a real anomaly data set and a real anomaly graphic description pair, corresponding training sets A, training sets B and training sets C are constructed, and the end-to-end anomaly detection network is trained in stages with training sets A, training sets B and training sets C; ensure that the end-to-end anomaly detection network can effectively learn the characteristics of the anomaly and reduce false alarms.

[0008] A further optimization scheme is to construct an end-to-end anomaly detection network based on the fusion of positive and negative samples; including the following methods: The Text Encoder branch in the contrastive language-image pre-training model (clip model) is used as the abnormal prompt text encoding network, and the improved yolov8 network is used as the positive and negative sample fusion network; The configuration includes a detection head that predicts the category feature vector, the predicted anomaly location box, and the anomaly confidence; The abnormality category is determined based on the comparison between the category feature vector and the text feature vector output by the abnormal prompt text encoding network, and the maximum cosine distance is used to determine the abnormality category.

[0009] A further optimization scheme is that the improved yolov8 network includes an A branch, a B branch and a feature fusion network; The A branch extracts features from a test image based on the traditional yolov8 network; the B branch extracts features from multiple base images based on 3D convolution and 2D convolution; The feature fusion network is used to fuse the features extracted by the A branch and the features extracted by the B branch.

[0010] A further optimization scheme is that the construction generates an abnormal data set, a real abnormal data set and a real abnormal graphic description pair, and correspondingly constructs training set A, training set B and training set C; including the method: Select N pictures of the same point at different times, generate abnormal images that need to be detected and abnormal images that do not need to be detected, and construct an abnormal data set from the abnormal images that need to be detected and the abnormal images that do not need to be detected as training set A; Select pictures with real anomalies as test pictures and pictures without anomalies as base pictures to generate a real anomaly dataset as training set B; The abnormal part is cut out from the picture with real abnormality, and the abnormality is described in text. Based on the cut-out abnormal part and the abnormal description, a real abnormal image and text description pair is generated as the training set C.

[0011] A further optimization scheme is to train the end-to-end anomaly detection network in stages using training set A, training set B and training set C respectively; including the method: Use training set A to train the positive and negative sample fusion network part of the end-to-end anomaly detection network, and perform box prediction and anomaly confidence prediction; The abnormal prompt text encoding network part of the end-to-end anomaly detection network is trained with the training set C; The entire end-to-end anomaly detection network is trained with training set B, and box prediction, anomaly confidence prediction, and category feature vector prediction are performed.

[0012] The further optimization scheme is that the prediction box loss is L CIou + DFL, where: ; ; in, IoU is the intersection-over-union ratio of the calibration frame and the detection frame, b and b gt are the center points of the detection frame and the calibration frame respectively, p 2 (*, #) is the square of the Euclidean distance between point * and point #, c is the diagonal distance between the detection frame and the closed area of ​​the calibration frame, ν To measure the relative proportion consistency between the detection frame and the calibration frame, a is the weight coefficient; S i 、S i+1 is the predicted value and the near-prediction value output by the network, y,y i 、y i+1 is the actual value of the label, the integral value of the label, and the integral value of the adjacent label; L CIou represents the first prediction box loss; DFL() represents the second prediction box loss; In the anomaly confidence prediction process, the anomaly confidence loss is: ; where y i is the sample label, p i is the prediction probability; N is the total number of detection boxes; i is the i-th detection box; L i is the abnormal confidence loss of the i-th detection box.

[0013] A further optimization scheme is that in the process of training the abnormal prompt text encoding network of the end-to-end anomaly detection network with training set B, the loss function is:

[0014] Where A and B are image feature vectors and text feature vectors; n is the feature vector length; θ is the angle between the two vectors; i is the number of the i-th feature; A i is the i-th image feature; B i is the i-th text feature.

[0015] A further optimization scheme is that the abnormal prompt text encoding network part of the end-to-end anomaly detection network is trained with the training set C, including the method: Based on the training set C, the contrastive language-image pre-training model corresponding to the abnormal prompt text encoding network is adjusted to align the image and text features of the abnormal pictures and abnormal description texts in the training set C.

[0016] This solution also provides an end-to-end anomaly detection system based on positive and negative sample fusion, which is used to implement the above-mentioned end-to-end anomaly detection method based on positive and negative sample fusion. The system includes: Network construction module, used to build an end-to-end anomaly detection network based on positive and negative sample fusion; A data set construction module is used to construct a generated anomaly data set, a real anomaly data set, and a real anomaly image-text description pair, and correspondingly construct a training set A, a training set B, and a training set C; the training set A and the training set B both include a test image set and a base image set; A training module, used for training the end-to-end anomaly detection network in stages using training set A, training set B and training set C respectively; The detection module is used to perform anomaly detection based on the trained end-to-end anomaly detection network.

[0017] The present solution also provides a computer-readable medium on which a computer program is stored, characterized in that the computer program is executed by a processor to implement the above-mentioned end-to-end anomaly detection method based on positive and negative sample fusion.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The present invention provides an end-to-end anomaly detection method, system and medium based on positive and negative sample fusion; by fusing positive and negative samples at the deep semantic feature level of the image, the advantages of the positive and negative samples are fully utilized, and the accuracy and recall rate of anomaly detection are improved;

[0019] 2. The present invention provides an end-to-end anomaly detection method, system and medium based on positive and negative sample fusion, which builds an end-to-end anomaly detection network based on the improved YoloV8 model, introduces 3D convolution to process multiple base images, and fuses them with the test image features extracted by 2D convolution, thereby enhancing the end-to-end anomaly detection network's ability to recognize abnormal features;

[0020] 3. The present invention provides an end-to-end anomaly detection method, system and medium based on positive and negative sample fusion, which ensures that the model can effectively learn the features of anomalies and reduce false positives by generating a data set containing anomalies that really need to be detected and anomalies that do not need to be detected, as well as a real anomaly data set and a pair of graphic descriptions;

[0021] 4. The present invention provides an end-to-end anomaly detection method, system and medium based on positive and negative sample fusion. Based on a phased training strategy, the model is first trained using a generated anomaly data set to reduce false positives, then fine-tuned using real anomaly image and text descriptions to achieve image and text feature alignment, and finally fine-tuned using real anomaly data sets to accurately locate and identify anomalies. It can maintain a high accuracy while also maintaining a high recall rate, effectively solving the problem of difficulty in balancing accuracy and recall rates in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 The flowchart of the end-to-end anomaly detection method based on positive and negative sample fusion is shown in Figure 2. Figure 2 Schematic diagram of the positive and negative sample fusion network structure; Figure 3 Schematic diagram of the network structure for encoding abnormal prompt text; Figure 4 Schematic diagram of the structure of the end-to-end anomaly detection system based on positive and negative sample fusion. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0024] The current problem of insufficient fusion of positive and negative samples in abnormal detection during substation inspection leads to low detection accuracy. In view of this, this solution provides the following embodiments to solve this technical problem: Embodiment 1: This embodiment provides an end-to-end anomaly detection method based on positive and negative sample fusion, such as Figure 1 As shown, including: Step 1: Build an end-to-end anomaly detection network based on positive and negative sample fusion; this step specifically includes the following methods: S11, the Text Encoder branch in the multimodal CLIP network is used as the abnormal prompt text encoding network (the abnormal prompt text encoding network structure is as follows Figure 3 As shown in Figure 2), the improved yolov8 network is used as the positive and negative sample fusion network. The positive and negative sample fusion network structure is as follows: Figure 2 As shown; The improved yolov8 network includes branch A, branch B and feature fusion network; The A branch extracts features from a test image based on the traditional yolov8 network; the B branch extracts features from multiple base images based on 3D convolution and 2D convolution; The feature fusion network is used to fuse the features extracted by the A branch and the features extracted by the B branch.

[0025] In this step, 2D convolution is used to extract features of the second feature layer of the test image, and 3D convolution is used to extract features of the first feature layer of the test image, where the first feature layer is smaller than the second feature layer.

[0026] S12, configuring a detection head including a predicted category feature vector, a predicted anomaly location box, and anomaly confidence; S13, based on the comparison between the category feature vector and the text feature vector output by the abnormal prompt text encoding network, the abnormal category is determined according to the maximum value of the cosine distance.

[0027] Step 2: construct an abnormal data set, a real abnormal data set and a real abnormal image-text description pair, and construct training set A, training set B and training set C accordingly; the training set A and training set B both include a test map set and a base map set; this step specifically includes the following method: S21, select N pictures of the same point at different times, generate abnormal images that need to be detected and abnormal images that do not need to be detected, and construct an abnormal data set from the abnormal images that need to be detected and the abnormal images that do not need to be detected as a training set A; Specifically, N pictures of the same point at different times are selected, one of which is used as a test picture and N-1 pictures are used as base pictures; anomalies are randomly generated on the test picture as the anomaly picture that needs to be detected, and the location of the anomaly is marked with an outer bounding rectangle; the same anomaly is randomly generated at the same position of the test picture and the base picture as the anomaly picture that does not need to be detected; an anomaly data set is constructed from the anomaly pictures that need to be detected and the anomaly pictures that do not need to be detected, as a training set A, and the training set A includes a test picture set and a base picture set; S22, select pictures with real anomalies as test pictures and pictures without anomalies as base pictures, and generate a real anomaly data set as training set B; Specifically, a picture with a real anomaly is selected as a test picture; then N-1 pictures without anomalies at the point are selected as base pictures, and the location of the anomaly is marked with an outer bounding rectangle; a real anomaly data set is generated from the test picture and the base picture as training set B, and training set B includes a test picture set and a base picture set.

[0028] S23, extract the abnormal part from the picture with real abnormality, describe the abnormality with text, and generate real abnormal image and text description pairs based on the extracted abnormal part and the abnormal description as training set C.

[0029] Step 3: training the end-to-end anomaly detection network in stages using training set A, training set B, and training set C respectively; this step specifically includes the following method: S31, training the positive and negative sample fusion network part of the end-to-end anomaly detection network with the training set A, and performing box prediction and anomaly confidence prediction; enabling the end anomaly detection network to have the ability to reduce false positives caused by random interference; specifically including the following steps: S311, training the positive and negative sample fusion network part of the end-to-end anomaly detection network with the training set A; S312, sending the test image in the training set A as a single detected image part to the 2D convolution part for feature extraction; S313, sending the multiple base images corresponding to the test image in the training set A as the multiple base image parts to the 3D convolution part for feature extraction; S314, after transforming the base image features extracted by the 3D convolution into 2D convolution, the base image features are fused with the features extracted by the 2D convolution of the test image, so that the deep features of the positive sample are integrated into the 2D features of the test image; S315, the final fused features are subjected to 2D convolution for box prediction and anomaly confidence prediction. This step of training does not perform prediction of specific category feature vectors.

[0030] The purpose of this round of training is to enable the model to have the following capabilities: when a certain area of ​​the test image and all the corresponding base images are different in the same corresponding area, the anomaly is judged to be a real anomaly; those that do not meet the above conditions are not anomalies; such capabilities can reduce false alarms caused by random interference.

[0031] S32, training the abnormal prompt text encoding network part of the end-to-end anomaly detection network with the training set C; this step specifically includes the method of: adjusting the contrastive language-image pre-training model corresponding to the abnormal prompt text encoding network based on the training set C, so that the abnormal pictures and abnormal description texts in the training set C can achieve image-text feature alignment.

[0032] S33, train the entire end-to-end anomaly detection network with training set B, and perform box prediction, anomaly confidence prediction, and category feature vector prediction.

[0033] S331, adding the entire end-to-end anomaly detection network as a pre-trained model to this model training, freezing the clip model and excluding it from parameter adjustment; S332, sending the test image in the training set B as a single detected image part to the 2D convolution part for feature extraction; S333, sending the multiple base images corresponding to the test image in the training data set B as the multiple base image parts to the 3D convolution part for feature extraction; S334, transforming the base image features extracted by 3D convolution into 2D convolution through 3D convolution, and fusing them with the features of the test image extracted by 2D convolution, so that the deep features of the positive sample are integrated into the 2D features of the test image; S335, the final fused features are subjected to 2D convolution for box prediction, abnormality confidence prediction, and category feature vector prediction; S336, the loss function calculates the loss of the prediction box, the anomaly confidence, and the cosine loss between the category feature vector and the text feature vector output by the Text Encoder branch (anomaly prompt text encoding network) in the above-mentioned adjusted clip model that describes the anomaly text.

[0034] The prediction box loss is L CIou + DFL, where:

[0035] ; in, IoU is the intersection-over-union ratio of the calibration frame and the detection frame, b and b gt are the center points of the detection frame and the calibration frame respectively, p 2(*, #) is the square of the Euclidean distance between point * and point #, c is the diagonal distance between the detection frame and the closed area of ​​the calibration frame, ν To measure the relative proportion consistency between the detection frame and the calibration frame, a is the weight coefficient; S i 、S i+1 is the predicted value and the near-prediction value output by the network, y,y i 、y i+1 is the actual value of the label, the integral value of the label, and the integral value of the adjacent label; L CIou represents the first prediction box loss; DFL() represents the second prediction box loss; In the anomaly confidence prediction process, the anomaly confidence loss is:

[0036] where y i is the sample label, p i is the prediction probability; N is the total number of detection boxes; i is the i-th detection box; L i is the abnormal confidence loss of the i-th detection box; In the process of training the entire end-to-end anomaly detection network with training set B, the loss function is:

[0037] Where A and B are image feature vectors and text feature vectors; n is the feature vector length; θ is the angle between the two vectors; i is the number of the i-th feature; A i is the i-th image feature; B i is the i-th text feature.

[0038] The purpose of this round of training is to use a smaller set of actual substation anomaly data (the amount of data is compared with the randomly generated anomaly data set) to fine-tune a network that can accurately locate anomalies, accurately identify anomalies, and accurately give anomaly categories based on the above steps. The network has very high anomaly recall and very high accuracy.

[0039] Step 4: Perform anomaly detection based on the trained end-to-end anomaly detection network. The specific process includes: Use the trained model to extract features from the test image, and input the test image into the 2D convolution part of the model; extract the features of the test image through 2D convolution. Use the trained model to extract features from the base image; input the base image into the 3D convolution part of the model; extract the features of the base image through 3D convolution. Fuse the base image features with the test image features; use the fused features for box prediction, anomaly confidence prediction, and category feature vector prediction; perform feature fusion on the fused features again through 2D convolution; send the fused features to the box prediction head, anomaly confidence prediction head, and category feature vector prediction head for prediction; compare the predicted category feature vector with the text feature vector output by the Text Encoder branch in the clip to determine the anomaly category. Compare the predicted category feature vector with the text feature vector output by the Text Encoder branch in the clip; calculate the cosine distance between the two; finally, select the anomaly category corresponding to the category feature vector with the largest cosine distance as the final anomaly category.

[0040] Embodiment 2: This embodiment provides an end-to-end anomaly detection system based on positive and negative sample fusion, such as Figure 4 As shown, for implementing an end-to-end anomaly detection method based on positive and negative sample fusion described in Example 1, the system includes: Network construction module, used to build an end-to-end anomaly detection network based on positive and negative sample fusion; A data set construction module is used to construct a generated anomaly data set, a real anomaly data set, and a real anomaly image-text description pair, and correspondingly construct a training set A, a training set B, and a training set C; the training set A and the training set B both include a test image set and a base image set; A training module, used for training the end-to-end anomaly detection network in stages using training set A, training set B and training set C respectively; The detection module is used to perform anomaly detection based on the trained end-to-end anomaly detection network.

[0041] Embodiment 3: This embodiment provides a computer-readable medium on which a computer program is stored. The computer program is executed by a processor to implement an end-to-end anomaly detection method based on positive and negative sample fusion as described in Embodiment 1; specifically, the following steps are performed: Step 1: Build an end-to-end anomaly detection network based on positive and negative sample fusion; Step 2: construct an abnormal data set, a real abnormal data set and a real abnormal image-text description pair, and construct training set A, training set B and training set C accordingly; the training set A and training set B both include a test map set and a base map set; Step 3, training the end-to-end anomaly detection network in stages using training set A, training set B, and training set C respectively; Step 4: Perform anomaly detection based on the trained end-to-end anomaly detection network.

[0042] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An end-to-end anomaly detection method based on positive and negative sample fusion, characterized in that: include: Construct an end-to-end anomaly detection network based on positive and negative sample fusion; Constructing an abnormal data set, a real abnormal data set and a real abnormal image-text description pair, and correspondingly constructing a training set A, a training set B and a training set C; the training set A and the training set B both include a test image set and a base image set; The end-to-end anomaly detection network is trained in stages using training set A, training set B, and training set C respectively; Anomaly detection is performed based on the trained end-to-end anomaly detection network.

2. According to claim 1, the end-to-end anomaly detection method based on positive and negative sample fusion is characterized in that: The construction is an end-to-end anomaly detection network based on the fusion of positive and negative samples; Included methods: The Text Encoder branch in the contrastive language-image pre-training model is used as the abnormal prompt text encoding network, and the improved yolov8 network is used as the positive and negative sample fusion network; The configuration includes a detection head that predicts the category feature vector, the predicted anomaly location box, and the anomaly confidence; The abnormality category is determined based on the comparison between the category feature vector and the text feature vector output by the abnormal prompt text encoding network, and the maximum cosine distance is used to determine the abnormality category.

3. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 2 is characterized in that: The improved yolov8 network includes an A branch, a B branch and a feature fusion network; The A branch extracts features from a test image based on the traditional yolov8 network; the B branch extracts features from multiple base images based on 3D convolution and 2D convolution; The feature fusion network is used to fuse the features extracted by the A branch and the features extracted by the B branch.

4. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 1, characterized in that: The construction generates an abnormal data set, a real abnormal data set and a real abnormal graphic description pair, and correspondingly constructs a training set A, a training set B and a training set C; including the method: Select N pictures of the same point at different times, generate abnormal images that need to be detected and abnormal images that do not need to be detected, and construct an abnormal data set from the abnormal images that need to be detected and the abnormal images that do not need to be detected as training set A; Select pictures with real anomalies as test pictures and pictures without anomalies as base pictures to generate a real anomaly dataset as training set B; The abnormal part is cut out from the picture with real abnormality, and the abnormality is described in text. Based on the cut-out abnormal part and the abnormal description, a real abnormal image and text description pair is generated as the training set C.

5. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 3 is characterized in that: The end-to-end anomaly detection network is trained in stages using training set A, training set B and training set C respectively; Included methods: Use training set A to train the positive and negative sample fusion network part of the end-to-end anomaly detection network, and perform box prediction and anomaly confidence prediction; The abnormal prompt text encoding network part of the end-to-end anomaly detection network is trained with the training set C; The entire end-to-end anomaly detection network is trained with training set B, and box prediction, anomaly confidence prediction, and category feature vector prediction are performed.

6. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 5, characterized in that: In the box prediction process, the prediction box loss is L CIou + DFL, where: ； ； in, IoU is the intersection-over-union ratio of the calibration frame and the detection frame, b and b gt are the center points of the detection frame and the calibration frame respectively, p 2 (*, #) is the square of the Euclidean distance between point * and point #, c is the diagonal distance between the detection frame and the closed area of ​​the calibration frame, ν To measure the relative proportion consistency between the detection frame and the calibration frame, a is the weight coefficient; S i 、S i+1 is the predicted value and the near-prediction value output by the network, y,y i 、y i+1 is the actual value of the label, the integral value of the label, and the integral value of the adjacent label; L CIou represents the first prediction box loss; DFL() represents the second prediction box loss; In the anomaly confidence prediction process, the anomaly confidence loss is: ； in y i is the sample label, p i is the prediction probability; N represents the total number of detection boxes; i represents the i-th detection box; L i is the abnormal confidence loss of the i-th detection box.

7. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 5, characterized in that: In the process of training the entire end-to-end anomaly detection network with training set B, the loss function is: ； Where A and B are image feature vectors and text feature vectors; n is the length of the feature vector; θ is the angle between two vectors; i is the number of the i-th feature; A i is the i-th image feature; B i is the i-th text feature.

8. The end-to-end anomaly detection method based on positive and negative sample fusion according to claim 5, characterized in that: The method of training the abnormal prompt text encoding network part of the end-to-end abnormal detection network with the training set C includes: Based on the training set C, the contrastive language-image pre-training model corresponding to the abnormal prompt text encoding network is adjusted to align the image and text features of the abnormal images and abnormal description texts in the training set C.

9. An end-to-end anomaly detection system based on positive and negative sample fusion, characterized in that: For implementing the end-to-end anomaly detection method based on positive and negative sample fusion according to any one of claims 1 to 8, the system comprises: Network construction module, used to build an end-to-end anomaly detection network based on positive and negative sample fusion; A data set construction module is used to construct a generated anomaly data set, a real anomaly data set, and a real anomaly image-text description pair, and correspondingly construct a training set A, a training set B, and a training set C; the training set A and the training set B both include a test image set and a base image set; A training module, used for training the end-to-end anomaly detection network in stages using training set A, training set B and training set C respectively; The detection module is used to perform anomaly detection based on the trained end-to-end anomaly detection network.

10. A computer readable medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement an end-to-end anomaly detection method based on positive and negative sample fusion as described in any one of claims 1 to 8.

Images