Road anomaly detection method for small sample condition

Abstract

The invention relates to a road anomaly detection method for a small sample condition, and the method comprises the following steps: S1, obtaining radar data of road detection, and obtaining a picture data set; s2, constructing a road anomaly detection network, and training the road anomaly detection network to obtain a road anomaly detection model; and S3, acquiring actual road image data, inputting the road image data into the road anomaly detection model to obtain actual anomaly regions, inputting the actual anomaly regions into the anomaly detection segmentation model, and merging the actual anomaly regions to form a complete anomaly region. Compared with the prior art, the method has the advantages of improving the accuracy of road anomaly detection under the condition of small samples and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer vision in industrial scenarios, and in particular to a road anomaly detection method for small sample situations. Background Art

[0002] With the development of the transportation system, the timely discovery and maintenance of road anomalies has become particularly important. Small sample image anomaly detection technology is a key means to ensure quality and stability in the industrial field. With the rapid development of intelligent manufacturing, the requirements for anomaly detection are getting higher and higher. The goal of anomaly detection is to identify potential defects or anomalies from normal images to avoid economic losses and reduced production efficiency caused by quality problems.

[0003] At present, the methods for detecting anomalies in small sample images are mainly divided into traditional statistical methods, machine learning methods, and unsupervised learning methods. Statistical methods detect anomalies by analyzing the statistical features of images. For example, grayscale histograms, edge detection and other technologies are used to determine whether there are abnormal areas in the image. Although these methods perform well in simple scenes, they are prone to insufficient detection accuracy in complex image backgrounds due to their strict assumptions on data distribution. Machine learning methods train a large number of images so that the model can learn effective feature representations. Unsupervised learning methods learn normal samples to identify abnormal samples that do not match them. Existing small sample image anomaly detection technologies have certain advantages in theory, but they are prone to some problems in practical applications. Existing technologies are often limited to local information when extracting image features, ignoring the relationship between different areas in the image. For complex images in industrial applications, local features are difficult to fully reflect the actual situation of the anomaly. Detection models usually require a large amount of labeled data for training, but in the case of small samples, the cost of obtaining labeled data is high. In the case of insufficient labeled samples, the learning effect of the model is significantly reduced, affecting the accuracy of detection. Summary of the invention

[0004] The purpose of the present invention is to provide a road anomaly detection method for small sample conditions in order to improve the accuracy of road anomaly detection in small sample conditions.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A road anomaly detection method for a small sample situation, the method comprising the following steps:

[0007] S1. Obtain radar data for road detection, perform image processing on the data, convert it into corresponding picture data, and perform data enhancement on all picture data to obtain a picture data set, wherein the training set only contains pictures of normal samples;

[0008] S2. Construct a road anomaly detection network, input the training set into the road anomaly detection network, decompose the training set into local patch blocks, identify the tail samples, output enhanced feature representation based on the local patch blocks of the tail samples, learn the embedded representation of each local patch block, aggregate node information through the graph convolution layer, obtain aggregated features, calculate feature similarity based on the aggregated features, and calculate reconstruction errors, adjust the weights of the tail samples in the training set based on the reconstruction errors, and calculate anomaly scores based on the reconstruction errors and feature similarities, output anomaly areas, train the road anomaly detection network based on the contrast loss function, and obtain a road anomaly detection model;

[0009] S3. Acquire actual road image data, input the road image data into a road anomaly detection model to obtain an actual abnormal region, input the actual abnormal region into an anomaly detection segmentation model, merge the actual abnormal region, and form a complete abnormal region.

[0010] Furthermore, the specific steps of S2 are:

[0011] Construct a road anomaly detection network, input the training set into the road anomaly detection network, and decompose the training set images into multiple local patch blocks;

[0012] The pre-trained model is used as a feature extractor to extract the corresponding feature vector for each local patch block, filter the feature vector, identify the tail sample, and input the filtered feature vector into the self-attention mechanism, which outputs the enhanced feature representation.

[0013] Strengthen the feature representation. The graph neural network input into the road anomaly detection network learns the embedding representation of the local patch block, aggregates the information, and obtains the aggregated features.

[0014] The feature similarity is calculated based on the aggregated features. The aggregated features are passed through the autoencoder to obtain the reconstructed features. The reconstruction error is calculated based on the reconstructed features. The weight of the tail samples in the training set is adjusted based on the reconstruction error. At the same time, the anomaly score is calculated based on the reconstruction error and feature similarity, and the abnormal area is output;

[0015] The contrast loss function is calculated based on the weight and feature similarity, and the road anomaly detection network is optimized based on the contrast loss function to obtain the road anomaly detection model.

[0016] Furthermore, the reconstruction error is:

[0017]

[0018] Among them, ReconstructionError i is the reconstruction error of the i-th local patch, f iis the original input feature of the i-th local patch block, It is the feature after reconstruction of the i-th local patch.

[0019] Furthermore, the feature similarity is:

[0020]

[0021] Among them, f i and f j It is the aggregate feature of two local patches.

[0022] Furthermore, the anomaly score is:

[0023]

[0024] Among them, S i is the abnormality score of the i-th local patch, α, β are weight coefficients, and ReconstructionErrori i is the reconstruction error of the ith local patch, N(i) represents the domain of the ith local patch, Similarity(f i ,f j ) is the similarity between the i-th local patch block and the j-th local patch block, f i is the aggregate feature of the i-th local patch.

[0025] Furthermore, the weight of the tail sample in the adjusted training set is:

[0026]

[0027] Among them, ReconstructionError i is the reconstruction error of the ith patch, μ is the mean of the reconstruction errors of all samples, and λ is a tuning hyperparameter.

[0028] Furthermore, the contrast loss function is:

[0029]

[0030] Where D = Similarity(f i ,f j ), m is a hyperparameter used to control the minimum distance between negative sample pairs, ω i is the adjusted weight of sample i, y i is the label of the sample pair.

[0031] Furthermore, the identification of tail samples is specifically as follows:

[0032] Through the density-based clustering algorithm, samples in the sparse area are selected as tail samples.

[0033] Furthermore, the specific steps of calculating the anomaly score based on the reconstruction error and feature similarity and outputting the abnormal area are as follows:

[0034] The anomaly score is calculated based on the reconstruction error and feature similarity, and the local patch block with anomaly score exceeding the threshold is regarded as the abnormal area.

[0035] Furthermore, the pre-trained model is a WideResnet 101 model.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention can fully utilize the relationship between adjacent patches and retain sufficient spatial context by performing feature aggregation on a local neighborhood. This local aggregation method improves the sensitivity of anomaly detection, especially when dealing with problems in industrial scenarios, and can accurately identify minor anomalies. In addition, the present invention introduces a dynamic weighting mechanism to optimize tail samples and reduce the overlap between normal samples and abnormal samples. This improvement can significantly improve the reliability of detection, reduce false positives and false negatives, and thus enhance the usability of the model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A road anomaly detection model training graph of the present invention;

[0039] Figure 2 This is the test abnormal segmentation result map;

[0040] Figure 3 This is the specific flow chart of S2. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0042] The present invention provides 1. a road anomaly detection method for a small sample situation, characterized in that the method comprises the following steps:

[0043] S1. Obtain radar data for road detection, perform image processing on the data, convert it into corresponding picture data, and perform data enhancement on all picture data to obtain a picture data set, wherein the training set only contains pictures of normal samples;

[0044] S2. Construct a road anomaly detection network, input the training set into the road anomaly detection network, decompose the training set into local patch blocks, identify the tail samples, output enhanced feature representation based on the local patch blocks, learn the embedded representation of each local patch block, aggregate node information through the graph convolution layer, obtain aggregated features, calculate feature similarity based on the aggregated features, and calculate reconstruction errors, adjust the weights of the tail samples in the training set based on the reconstruction error, and calculate the anomaly score based on the reconstruction error and feature similarity, output the abnormal area, train the road anomaly detection network based on the contrast loss function, and obtain a road anomaly detection model;

[0045] S3. Acquire actual road image data, input the road image data into a road anomaly detection model to obtain an actual abnormal region, input the actual abnormal region into an anomaly detection segmentation model, merge the actual abnormal region, and form a complete abnormal region.

[0046] The test abnormal segmentation results are shown in the figure Figure 2 As shown in Figure 2, the training process of the model is as follows: Figure 1 The specific process of S2 is as follows Figure 3 shown.

[0047] The specific steps of the present invention include:

[0048] (1) Obtain radar data for road detection, perform image processing on the data, convert it into corresponding image data, and perform data enhancement on all image data to obtain the final image dataset. The training set here only contains images of normal samples.

[0049] (2) Construct a road anomaly detection model and a graph structure. Decompose the enhanced sample image into 64*64 pixel local patches. Each patch is a node in the graph. According to the similarity of color and texture features between patches, connections are established to form a graph structure. GNN is used to learn the embedded representation of the patch. The node information is aggregated through the graph convolution layer to improve the feature representation ability of the patch. The anomaly score is generated by calculating the similarity between features. At the same time, during the detection process, the reconstruction error of the patch is calculated through the autoencoder to further assist segmentation and mark the area with large error as an abnormal area. During the training process, the weight is dynamically adjusted according to the difference between the feature of each pixel and the category mean. The tail sample can obtain a higher weight, which can guide the model to learn the corresponding features. By increasing the weight of the tail sample, the overlap between normal samples and abnormal samples can be reduced, improving the detection ability of the model. By focusing on the ratio of the individual difference of each pixel to the average value, its importance in training is measured.

[0050] (3) Based on the detection, an anomaly detection and segmentation model is constructed to post-process the patch blocks marked as abnormal, merge adjacent abnormal patch blocks to form a complete abnormal area, and finally visualize it in the image for subsequent analysis and processing.

[0051] (4) Using the image data set to train and test the detection model, selecting the optimal weight parameters of the detection model, and obtaining the optimal detection model for detecting abnormal conditions in road data.

[0052] For the step (1) of enhancing all the image data, the image data set obtained includes:

[0053] First, all converted images are resized to a uniform size for use as training sets. The images containing abnormal holes and pipelines are annotated and converted into mask images for use as test sets. The images are flipped and rotated to obtain enhanced data, which are then divided into training set, validation set, and test set in a ratio of 70%:15%:15%.

[0054] The image data and the mask image are constructed together to obtain an image dataset.

[0055] For the anomaly detection model described in step (2), a model for anomaly detection can be obtained by training based on the acquired normal sample data. At the same time, by focusing on the ratio of the individual difference of each pixel to the average value, its importance in training is measured to reduce the overlap between normal samples and abnormal samples.

[0056] Each image is decomposed into multiple local patches of 64*64 pixels in size. Each patch is a graph node, and the edges represent the similarity between them. The similarity between patches is constructed using color and texture features, and the cosine similarity is used for calculation.

[0057] The WideResnet 101 pre-trained model is selected as the feature extractor to extract the corresponding feature vector for each patch. At the same time, the self-attention mechanism is introduced to strengthen the feature representation and capture the long-distance dependency between them. The feature representation of the middle layer can avoid the excessive deviation of deep features from the data while retaining sufficient detail information.

[0058] Through the density-based clustering algorithm, samples are selected according to the data distribution density, samples in sparse areas are given priority, and detection is performed in areas that are more likely to contain outliers.

[0059] Use graph neural networks to learn the embedded representation of graphs, and use the convolutional layer of graph neural networks to aggregate node information and enhance the representation ability of features.

[0060] The contrast loss function is introduced to optimize the feature distribution difference between normal and abnormal parts, concentrate the features of normal samples in the embedding space, disperse the features of abnormal samples, and reduce the overlapping parts.

[0061] During the training process, the weights are dynamically adjusted according to the ratio of the individual differences of each patch to the category mean. The tail samples will receive higher weights, optimizing the differences between patches and further improving the detection ability of the model.

[0062] For step (3), pixel-level patch reconstruction loss is used to calculate the corresponding reconstruction error to locate the abnormal area, and the reconstruction error of each patch block is compared with the global threshold to determine whether there is an abnormality.

[0063] In (2) and (3) of the present invention, each image is decomposed into multiple local patch blocks; the WideResnet101 pre-trained model is selected as a feature extractor, and the corresponding feature vector is extracted for each patch. The feature representation of the intermediate layer can avoid excessive deviation of the deep features from the data while retaining sufficient detail information. The features of the intermediate layer are the output features of the intermediate layer of the pre-trained model, and the features are enhanced by the self-attention mechanism for subsequent anomaly detection. The self-attention mechanism is introduced to enhance the feature representation. The output feature of the self-attention mechanism is a new feature vector, which contains the enhanced feature representation, that is, the features weighted and adjusted by the correlation of other patch blocks. These enhanced features can better represent the real semantics of the patch in the image and can better capture the relationship between local areas. The output features of the self-attention mechanism are passed to the graph convolution layer to aggregate node information and further optimize the feature representation of each patch block. Through the density-based clustering algorithm, samples in sparse areas are preferentially selected, and detection is performed in areas that are more likely to contain abnormal points. The feature vector of the selected sample is input into the self-attention mechanism for further enhancement. The enhanced features are passed to the graph neural network to aggregate node information through the graph convolution layer and further optimize the feature representation of each patch. The graph neural network is used to learn the embedded representation of the patch block, and the convolution layer of the graph neural network is used to aggregate node information to enhance the representation ability of the features. The output aggregated features are passed to the anomaly score calculation module to generate anomaly scores based on the reconstruction error. The similarity between features is calculated to generate anomaly scores:

[0064]

[0065] where f i and f jare the feature vectors of the two patches. The features used to calculate similarity are the embedded representations of the output of the graph neural network, that is, the features of each patch after aggregation through the graph convolution layer. These features can reflect the spatial relationship and feature similarity between patches. The output features of the feature extractor and the self-attention mechanism are intermediate representations that assist in generating the features that are ultimately used to calculate similarity.

[0066] The reconstruction error of the patch is calculated by the autoencoder to further assist segmentation and mark the areas with larger errors as abnormal areas.

[0067]

[0068] Where: f i ∈R d is the original input feature of the ith patch, is the feature after reconstruction of the i-th patch. It is the square of the L2 norm. The larger the error value, the poorer the reconstruction effect, which may be an abnormal area.

[0069] Based on the similarity calculation and reconstruction error, an anomaly score is generated.

[0070] Anomaly scoring formula:

[0071]

[0072] Where: Si is the abnormal score of the ith patch, α, β are weight coefficients, ReconstructionError i is the reconstruction error of the ith patch, N(i) represents the domain of the ith patch, Similarity(f i ,f j ) is the similarity between the i-th patch and the j-th patch, f i is the feature representation of the ith patch. The anomaly score is used to locate the abnormal area in the sample, calculate the anomaly score of all patches, and filter out possible abnormal areas based on the score.

[0073] During the training process, the weight is adjusted dynamically according to the difference between the features of each pixel and the category mean. The tail sample can obtain a higher weight, which can guide the model to learn the corresponding features. By increasing the weight of the tail sample, the overlap between normal samples and abnormal samples can be reduced, and the detection ability of the model can be improved.

[0074] Tail samples are obtained by identifying sparse regions, usually corresponding to abnormal samples in the data. The weights are weighted in the loss function so that the tail samples can guide the model to better learn features during training.

[0075] Dynamic weight adjustment formula:

[0076]

[0077] Where ReconstructionError i is the reconstruction error of the ith patch, μ is the mean of the reconstruction errors of all samples, and λ is a tuning hyperparameter.

[0078] The features of each pixel are obtained by further processing the features extracted by the pre-trained model through the graph neural network and calculating the autoencoder reconstruction error.

[0079] Contrastive loss function:

[0080]

[0081] Where D is Similarity(f i ,f j ), m is a hyperparameter used to control the minimum distance between negative sample pairs, ω i is the dynamic weight of sample i, y i is the label of the sample pair, indicating whether the samples belong to the same category, y i =1, indicating that the sample pair belongs to the same category, u i =0, indicating that the sample pair belongs to different categories.

[0082] Areas with larger errors are marked as abnormal and displayed visually to facilitate subsequent analysis and processing of the data.

[0083] Compared with the existing anomaly detection methods, the present invention has achieved the beneficial technical effect of enhancing the representation ability of features, and using the features collected in the middle layer for anomaly detection. This feature achieves a good balance between detail information and abstract semantic information, and effectively avoids the deviation of deep features to data. The use of this feature enables the model to better capture abnormal patterns while retaining sufficient spatial context information, thereby improving detection accuracy. By performing feature aggregation on a local neighborhood, the relationship between adjacent patches can be fully utilized and sufficient spatial context can be retained. This local aggregation method improves the sensitivity of anomaly detection, especially when dealing with problems in industrial scenarios, and can accurately identify minor anomalies. A dynamic weighting mechanism is introduced to optimize the tail samples and reduce the overlap between normal samples and abnormal samples. This improvement can significantly improve the reliability of detection, reduce false positives and false negatives, and thus enhance the usability of the model in practical applications.

[0084] The present invention performs well in small sample conditions, has good scalability and adaptability, and is suitable for various industrial application scenarios. This makes the method not limited to specific types of anomaly detection, but can be widely used in multiple fields.

[0085] The key innovations of the present invention are:

[0086] (1) Obtain radar data for road detection, perform image processing on the data, convert it into corresponding image data, and perform data enhancement on all image data to obtain the final image data set, where the training set only contains images of normal samples; (2) Adjust all converted images to a uniform size for use as a training set, annotate the hole images containing anomalies, convert them into mask images, use them as a test set, flip and rotate the images to obtain enhanced data, and divide the training set, validation set, and test set into a ratio of 70%:15%:15%; (3) Construct the image data and mask images together to obtain an image data set; (4) Construct a road anomaly detection model and a graph structure, decompose the enhanced sample image into local patches of 64*64 pixels, and each patch is used as a node in the graph. According to the similarity of color, texture features, etc. between patches, connections are established to form a graph structure; (5) Use GNN to learn the embedded representation of the patch block, and aggregate node information through the graph convolution layer , improve the feature representation ability of the patch block, and generate anomaly scores by calculating the similarity between features; (6) At the same time, during the detection process, the reconstruction error of the patch is calculated by the autoencoder to further assist segmentation, and the area with larger error is marked as an abnormal area; (7) During the training process, the weight is dynamically adjusted according to the difference between the feature of each pixel and the category mean, and the tail sample can obtain a higher weight, thereby guiding the model to learn the corresponding features. By increasing the weight of the tail sample, the overlap between normal samples and abnormal samples can be reduced, thereby improving the detection ability of the model; (8) Based on the detection, an anomaly detection segmentation model is constructed, and the patch blocks marked as abnormal are post-processed, and adjacent abnormal patch blocks are merged to form a complete abnormal area, which is finally visualized in the image for subsequent analysis and processing; (9) The detection model is trained and tested using the image dataset, and the optimal weight parameters of the detection model are selected to obtain the optimal detection model for detecting abnormal conditions in road data.

[0087] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

[0088] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A road anomaly detection method for small sample conditions, characterized in that: The method comprises the following steps: S1. Obtain radar data for road detection, perform image processing on the data, convert it into corresponding picture data, and perform data enhancement on all picture data to obtain a picture data set, wherein the training set only contains pictures of normal samples; S2. Construct a road anomaly detection network, input the training set into the road anomaly detection network, decompose the training set into local patch blocks, identify the tail samples, output enhanced feature representation based on the local patch blocks of the tail samples, learn the embedded representation of each local patch block, aggregate node information through the graph convolution layer, obtain aggregated features, calculate feature similarity based on the aggregated features, and calculate reconstruction errors, adjust the weights of the tail samples in the training set based on the reconstruction errors, and calculate anomaly scores based on the reconstruction errors and feature similarities, output anomaly areas, train the road anomaly detection network based on the contrast loss function, and obtain a road anomaly detection model; S3. Acquire actual road image data, input the road image data into a road anomaly detection model to obtain an actual abnormal region, input the actual abnormal region into an anomaly detection segmentation model, merge the actual abnormal region, and form a complete abnormal region.

2. The road anomaly detection method for small sample conditions according to claim 1, characterized in that: The specific steps of S2 are: Construct a road anomaly detection network, input the training set into the road anomaly detection network, and decompose the training set images into multiple local patch blocks; The pre-trained model is used as a feature extractor to extract the corresponding feature vector for each local patch block, filter the feature vector, identify the tail sample, and input the filtered feature vector into the self-attention mechanism, which outputs the enhanced feature representation. Strengthen the feature representation. The graph neural network input into the road anomaly detection network learns the embedding representation of the local patch block, aggregates the information, and obtains the aggregated features. The feature similarity is calculated based on the aggregated features. The aggregated features are passed through the autoencoder to obtain the reconstructed features. The reconstruction error is calculated based on the reconstructed features. The weight of the tail samples in the training set is adjusted based on the reconstruction error. At the same time, the anomaly score is calculated based on the reconstruction error and feature similarity, and the abnormal area is output; The contrast loss function is calculated based on the weight and feature similarity, and the road anomaly detection network is optimized based on the contrast loss function to obtain the road anomaly detection model.

3. The road anomaly detection method for small sample conditions according to claim 2, characterized in that: The reconstruction error is: Among them, ReconstructionError i is the reconstruction error of the ith local patch, f i is the original input feature of the i-th local patch block, is the feature reconstructed from the i-th local patch.

4. The road anomaly detection method for small sample conditions according to claim 3, characterized in that: The feature similarity is: Among them, f i and f j It is the aggregate feature of two local patches.

5. The road anomaly detection method for small sample conditions according to claim 4, characterized in that: The anomaly scores are: Among them, S i is the anomaly score of the ith local patch, α, β are weight coefficients, and ReconstructionError i is the reconstruction error of the ith local patch, N(i) represents the domain of the ith local patch, Similarity(f i ,f j ) is the similarity between the i-th local patch block and the j-th local patch block, f i is the aggregate feature of the i-th local patch.

6. The road anomaly detection method for small sample conditions according to claim 5, characterized in that: The weight of the tail sample in the adjusted training set is: Among them, ReconstructionError i is the reconstruction error of the ith patch, μ is the mean of the reconstruction errors of all samples, and λ is a tuning hyperparameter.

7. The road anomaly detection method for small sample conditions according to claim 6, characterized in that: The contrast loss function is: Where D = Similarity(f i ,f j ), m is a hyperparameter used to control the minimum distance between negative sample pairs, ω i is the adjusted weight of sample i, y i is the label of the sample pair.

8. The road anomaly detection method for small sample conditions according to claim 2, characterized in that: The identification tail samples are as follows: Through the density-based clustering algorithm, samples in the sparse area are selected as tail samples.

9. The road anomaly detection method for small sample conditions according to claim 2, characterized in that: The specific steps of calculating the anomaly score based on the reconstruction error and feature similarity and outputting the abnormal area are as follows: The anomaly score is calculated based on the reconstruction error and feature similarity, and the local patch block with anomaly score exceeding the threshold is regarded as the abnormal area.

10. The road anomaly detection method for small sample conditions according to claim 2, characterized in that: The pre-trained model is the WideResnet 101 model.

Images