A road surface anomaly detection method based on unknown class target perception

Abstract

The application discloses a kind of based on unknown class target perception's road surface anomaly detection method, comprising the following steps: constructing road surface anomaly detection network model, road surface anomaly detection network model includes decoder, encoder, general perception scalar generation model and unknown class perception scalar generation module;Image acquisition, image data is obtained training picture, training picture is divided into known class and unknown class;Known training picture is used to train road surface anomaly detection network model to obtain model, and model is trained using mixed known class training picture and unknown class training picture to obtain model and calculate to obtain distribution, and image is detected using model, distribution and are used to judge image classification, and prediction result set is updated.For various categories of road surface anomalies, including road surface litter, water, potholes, etc., can effectively detect in real time, avoid the pure artificial work mode, alleviate the problem of timely discovery of road surface anomalies, help real-time discovery and timely processing of road surface anomalies.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a method for detecting road anomalies based on the perception of unknown target categories. Background Technology

[0002] In recent years, my country's transportation network has become increasingly sophisticated, greatly facilitating the flow of people and goods. Good road conditions are a prerequisite for the effective operation of this network. However, traffic accidents caused by road surface anomalies such as debris spills and water accumulation occur frequently, posing a significant threat to people's lives and property and being one of the main factors affecting driving safety. Road surface anomalies need to be detected and addressed promptly to eliminate hidden dangers and prevent accidents. However, in practice, the difficulty in timely detection of road surface anomalies hinders their effective handling. This is because the occurrence of road surface anomalies is highly random; their location and timing are unpredictable.

[0003] Traditional methods such as video patrols and road inspections are inefficient, not only having a low probability of detecting road anomalies but also typically lacking real-time capabilities. With the development of deep learning technology, the real-time performance and accuracy of detection models have greatly improved, providing a new approach to solving the problem of timely detection of road anomalies. Therefore, a general detection model based on deep learning is adopted for road anomaly detection.

[0004] Existing general-purpose detection models based on deep learning can usually only identify specific categories and are not directly applicable to situations where there are many types of road anomalies. Therefore, we propose a road anomaly detection method based on the perception of unknown category targets. Summary of the Invention

[0005] To address the problem of numerous road surface anomaly categories, the purpose of this invention is to provide a road surface anomaly detection method based on the perception of unknown category targets.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A road surface anomaly detection method based on unknown category target perception includes the following steps:

[0008] Step 1: Construct a road surface anomaly detection network model. The road surface anomaly detection network model includes a decoder, an encoder, a general perception scalar generation module, and an unknown category perception scalar generation module.

[0009] Step 2: Acquire images and digitize them to obtain training images, which include known and unknown categories;

[0010] Step 3: Train the road anomaly detection network model using known training images to obtain the model. ;

[0011] Step 4: Train the model using a mixture of training images with known categories and training images with unknown categories. Obtain the model And calculate the distribution and ;

[0012] Step 5: Use the model Detect images, using segmentation cloth and Determine the image classification and update the prediction result set.

[0013] Furthermore, the image from step two is processed by the encoder to obtain a training image.

[0014] Furthermore, in step three, the model is constructed. , including the output logic matrix Calculate the prediction result matrix Generate a set of scalars with unknown category awareness Given a set of category-aware scalars, calculate the uncertainty loss. .

[0015] Furthermore, the image is processed by the encoder and decoder to output a logic matrix. .

[0016] Furthermore, logical matrix The prediction result matrix is ​​obtained by calculating using the Softmax function. .

[0017] Furthermore, logical matrix The unknown category-aware scalar generation module generates a set of unknown category-aware scalars. .

[0018] Furthermore, logical matrix and corresponding tags The input general-purpose perceptual scalar generation module generates a set of perceptual scalars for known categories. .

[0019] Furthermore, an unknown category-aware scalar set is adopted. and known category-aware scalar sets Calculate uncertainty loss .

[0020] Furthermore, in step four, the model is constructed. The output logic matrix Logical matrix and corresponding tags The general-purpose perceptual scalar generation module generates a perceptual scalar matrix. Perceive scalar matrix and tags composition deposit .

[0021] Furthermore, the image detection in step five includes outputting logical values. Calculate the prediction results Generate perceptual scalar Computational perception scalar mask .

[0022] Furthermore, a distributed approach is adopted. and From the perspective of scalar Calculate the output of the sensing scalar mask .

[0023] The above solution achieved the following beneficial effects:

[0024] Compared to existing general detection models based on deep learning, which can usually only identify specific categories and are not directly applicable to situations with a wide variety of road surface anomalies, this solution can effectively detect various types of road surface anomalies in real time, including road debris, water accumulation, and potholes. This avoids purely manual work, alleviates the difficulty of timely detection of road surface anomalies, and helps to detect and handle road surface anomalies in real time. Attached Figure Description

[0025] Figure 1 This is a schematic flowchart of the road surface anomaly detection method according to an embodiment of the present invention.

[0026] Figure 2 This is a road surface anomaly detection network model according to an embodiment of the present invention. The training diagram is shown below.

[0027] Figure 3 This is a road surface anomaly detection network model according to an embodiment of the present invention. The training diagram is shown below.

[0028] Figure 4 This is a flowchart illustrating the unknown category-aware scalar generation module according to an embodiment of the present invention.

[0029] Figure 5 This is a flowchart illustrating the general perception scalar generation module according to an embodiment of the present invention.

[0030] Figure 6 This is a flowchart for obtaining a perceived scalar distribution according to an embodiment of the present invention.

[0031] Figure 7This is a schematic diagram of the image preprocessing process according to an embodiment of the present invention.

[0032] Figure 8 This is a schematic diagram of obtaining the point mask according to an embodiment of the present invention.

[0033] Figure 9 This is a schematic diagram illustrating the process of updating prediction results in an embodiment of the present invention. Detailed Implementation

[0034] The following detailed description illustrates the specific implementation method:

[0035] The basic implementation examples are as follows: Figure 1 As shown:

[0036] A road surface anomaly detection method based on unknown category target perception includes the following steps:

[0037] Step 1: Construct a road surface anomaly detection network model. The road surface anomaly detection network model includes a decoder, an encoder, a general perception scalar generation module, and an unknown category perception scalar generation module.

[0038] Step 2: Acquire images and digitize them to obtain training images, which include both categorized and unknown categories;

[0039] Step 3: Train the road anomaly detection network model using known training images to obtain the model. ;

[0040] Step 4: Train the model using a mixture of training images with known categories and training images with unknown categories. Obtain the model And calculate the distribution and ;

[0041] Step 5: Use the model Detecting images, utilizing distribution and Determine the image classification and update the prediction result set.

[0042] The image obtained in step two is processed by the encoder to obtain the training image.

[0043] The model is constructed in step three. , including the output logic matrix Calculate the prediction result matrix Generate a set of scalars with unknown category awareness Known category-aware scalar set Calculate the uncertainty loss .

[0044] The image is processed by the encoder and decoder to output a logic matrix. Logical matrix The prediction result matrix is ​​obtained by calculating using the Softmax function. Logical matrix The unknown category-aware scalar generation module generates a set of unknown category-aware scalars. ; Using an unknown category-aware scalar set and known category-aware scalar sets Calculate uncertainty loss .

[0045] Step four involves building the model. The output logic matrix Logical matrix and corresponding tags The general-purpose perceptual scalar generation module generates a perceptual scalar matrix. Perceive scalar matrix and tags composition deposit .

[0046] The image detection in step five includes outputting logical values. Calculate the prediction results Generate perceptual scalar Computational perception scalar mask ; adopting distribution and From the perspective of scalar Calculate the output of the sensing scalar mask .

[0047] The specific implementation process is as follows:

[0048] The first step is to construct a road anomaly detection network model, which includes a decoder, an encoder, a general perception scalar generation module, and an unknown category perception scalar generation module. The encoder is used to extract feature maps; the decoder is used to fuse feature maps and output predicted logical values; the general perception scalar generation module is used to generate known category perception scalars; and the unknown category perception scalar generation module is used to generate unknown category perception scalars.

[0049] The second step is to acquire images and then use an encoder to process the images into data to obtain training images.

[0050] The third step is to train a road surface anomaly detection network model using known training images to obtain the model. During training, images are input into the road anomaly detection model. After passing through the encoder and decoder, the output logic value matrix is ​​completed. Logical value matrix After passing through the Softmax function, the prediction result matrix is ​​obtained. Prediction result matrix and corresponding tags Calculate the classification loss and set similarity loss separately. Also, calculate the uncertainty loss. Specifically, the logic value matrix input to the unknown category-aware scalar generation module generates a set of unknown category-aware scalars. Logical value matrix and tags The input general-purpose perceptual scalar generation module generates a set of perceptual scalars for known categories. Unknown category-aware scalar set and known category-aware scalar sets Calculate uncertainty loss ;

[0051]

[0052] in, Scalar sets for perceiving unknown categories The first in A perceptual scalar A set of scalars for the perception of known categories The first in A perceptual scalar. For set The number of perceived scalars stored. It is the distance parameter of the unknown category perceptual scalar. For set The number of perceived scalars stored. Distance parameters for scalars of known categories.

[0053] The unknown category-aware scalar generation module performs the following operations during training:

[0054] (i) Obtain the logical value matrix (ii) Calculate the perception scalar matrix Perception scalar matrix The middle position is The perception scalar is calculated using the following formula:

[0055]

[0056] in, It is a logical value matrix The middle position is The (iii) Extract the perceptual scalar fraction. (iv) Output sorting in all perceptual scalar scores, and sorting the negative perceptual scalar scores in descending order. and All negative perceptual scalar scores between these values ​​constitute the set of perceptual scalars for the unknown category. .

[0057] The general-purpose perceptual scalar generation module performs the following operations during training:

[0058] (i) Obtain the logical value matrix and tags (ii) Calculate the perception scalar matrix (iii) Perception scalar matrix (iv) Set the perceptual scalar belonging to the background category to 0; In this step, the perceived scalar fraction is... Extract all perceptual scalars to form a set of perceptual scalars of known categories. .

[0059] The fourth step is to train the model using a mixture of training images of known categories and training images of unknown categories. Obtain the model And calculate the distribution and ;

[0060] During training, give the model Input training images After passing through the encoder and decoder, the output logic value matrix is ​​completed. Logical value matrix After passing through the Softmax function, the prediction result matrix is ​​obtained. Prediction result matrix and corresponding tags Calculate the classification loss and set similarity loss separately.

[0061] At the same time, the logical value matrix and tags The general-purpose perceptual scalar generation module generates a perceptual scalar matrix. Perception scalar matrix and tags composition deposit .

[0062] After training, distribution and Obtain it through the following steps:

[0063] (i) Obtain the set Let set Any element in is represented as ,in , (ii) For any training batch size in this step; ,if Then the negative Store in an unknown category-aware scalar set ,if and Then the negative Store in a known category-aware scalar set ,in , Take in sequence After repeating the above steps, the final set of unknown category-aware scalars can be obtained. and known category-aware scalar sets (iii) Calculate the sets respectively and Gaussian distribution and .

[0064] Fifth step, adopt the model Detecting images, utilizing distribution and Determine image classification and update the prediction result set. During the detection process, detect images... After being processed by the image preprocessing module, it is input into the road surface anomaly detection model. , obtain logical value Logical value After applying Softmax, the prediction result is obtained. Logical value After inputting into the general perception scalar generation module, the perception scalar is obtained. Using distribution and From the perspective of scalar The perceptual scalar mask is calculated in the middle. Perceiving scalar masks and prediction results The prediction results were obtained after updating the prediction process. Prediction results The output prediction process determines whether to output the detection results.

[0065] Image preprocessing, during the detection process, involves the following operations:

[0066] (i) Obtain test image And load the point mask (ii) For the test image (iii) Noise reduction processing; and point masking film Perform an AND operation to generate the preprocessed image. .

[0067] Perception scalar mask During the testing process, the following operations shall be performed:

[0068] (i) Initialize the width to Height is And the tensor with all zeros is a perceptual scalar mask. (ii) For perceptual scalars A certain value in , , Using unknown category distribution Calculate the negative perceptual scalar value probability Using known category distributions Calculate the negative perceptual scalar value probability ;when Update the perception scalar mask at that time. (iii) Take all perceptual scalar values ​​in sequence. After repeating the above steps to update the perceptual scalar mask, the final perceptual scalar mask can be obtained. .

[0069] To update the prediction results, the following operations are performed during the detection process:

[0070] (i) For a perceptual scalar mask A certain value in ,when Record the index at that time. (ii) Update the prediction results The value is -1; (iii) Take all of them in sequence. Repeat the above steps to update the prediction results. , , The updated prediction results can then be obtained. .

[0071] Post-processing of prediction results involves performing the following operations during the detection process:

[0072] (i) Using an expansion rate Gaussian convolution kernels predict results Perform filtering. Let the standard deviation be... The kernel size is The weights of the coordinates are :

[0073]

[0074] in, and .

[0075] (ii) Under the condition of eight adjacent connections, the prediction results Perform connected component analysis on regions with a median of -1 to obtain the set of maximum bounding rectangles for each connected component. and the corresponding rectangular area , This represents the number of connected components. Represents any maximum bounding rectangle. This represents the area of ​​the corresponding rectangle. When the area of ​​the rectangle At that time, output road surface anomaly prediction .

[0076] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0077] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting road surface anomalies based on the perception of unknown category targets, characterized in that: Includes the following steps: Step 1: Construct a road surface anomaly detection network model. The road surface anomaly detection network model includes a decoder, an encoder, a general perception scalar generation module for generating known category perception scalars, and an unknown category perception scalar generation module for generating unknown category perception scalars. The encoder is used to extract feature maps, and the decoder is used to fuse feature maps and output predicted logical values. Step 2: Acquire images and digitize them to obtain training images, which include known and unknown categories; Step 3: Train the road anomaly detection network model using known training images to obtain the model. ;Model Including output logic matrix Generate a set of scalars with unknown category awareness Known category-aware scalar set And calculate uncertainty loss The image is processed by the encoder and decoder to output a logic matrix. Logical value matrix The unknown category-aware scalar generation module generates a set of unknown category-aware scalars. Logical value matrix and tags The input general-purpose perceptual scalar generation module generates a set of perceptual scalars for known categories. Unknown category-aware scalar set and known category-aware scalar sets Calculate uncertainty loss ; ； in, Scalar sets for perceiving unknown categories The first in A perceptual scalar A set of scalars for the perception of known categories The first in A perceptual scalar; For set The number of perceived scalars stored. It is the distance parameter of the scalar for unknown category perception; For set The number of perceived scalars stored. Distance parameters for scalars of known categories; Step 4: Train the model using a mixture of training images with known categories and training images with unknown categories. Obtain the model And calculate the distribution and ; Step 5: Use the model Detecting images, utilizing distribution and Determine the image classification and update the prediction result set.

2. The road surface anomaly detection method based on unknown category target perception according to claim 1, characterized in that: The image obtained in step two is processed by the encoder to obtain the training image.

3. The road surface anomaly detection method based on unknown category target perception according to claim 1, characterized in that: The model is constructed in step three. ,Model It also includes calculating the prediction result matrix. .

4. The road surface anomaly detection method based on unknown category target perception according to claim 3, characterized in that: Logical matrix The prediction result matrix is ​​obtained by calculating using the Softmax function. Based on the prediction result matrix and corresponding tags Calculate the classification loss and set similarity loss separately.

5. The road surface anomaly detection method based on unknown category target perception according to claim 1, characterized in that: Step four involves building the model. The output logic matrix Logical matrix and corresponding tags The general-purpose perceptual scalar generation module generates a perceptual scalar matrix. Perceive scalar matrix and tags composition deposit .

6. The road surface anomaly detection method based on unknown category target perception according to claim 1, characterized in that: The image detection in step five includes outputting logical values. Calculate the prediction results Generate perceptual scalar Computational perception scalar mask , adopting distribution and From the perspective of scalar Calculate the output of the sensing scalar mask .

Images