Road spilled object detection method based on misinformation library feature comparison, storage medium, equipment and computer program product

Abstract

The invention discloses a road spilled object detection method based on misinformation library feature comparison, a storage medium, equipment and a computer program product. The method comprises the steps of collecting traffic scene pictures in real time, inputting the traffic scene pictures into a YOLOv8 model, predicting road spilled objects, inputting the traffic scene pictures into a trained classification network, and obtaining predicted feature vectors and classifications of the road spilled objects; according to the classification result, calculating the similarity between the predicted feature vector of the road spilled object and the feature vectors of the misreported road spilled objects in all the misreported pictures of the same category in a misreported library, if the maximum similarity is higher than a set similarity threshold, determining that the road spilled objects are the misreported road spilled objects, and storing the feature vectors of the misreported road spilled objects and the classification in the misreported library; and otherwise, determining that the object is the identified road throwing object. According to the invention, through comparison with the misinformation library features, the detection precision of the road spilled objects is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road spilled object detection, and in particular to a road spilled object detection method, storage medium, device and computer program product based on false alarm library feature comparison. Background Art

[0002] Using surveillance video for target detection is currently the mainstream method for detecting road spills. However, the actual situation is more complicated, and there are many objects that interfere with detection, such as light spots, construction tools, etc. Manual monitoring can easily miss or detect errors, resulting in errors in the results. Therefore, computer vision technology is often used to improve the accuracy of road spill detection.

[0003] At present, the solutions for improving the accuracy of road spillage detection using computer vision technology include: (1) obtaining image frame difference maps by comparing images at adjacent moments in the monitoring video stream, and using deep learning algorithms to predict the image frame difference maps, thereby detecting the location and type of road spillage. This method is a processing method for dynamic changes in monitoring videos, but there may be multiple changing non-road spillage targets within the monitoring range at the same time, such as mobile workers, mobile cones, lens water droplets, etc. This method cannot distinguish the changing non-road spillage, so the final road spillage detection accuracy is limited; (2) Obtain a highway background dataset and a spilled object image dataset, perform image fusion to generate a road spilled object dataset with richer category samples, and use the target detection model for fine-tuning training to obtain an algorithm for road spilled object detection. This method improves the accuracy of road spilled object detection by supplementing the information of the spilled object dataset. However, the types of actual spilled objects are relatively complex. For example, cargo dropped from a truck is considered road spilled objects, and the types of cargo are too rich and complex. The dataset generated by the fusion of the two image datasets still has the problem of uneven types of road spilled objects, resulting in missed detection of real road spilled objects. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a road spilled object detection method, storage medium, device and computer program product based on false alarm library feature comparison, which improves the road spilled object detection accuracy by comparing with the false alarm library features.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a method for detecting road spilled objects based on false alarm library feature comparison, specifically comprising the following steps:

[0006] Step S1: collect a large number of traffic scene pictures, input them into the YOLOv8 model, and identify traffic scene pictures that may contain road spills;

[0007] Step S2, manually reviewing the identified traffic scene pictures that may contain road spills and classifying them into false positive pictures and correct pictures;

[0008] Step S3: construct a classification network, input traffic scene pictures that may contain road spills into the classification network for training until the cross entropy loss function converges, and the training of the classification network is completed;

[0009] Step S4: Input the false alarm picture into the trained classification network, obtain the feature vector and classification of the false alarm picture of the road spilled objects, and save them to the false alarm library; input the correct picture into the trained classification network, obtain the feature vector and classification of the road spilled objects in the correct picture;

[0010] Step S5: collect traffic scene pictures in real time, input them into the YOLOv8 model, predict road spills, input them into the trained classification network, and obtain the feature vector and classification of the predicted road spills;

[0011] Step S6: Based on the classification result, calculate the similarity between the feature vector of the predicted road spilled object and the feature vector of the falsely reported road spilled object in all false alarm pictures of the same category in the false alarm library. If the maximum similarity is higher than the set similarity threshold, it is a falsely reported road spilled object, and the feature vector and classification of the falsely reported road spilled object are saved in the false alarm library; otherwise, it is an identified road spilled object.

[0012] Furthermore, step S1 includes the following sub-steps:

[0013] Step S1.1, collect a large number of traffic scene pictures, mark the road spillage in each traffic scene picture as the training label of the YOLOv8 model, use the traffic scene picture as the input of the YOLOv8 model, and train the YOLOv8 model;

[0014] Step S1.2, repeat step S1.1 until the mean square error loss function converges, completing the training of the YOLOv8 model;

[0015] Step S1.3: input the collected traffic scene pictures into the trained YOLOv8 model to identify traffic scene pictures containing road spilled objects.

[0016] Furthermore, the classification network consists of an image preprocessing module, a feature extraction module and a classification output head. The image preprocessing module corrects and unifies the resolution of the input traffic scene picture containing road spilled objects; the feature extraction module extracts road spilled object features from the resolution-corrected and unified traffic scene picture containing road spilled objects to obtain a feature vector of the road spilled objects; and the classification output head classifies the extracted road spilled object features through a fully connected network and a softmax function.

[0017] Furthermore, the cross entropy loss function loss is specifically:

[0018]

[0019] Where N represents the total number of traffic scene images with road spills participating in the training, n represents the index of N, and t n represents the classification category of the nth traffic scene picture containing road spills, prob n Represents the classification probability of the nth traffic scene picture containing road spills.

[0020] Furthermore, the similarity between the feature vector of the road spillage predicted in step S6 and the feature vector of the falsely reported road spillage in the false alarm image of the same category in the false alarm library is The calculation process is:

[0021]

[0022] Where f represents the feature vector of predicted road spills, represents the feature vector of the false alarm image of the same category in the false alarm library, and |||| represents the modulus of the calculated feature vector.

[0023] Furthermore, the calculation process of the similarity threshold set in step S6 is:

[0024]

[0025] Among them, i represents the similarity threshold set for the i-th category of road spills, represents the feature vector of road spillage in the mth correct image of the same category, represents f and The similarity of ε i Represents the adjustment degree of the similarity threshold of the road spills of the ith category.

[0026] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for detecting road spilled objects based on false alarm library feature comparison.

[0027] Furthermore, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting road spilled objects based on false alarm library feature comparison is implemented.

[0028] Furthermore, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for detecting road spilled objects based on false alarm library feature comparison.

[0029] Compared with the prior art, the present invention has the following beneficial effects: the road spilled object detection method based on false alarm library feature comparison of the present invention extracts and classifies the traffic scene pictures that may contain road spilled objects identified by the YOLOv8 model through a lightweight classification network, and calculates the similarity with the same type of features in the false alarm library, and determines the ones with similarity higher than the similarity threshold as falsely reported road spilled objects, thereby improving the accurate recognition of road spilled objects. Since the classification network adopts a lightweight network structure, the detection speed can be improved. In addition, the similarity threshold in the present invention is calculated by the similarity between the feature vector of the predicted road spilled object and the feature vector of the road spilled object in the correct picture of the same category, thereby increasing the adjustment degree of the similarity threshold. For each predicted road spilled object picture, a similarity threshold is set, which more accurately reflects the difference between each predicted road spilled object picture and the correct picture of the same category, and has higher flexibility and stronger adaptability, which is conducive to improving the detection accuracy of road spilled objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the road spilled object detection method based on false alarm library feature comparison of the present invention;

[0031] Figure 2 It is a schematic diagram of the classification network in the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0033] like Figure 1 This is a flow chart of a road spilled object detection method based on false alarm library feature comparison of the present invention, and the road spilled object detection method specifically includes the following steps:

[0034] Step S1: collect a large number of traffic scene pictures, input them into the YOLOv8 model, and identify traffic scene pictures that may contain road spills. The YOLOv8 model can learn a large number of road spills characteristics, thereby realizing the prediction of road spills. Specifically, it includes the following sub-steps:

[0035] Step S1.1, collect a large number of traffic scene pictures, mark the road spillage in each traffic scene picture as the training label of the YOLOv8 model, use the traffic scene picture as the input of the YOLOv8 model, and train the YOLOv8 model;

[0036] Step S1.2, repeat step S1.1 until the mean square error loss function converges, completing the training of the YOLOv8 model;

[0037] Step S1.3: input the collected traffic scene pictures into the trained YOLOv8 model to identify traffic scene pictures containing road spilled objects.

[0038] Step S2: manually review the identified traffic scene images that may contain road spills and classify them into false positive images and correct images.

[0039] Step S3. However, due to the complexity of the traffic scene, there may be light spots, construction tools, etc., which may cause false detection. Therefore, the present invention further analyzes the road spilled objects identified by the YOLOv8 model to improve the prediction accuracy of road spilled objects. By constructing a classification network, traffic scene pictures that may contain road spilled objects are input into the classification network for training until the cross entropy loss function converges, completing the training of the classification network.

[0040] like Figure 2 In the present invention, the classification network consists of an image preprocessing module, a feature extraction module and a classification output head. The image preprocessing module corrects and unifies the resolution of the input traffic scene picture containing road spills; the feature extraction module extracts the road spills features from the traffic scene picture containing road spills after the resolution correction, and obtains the feature vector of the road spills; the classification output head classifies the extracted road spills features through a fully connected network and a softmax function:

[0041] prob = Softmax(w·f+b)

[0042] Among them, prob represents the predicted category probability, w represents the fully connected layer weight, b represents the bias vector, and f represents the extracted road spill features.

[0043] The cross entropy loss function loss in the present invention is specifically:

[0044]

[0045] Where N represents the total number of traffic scene images with road spills participating in the training, n represents the index of N, and t n represents the classification category of the nth traffic scene picture containing road spills, prob n Represents the classification probability of the nth traffic scene picture containing road spills.

[0046] Step S4, input the false alarm picture into the trained classification network, obtain the feature vector and classification of the falsely reported road spilled objects in the false alarm picture, and save them to the false alarm library; input the correct picture into the trained classification network, obtain the feature vector and classification of the road spilled objects in the correct picture. The present invention can improve the classification speed of pictures through a lightweight classification network structure.

[0047] Step S5: collect traffic scene pictures in real time, input them into the YOLOv8 model, predict road spills, input them into the trained classification network, and obtain the feature vector and classification of the predicted road spills.

[0048] Step S6. Based on the classification result, calculate the similarity between the feature vector of the predicted road spilled objects and the feature vector of the falsely reported road spilled objects in all false alarm pictures of the same category in the false alarm library. If the maximum similarity is higher than the set similarity threshold, it is a falsely reported road spilled object. The feature vector and classification of the falsely reported road spilled object are saved in the false alarm library to enrich the feature vectors in the false alarm library and provide data support for subsequent false alarm comparisons, thereby improving the detection accuracy of road spilled objects. Otherwise, it is an identified road spilled object. The present invention calculates similarity with the same type of features in the false alarm library, and determines the ones with similarity higher than the similarity threshold as falsely reported road spilled objects, thereby improving the accurate recognition of road spilled objects; in addition, the similarity threshold in the present invention is obtained by calculating the similarity between the feature vector of the predicted road spilled object and the feature vector of the road spilled object in the correct picture of the same category, thereby increasing the adjustability of the similarity threshold, and a similarity threshold is set for each predicted road spilled object picture, which more accurately reflects the difference between each predicted road spilled object picture and the correct picture of the same category, has higher flexibility and stronger adaptability, and is conducive to improving the detection accuracy of road spilled objects.

[0049] The similarity between the feature vector of the road spillage predicted in the present invention and the feature vector of the falsely reported road spillage in the same category of false alarm pictures in the false alarm library The calculation process is:

[0050]

[0051] Where f represents the feature vector of predicted road spills, represents the feature vector of the false alarm image of the same category in the false alarm library, and |||| represents the modulus of the calculated feature vector.

[0052] The calculation process of the similarity threshold set in the present invention is:

[0053]

[0054] Among them, irepresents the similarity threshold set for the i-th category of road spills, represents the feature vector of road spillage in the mth correct image of the same category, represents f and The similarity of ε i Represents the adjustment degree of the similarity threshold of the road spills of the ith category.

[0055] The road spilled objects detection method based on false alarm library feature comparison of the present invention uses a lightweight classification network to extract and classify features of traffic scene pictures that may contain road spilled objects identified by the YOLOv8 model, and calculates similarity with the same type of features in the false alarm library, and determines those with similarities higher than a similarity threshold as false reports of road spilled objects, thereby improving the accurate identification of road spilled objects. Since the classification network adopts a lightweight network structure, the detection speed can be improved.

[0056] In a technical solution of the present invention, a computer-readable storage medium is also provided, storing a computer program, wherein the computer program enables a computer to execute the road spilled object detection method based on false alarm library feature comparison.

[0057] In a technical solution of the present invention, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting road spilled objects based on false alarm library feature comparison is implemented.

[0058] In a technical solution of the present invention, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the road spilled object detection method based on false alarm library feature comparison is implemented.

[0059] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0060] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0061] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for detecting road spilled objects based on false alarm library feature comparison, characterized in that: The specific steps include: Step S1: collect a large number of traffic scene pictures, input them into the YOLOv8 model, and identify traffic scene pictures that may contain road spills; Step S2, manually reviewing the identified traffic scene pictures that may contain road spills and classifying them into false positive pictures and correct pictures; Step S3: construct a classification network, input traffic scene pictures that may contain road spills into the classification network for training until the cross entropy loss function converges, and the training of the classification network is completed; Step S4: Input the false alarm picture into the trained classification network, obtain the feature vector and classification of the false alarm picture of the road spilled objects, and save them to the false alarm library; input the correct picture into the trained classification network, obtain the feature vector and classification of the road spilled objects in the correct picture; Step S5: collect traffic scene pictures in real time, input them into the YOLOv8 model, predict road spills, input them into the trained classification network, and obtain the feature vector and classification of the predicted road spills; Step S6: Based on the classification result, calculate the similarity between the feature vector of the predicted road spilled object and the feature vector of the falsely reported road spilled object in all false alarm pictures of the same category in the false alarm library. If the maximum similarity is higher than the set similarity threshold, it is a falsely reported road spilled object, and the feature vector and classification of the falsely reported road spilled object are saved in the false alarm library; otherwise, it is an identified road spilled object.

2. A method for detecting road spilled objects based on false alarm library feature comparison according to claim 1, characterized in that: Step S1 includes the following sub-steps: Step S1.1, collect a large number of traffic scene pictures, mark the road spillage in each traffic scene picture as the training label of the YOLOv8 model, use the traffic scene picture as the input of the YOLOv8 model, and train the YOLOv8 model; Step S1.2, repeat step S1.1 until the mean square error loss function converges, completing the training of the YOLOv8 model; Step S1.3: input the collected traffic scene pictures into the trained YOLOv8 model to identify traffic scene pictures containing road spilled objects.

3. A method for detecting road spilled objects based on false alarm library feature comparison according to claim 2, characterized in that: The classification network consists of an image preprocessing module, a feature extraction module and a classification output head. The image preprocessing module corrects and unifies the resolution of the input traffic scene picture containing road spilled objects; the feature extraction module extracts the road spilled object features from the resolution-corrected and unified traffic scene picture containing road spilled objects to obtain the feature vector of the road spilled objects; the classification output head classifies the extracted road spilled object features through a fully connected network and a softmax function.

4. The method for detecting road spilled objects based on false alarm library feature comparison according to claim 3 is characterized in that: The cross entropy loss function loss is specifically: Where N represents the total number of traffic scene images with road spills participating in the training, n represents the index of N, and t n represents the classification category of the nth traffic scene picture containing road spills, prob n Represents the classification probability of the nth traffic scene picture containing road spills.

5. The method for detecting road spilled objects based on false alarm library feature comparison according to claim 3, characterized in that: The similarity between the feature vector of the road spillage predicted in step S6 and the feature vector of the falsely reported road spillage in the false alarm image of the same category in the false alarm library The calculation process is: Where f represents the feature vector of predicted road spills, represents the feature vector of the false alarm image of the same category in the false alarm database, and || || represents the modulus of the calculated feature vector.

6. The method for detecting road spilled objects based on false alarm library feature comparison according to claim 5, characterized in that: The calculation process of the similarity threshold set in step S6 is: Among them, i represents the similarity threshold set for the i-th category of road spills, represents the feature vector of road spillage in the mth correct image of the same category, represents f and The similarity of ε i Represents the adjustment degree of the similarity threshold of the road spills of the ith category.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the road spilled object detection method based on false alarm library feature comparison as described in any one of claims 1 to 6.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting spilled objects on the road based on false alarm library feature comparison as described in any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting road spilled objects based on false alarm library feature comparison as described in any one of claims 1 to 6 is implemented.

Images