Road object target detection method based on detection and semantic segmentation model
By combining multi-model collaboration and multi-level verification methods with YOLOv8, Vision Transformer, ResNet and hyperspectral image analysis technologies, the problems of low target detection accuracy and missed detection in highway scenarios are solved, and higher detection accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411838576.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-03-25
AI Technical Summary
The existing road object object detection methods have low detection accuracy in highway scenarios, and there are problems of false detection and missed detection, especially in complex environments and low-probability events, which are difficult to improve model accuracy.
The road object object detection method based on detection and semantic segmentation models is adopted, combined with YOLOv8, Vision Transformer, ResNet and hyperspectral image analysis technology, and through multi-model collaboration and multi-level verification, false detection and missed detection are reduced.
It improves detection accuracy, enhances the robustness and generalization of the model, effectively deals with false detection in complex scenarios, and ensures the reliability and accuracy of the final output results.
Smart Images
Figure CN119693621A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of traffic management, and in particular relates to a road object target detection method based on detection and semantic segmentation model. Background Art
[0002] At present, with the continuous acceleration of my country's modernization process, the number of cars has increased dramatically, and the issue of vehicle driving safety has received widespread attention. Non-target detection of people and vehicles based on highway cameras can significantly improve the safety management level of highways. Through real-time monitoring and intelligent recognition, the system can detect illegal entry or driving of pedestrians, non-motor vehicles and abnormal vehicles at the first time, thereby effectively avoiding potential safety hazards. At the same time, non-target detection of people and vehicles based on highway intersection cameras can reduce the workload of traffic management personnel and improve work efficiency.
[0003] With the development of intelligent transportation systems, research on non-target detection of people and vehicles based on highway cameras has also attracted widespread attention, and many researchers have achieved remarkable results in this field. At present, deep learning models for target detection are mainly divided into the YOLO series and the RCNN series. The RCNN series is a representative algorithm based on region detection, and YOLO is a representative algorithm based on region extraction. Target detection based on highway scenes requires accurate recognition of target detection in a bird's-eye view and in complex traffic environments. However, the robustness and generalization of traditional target detection models still need to be further improved, which also causes certain challenges in target detection in highway scenes, which needs further improvement.
[0004] Due to weather, traffic accidents and other reasons, it is easy for emergencies such as rear-end collisions to occur on highways, making the highway scene cluttered and difficult to detect targets. Therefore, using only deep learning models such as the YOLO system and the RCNN series for target detection on highways may result in false detections and missed detections. Traditionally, the accuracy of the model is usually improved by improving the quality of the data set, training negative samples, and hard example mining. However, for the identification of scattered objects (such as rainproof cloths, debris, etc.) on highways, it is difficult to collect data sets for such low-probability events, which also makes it difficult to improve the quality of the data sets for training models. Therefore, it is difficult to improve the accuracy of the model and reduce the occurrence of false detections and missed detections in target detection by relying solely on traditional methods. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide a road object target detection method based on detection and semantic segmentation model. The present invention aims to solve the problems of low detection accuracy, false detection and missed detection in existing road object target detection methods.
[0006] The present invention provides a road object target detection method based on detection and semantic segmentation model, comprising the following steps:
[0007] S1. Training and recognition of YOLOv8 model;
[0008] Collect annotated datasets suitable for road target detection, use the annotated data to train the YOLOv8 model, and optimize the model parameters to improve detection accuracy;
[0009] S2. Train the Vision Transformer model to perform semantic segmentation of highway images to distinguish highway areas from non-highway areas;
[0010] S3. Perform preliminary target detection based on highway surveillance video images;
[0011] The highway surveillance video is converted into frame images and input into the trained YOLOv8 target detection model. The target objects in the road scene are identified by the YOLOv8 target detection model, and the target frame is generated as the detection result.
[0012] S4. Reduce the number of false detection targets outside the highway area;
[0013] Input the target frame and target frame coordinate information of the detection result of the YOLOv8 model in step S3 into the trained Vision Transformer model; perform semantic segmentation on the entire image scene through the Vision Transformer model to determine which areas belong to the highway range, and spatially match the segmentation result with the position of the target frame of the YOLOv8 model;
[0014] If the target frame is located in the highway area, the detected target is retained and the process goes to step S5 for the next step;
[0015] If the target box is not within the highway area, it is considered a false detection and deleted directly;
[0016] S5. Reduce the number of false detection targets in the highway area;
[0017] For the target box retained in step S4, input the trained ResNet model for secondary recognition, and then compare the semantic classification results of the ResNet model with the detection results of the YOLOv8 model:
[0018] If the two results are consistent, the recognition result of the target frame is considered reliable and the target frame is directly output as the final detection result;
[0019] If the two results are inconsistent, the target box is marked as "possible false detection" and the process goes to step S6 for further analysis;
[0020] S6. Secondary analysis of possible misdetected targets through the hyperspectral imaging system;
[0021] According to the target frame position of YOLOv8, that is, the X and Y coordinates of the target frame, the hyperspectral image data of the corresponding area is called, and the hyperspectral analysis technology is used to determine whether there is a real object in the area:
[0022] If an object exists: the original image is re-cropped based on the target frame recognized by YOLOv8, redundant information in the photo is removed, the cropped image is re-input into the YOLOv8 model for detection, the corrected target frame is generated and the detection result is updated;
[0023] If the object does not exist: the target box is considered to be a false detection, and the target box is directly deleted to complete the final result output.
[0024] Furthermore, in step S1, the collected annotated data set includes pedestrians, construction workers, cars, trucks, buses and road spills.
[0025] Furthermore, in step S6, the redundant information includes irrelevant parts of the background and edges in the photo.
[0026] The present invention also provides a road object target detection system based on detection and semantic segmentation model, including a YOLOv8 line target detection unit, a recognition result judgment unit, a recognition result misdetection processing unit and a query and detection unit.
[0027] The YOLOv8 target detection unit takes the highway monitoring video frame image as input and uses the YOLOv8 model to perform target detection on the image;
[0028] The recognition result judgment unit is used to input the recognition result of the YOLOv8 model into the Vision Transformer model to determine whether the recognition result of the YOLOv8 model is located on a highway;
[0029] The recognition result misdetection processing unit is used to use the Resnet model to determine whether the semantic classification result of the Resnet model is consistent with the recognition result of the YOLOv8 model. If they are inconsistent, the result will be output, and the misdetection situation will be secondary processed by using the hyperspectral image calling system. If an object exists, the image is re-cropped according to the recognition result of the YOLOv8 model to remove redundant information in the photo, and the cropped image is re-input into the YOLOv8 model for recognition; if the object does not exist, the target box is deleted and the final result is output;
[0030] The query and detection unit is used for quickly retrieving and querying recorded road object targets.
[0031] Beneficial effects:
[0032] 1. Multi-model collaboration to improve detection accuracy. YOLOv8, Vision Transformer (ViT), ResNet and hyperspectral image analysis technology are combined to reduce the misdetection and missed detection of YOLOv8 model recognition results through multi-model cross-validation.
[0033] 2. Multi-level verification to enhance robustness. A progressive verification mechanism is adopted to eliminate low-confidence or unreasonable target boxes at each step, and output a more reliable final result. At the same time, through the introduction of hyperspectral images, it can effectively handle misdetection in complex scenes and avoid the problem of insufficient performance of a single model in a specific scene.
[0034] 3. Modular design facilitates expansion and optimization. Each module (YOLOv8, ViT, ResNet, hyperspectral analysis) is independent, and a single module can be upgraded or the processing flow can be adjusted according to actual needs without affecting the operation of the overall system. This solution can introduce other models that are more suitable for local conditions to further improve the recognition effect according to actual conditions.
[0035] 4. Strong interpretability. Through the clear target frame screening and correction process at each stage, the possible sources of errors in the detection process can be traced, which is convenient for result analysis and subsequent optimization of the system. When further checking the misdetection of the YOLOv8 model, the introduction of hyperspectral images provides a physical basis for the existence and spatial distribution of objects, making the detection results more credible.
[0036] 5. Practical application-oriented and adaptable to complex scenarios. The solution is designed based on the characteristics of highway scenarios and can maintain high detection performance in a variety of complex scenarios (such as highway construction, traffic congestion, pedestrian crossing, etc.).
[0037] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 A flow chart of a road object target detection method based on detection and semantic segmentation model of the present invention;
[0039] Figure 2 It is a component unit of a road object target detection system based on detection and semantic segmentation model of the present invention. DETAILED DESCRIPTION
[0040] In order to make the technical solutions, advantages and purposes of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of this application.
[0041] like Figure 1 As shown, the present invention provides a road object target detection method based on detection and semantic segmentation model, comprising the following steps:
[0042] S1. Training and recognition of YOLOv8 model;
[0043] Collect annotated datasets suitable for road target detection (such as pedestrians, construction workers, cars, trucks, buses, and road spills), use the annotated data to train the YOLOv8 model, and optimize the model parameters to improve detection accuracy;
[0044] S2. Train the Vision Transformer model to perform semantic segmentation of highway images to distinguish highway areas from non-highway areas;
[0045] S3. Perform preliminary target detection based on highway surveillance video images;
[0046] The highway surveillance video is converted into frame images and input into the trained YOLOv8 target detection model. The target objects (such as vehicles, pedestrians, obstacles, etc.) in the road scene are identified by the YOLOv8 target detection model, and a bounding box is generated as the detection result.
[0047] S4. Reduce the number of false detection targets outside the highway area;
[0048] Input the target detection result (including the target frame and its coordinate information) of the YOLOv8 model in step S3 into the trained Vision Transformer model; perform semantic segmentation on the entire image scene through the Vision Transformer model to determine which areas belong to the highway range, and spatially match the segmentation result with the position of the target frame of the YOLOv8 model;
[0049] If the target frame is located in the highway area, the detected target is retained and the process goes to step S5 for the next step;
[0050] If the target box is not within the highway area, it is considered a false detection and deleted directly;
[0051] This step can significantly reduce false detections caused by the external environment of the highway (such as background, pedestrians or other irrelevant objects).
[0052] S5. Reduce the number of false detection targets in the highway area;
[0053] For the target box retained in step S4, input the trained ResNet model for secondary recognition, and then compare the semantic classification results of the ResNet model with the detection results of the YOLOv8 model:
[0054] If the two results are consistent, the recognition result of the target frame is considered reliable and the target frame is directly output as the final detection result;
[0055] If the two results are inconsistent, the target box is marked as "possible false detection" and the process goes to step S6 for further analysis;
[0056] This step further optimizes the accuracy of object recognition by combining the semantic classification capabilities of the ResNet model.
[0057] S6. Secondary analysis of possible misdetected targets through the hyperspectral imaging system;
[0058] According to the target frame position (X, Y coordinates) of YOLOv8, the hyperspectral image data of the corresponding area is called, and the hyperspectral analysis technology is used to determine whether there is a real object in the area:
[0059] If there is an object: the original image is re-cropped based on the target frame recognized by YOLOv8, the redundant information in the photo (including the background in the photo and irrelevant parts of the edge) is removed, the cropped image is re-input into the YOLOv8 model for detection, the corrected target frame is generated and the detection result is updated;
[0060] If there is no object: the target box is considered to be a false detection, and the target box is directly deleted to complete the final result output;
[0061] This step effectively improves the model's ability to judge suspected misdetected targets by introducing hyperspectral image analysis technology, ensuring the reliability and accuracy of the final output results.
[0062] like Figure 2 As shown, the present invention provides a road object target detection system based on detection and semantic segmentation model, including a YOLOv8 line target detection unit, a recognition result judgment unit, a recognition result misdetection processing unit and a query and detection unit;
[0063] The YOLOv8 target detection unit takes the highway monitoring video frame image as input and uses the YOLOv8 model to perform target detection on the image;
[0064] The recognition result judgment unit is used to input the recognition result of the YOLOv8 model into the Vision Transformer model to determine whether the recognition result of the YOLOv8 model is located on a highway;
[0065] The recognition result misdetection processing unit is used to use the Resnet model to determine whether the semantic classification result of the Resnet model is consistent with the recognition result of the YOLOv8 model. If not, the result will be output, and the hyperspectral image will be used to call the system for secondary processing of the misdetection. If an object exists, the image will be re-cropped according to the recognition result of the YOLOv8 model to remove redundant information in the photo, and the cropped image will be re-input into the YOLOv8 model for recognition; if the object does not exist, the target box will be deleted and the final result will be output;
[0066] The query and detection unit is used to quickly retrieve and query recorded road object targets.
[0067] It is hereby declared that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should be included in the scope of the claims of the present invention.
Claims
1. A road object detection method based on detection and semantic segmentation model, characterized in that: The following steps are involved: S1. Training and recognition of YOLOv8 model; Collect annotated datasets suitable for road target detection, use the annotated data to train the YOLOv8 model, and optimize the model parameters to improve detection accuracy; S2. Train the Vision Transformer model to perform semantic segmentation of highway images to distinguish highway areas from non-highway areas; S3. Perform preliminary target detection based on highway surveillance video images; The highway surveillance video is converted into frame images and input into the trained YOLOv8 target detection model. The target objects in the road scene are identified by the YOLOv8 target detection model, and the target frame is generated as the detection result. S4. Reduce the number of false detection targets outside the highway area; Input the detection result target frame and target frame coordinate information of the YOLOv8 model in step S3 into the trained VisionTransformer model; perform semantic segmentation on the entire image scene through the Vision Transformer model to determine which areas belong to the highway range, and spatially match the segmentation result with the position of the target frame of the YOLOv8 model; If the target frame is located in the highway area, the detected target is retained and the process goes to step S5 for the next step; If the target box is not within the highway area, it is considered a false detection and deleted directly; S5. Reduce the number of false detection targets in the highway area; For the target box retained in step S4, input the trained ResNet model for secondary recognition, and then compare the semantic classification results of the ResNet model with the detection results of the YOLOv8 model: If the two results are consistent, the recognition result of the target frame is considered reliable and the target frame is directly output as the final detection result; If the two results are inconsistent, the target box is marked as "possible false detection" and the process goes to step S6 for further analysis; S6. Secondary analysis of possible misdetected targets through the hyperspectral imaging system; According to the target frame position of YOLOv8, that is, the X and Y coordinates of the target frame, the hyperspectral image data of the corresponding area is called, and the hyperspectral analysis technology is used to determine whether there is a real object in the area: If an object exists: the original image is re-cropped based on the target frame recognized by YOLOv8, redundant information in the photo is removed, the cropped image is re-input into the YOLOv8 model for detection, the corrected target frame is generated and the detection result is updated; If the object does not exist: the target box is considered to be a false detection, and the target box is directly deleted to complete the final result output.
2. The method for detecting road objects based on detection and semantic segmentation model according to claim 1, characterized in that: In step S1, the collected annotated data set includes pedestrians, construction workers, cars, trucks, buses and road spills.
3. The method for detecting road objects based on detection and semantic segmentation model according to claim 2, characterized in that: In step S6, the redundant information includes irrelevant parts of the background and edges in the photo.
4. A road object detection system based on detection and semantic segmentation model, characterized by: It includes a YOLOv8 target detection unit, a recognition result judgment unit, a recognition result misdetection processing unit, and a query and detection unit; The YOLOv8 target detection unit takes the highway monitoring video frame image as input and uses the YOLOv8 model to perform target detection on the image; The recognition result judgment unit is used to input the recognition result of the YOLOv8 model into the Vision Transformer model to determine whether the recognition result of the YOLOv8 model is located on a highway; The recognition result misdetection processing unit is used to use the Resnet model to determine whether the semantic classification result of the Resnet model is consistent with the recognition result of the YOLOv8 model. If they are inconsistent, the result will be output, and the misdetection situation will be secondary processed by using the hyperspectral image calling system. If an object exists, the image is re-cropped according to the recognition result of the YOLOv8 model to remove redundant information in the photo, and the cropped image is re-input into the YOLOv8 model for recognition; if the object does not exist, the target box is deleted and the final result is output; The query and detection unit is used for quickly retrieving and querying recorded road object targets.