Multi-target detection method for complex traffic scene

Abstract

The invention discloses a multi-target detection method for a complex traffic scene, and the method comprises the steps: collecting the original data in the traffic scene through a plurality of sensors disposed in a traffic environment, and enabling the sensors to capture the information of the traffic scene from different angles and dimensions, thereby providing a rich data basis for the subsequent data fusion and target detection, and improving the detection precision. Meanwhile, multi-modal data fusion can make full use of the advantages of different sensors, make up for the defects of a single sensor, improve the accuracy and robustness of target detection, construct an algorithm specially used for processing multi-target detection and tracking in a complex traffic scene, realize stable tracking and detection of multiple targets, and improve the accuracy and robustness of target detection. The traffic monitoring and management efficiency is improved, a plurality of targets in a traffic scene can be accurately detected by analyzing and processing the fused features, the dynamic changes of the targets can be tracked in real time, and accurate decision support is provided for automatic driving and an intelligent traffic system.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic scene detection, and in particular to a multi-target detection method for complex traffic scenes. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, machine vision detection technology is increasingly used in the transportation field. In complex traffic scenarios, such as urban streets and highways, the coexistence of multiple traffic participants (such as vehicles, pedestrians, bicycles, etc.) and dynamically changing environmental conditions (such as lighting changes, occlusions, weather effects, etc.) bring huge challenges to target detection and tracking.

[0003] Although existing machine vision detection technology can handle target detection and tracking tasks in traffic scenes to a certain extent, it still has many shortcomings when facing complex and changeable traffic environments. First, the data from a single sensor is often difficult to fully reflect the complexity of traffic scenes, which limits the accuracy and robustness of target detection. Second, problems such as target occlusion, deformation, and motion blur in multi-target scenes also increase the difficulty of target tracking. Summary of the invention

[0004] In view of this, the present invention proposes a multi-target detection method for complex traffic scenes, which can solve the defects of low accuracy and robustness, and great difficulty in multi-target tracking in the prior art.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A multi-target detection method for complex traffic scenes, specifically comprising:

[0007] Collect raw data in traffic scenes through multimodal sensors deployed in traffic environments;

[0008] The raw data in the traffic scene collected by multimodal sensors are fused to obtain sensor-level fusion features and feature-level fusion features;

[0009] Building a target detection and tracking algorithm, the target detection and tracking algorithm is used to process and track multiple targets in complex traffic scenarios;

[0010] According to the target detection and tracking algorithm, the sensor-level fusion features and the feature-level fusion features are analyzed to obtain multi-target detection and tracking results.

[0011] As a further optional solution of the multi-target detection method for complex traffic scenes, the raw data in the traffic scene collected by the multimodal sensor is fused to obtain sensor-level fusion features and feature-level fusion features, which specifically includes:

[0012] Preprocessing the raw data in the traffic scene collected by the multimodal sensor to obtain preprocessed data;

[0013] Align and integrate the preprocessed data in time and space to obtain sensor-level fusion features;

[0014] Extract feature information from preprocessed data;

[0015] The feature information is integrated according to the feature fusion algorithm to obtain feature-level fusion features.

[0016] As a further optional solution of the multi-target detection method for complex traffic scenes, the construction of the target detection and tracking algorithm specifically includes:

[0017] Based on the YOLO model, the model structure of the YOLO model is optimized to obtain an optimized YOLO model;

[0018] The optimized YOLO model is trained to obtain a target detection model, wherein the input of the target detection model is sensor-level fusion features and feature-level fusion features, and the output is the position, size, category and state information of the target.

[0019] As a further optional solution of the multi-target detection method for complex traffic scenes, the model structure of the YOLO model is optimized, specifically including:

[0020] Analyze the diversity of target scales in complex traffic scenes and determine the multi-scale feature fusion mechanism of the YOLO model;

[0021] According to the determined multi-scale feature fusion mechanism of the YOLO model, the feature pyramid network of the YOLO model is modified;

[0022] Adjust the number, size, and stride parameters of the convolutional layers in the YOLO model;

[0023] The input of the YOLO model is defined as sensor-level fusion features and feature-level fusion features, and the output is the location, size, category, and state information of the target.

[0024] As a further optional solution of the multi-target detection method for complex traffic scenes, the analysis of the diversity of target scales in complex traffic target scenes and determination of the multi-scale feature fusion mechanism of the YOLO model specifically include:

[0025] Collect data sets of complex traffic target scenes;

[0026] Annotate the collected data sets, marking the category, location and scale information of each target;

[0027] Perform statistical analysis on the annotated data set and calculate the scale distribution of different categories of targets;

[0028] According to the scale analysis results, determine the scale range that the YOLO model needs to handle and the degree of dependence of the target detection task on multi-scale features;

[0029] According to the scale range and the degree of dependence, a multi-scale feature fusion mechanism of the YOLO model is designed, and the multi-scale feature fusion mechanism includes a feature pyramid network, and a level-by-level or cross-layer fusion method.

[0030] As a further optional solution of the multi-target detection method for complex traffic scenes, the multi-target detection method further includes performing scene analysis on the traffic scene, specifically including:

[0031] Based on the sensor-level fusion features and feature-level fusion features, traffic scene information is obtained;

[0032] Based on semantic segmentation technology, the pixels in the traffic scene are segmented into different semantic categories to obtain the segmented traffic scene information;

[0033] Based on scene understanding technology, the targets, actions and relationships in the segmented traffic scene information are analyzed to obtain the scene analysis results.

[0034] As a further optional solution of the multi-target detection method for complex traffic scenes, the scene understanding technology is used to analyze the targets, actions and relationships in the segmented traffic scene information to obtain scene analysis results, which specifically include:

[0035] In the segmented traffic scene information, identify each independent target object;

[0036] Analyze the motion state of each target, the interactive actions between targets, and the motion trajectory of the targets to obtain the motion analysis results;

[0037] Analyze the spatial relationship between objects in the scene, traffic rules compliance, static spatial relationship and dynamic time dependency between objects to obtain relationship understanding results;

[0038] Integrate the target recognition, action analysis and relationship understanding results to obtain the scene analysis results.

[0039] A multi-target detection device for complex traffic scenes, comprising:

[0040] A collection module, used to collect raw data in traffic scenes through multimodal sensors deployed in the traffic environment;

[0041] The fusion module is used to fuse the raw data in the traffic scene collected by the multimodal sensors to obtain sensor-level fusion features and feature-level fusion features;

[0042] A construction module for constructing a target detection and tracking algorithm, wherein the target detection and tracking algorithm is used to process and track multiple targets in a complex traffic scene;

[0043] The analysis module is used to analyze the sensor-level fusion features and the feature-level fusion features according to the target detection and tracking algorithm to obtain multi-target detection and tracking results.

[0044] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned multi-target detection method for complex traffic scenes when executing the computer program.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-target detection method for complex traffic scenes.

[0046] The beneficial effects of the present invention are as follows: raw data in traffic scenes are collected by deploying a variety of sensors in traffic environments. These sensors can capture information about traffic scenes from different angles and dimensions, providing a rich data basis for subsequent data fusion and target detection. At the same time, multimodal data fusion can make full use of the advantages of different sensors, make up for the shortcomings of a single sensor, improve the accuracy and robustness of target detection, and construct an algorithm specifically for processing multi-target detection and tracking in complex traffic scenes, thereby achieving stable tracking and detection of multiple targets, and improving the efficiency of traffic monitoring and management. By analyzing and processing the fused features, multiple targets in traffic scenes can be accurately detected, and their dynamic changes can be tracked in real time, providing accurate decision support for autonomous driving and intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a step flow of a multi-target detection method for complex traffic scenes provided by an embodiment of the present invention;

[0049] Figure 2A schematic diagram of the composition of a multi-target detection device for complex traffic scenes provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] refer to Figure 1 to Figure 2 , a multi-target detection method for complex traffic scenes, specifically including:

[0052] Collect raw data in traffic scenes through multimodal sensors deployed in traffic environments;

[0053] The raw data in the traffic scene collected by multimodal sensors are fused to obtain sensor-level fusion features and feature-level fusion features;

[0054] Constructing a target detection and tracking algorithm, wherein the target detection and tracking algorithm is used to process and track multiple targets in complex traffic scenarios;

[0055] According to the target detection and tracking algorithm, the sensor-level fusion features and the feature-level fusion features are analyzed to obtain multi-target detection and tracking results.

[0056] In this embodiment, raw data in traffic scenes are collected by deploying a variety of sensors (such as cameras, radars, lidars, etc.) in the traffic environment. These sensors can capture information about traffic scenes from different angles and dimensions, providing a rich data basis for subsequent data fusion and target detection. At the same time, multimodal data fusion can make full use of the advantages of different sensors, make up for the shortcomings of a single sensor, improve the accuracy and robustness of target detection, and construct an algorithm specifically for processing multi-target detection and tracking in complex traffic scenes, thereby achieving stable tracking and detection of multiple targets, and improving the efficiency of traffic monitoring and management. By analyzing and processing the fused features, multiple targets in traffic scenes can be accurately detected, and their dynamic changes can be tracked in real time, providing accurate decision support for autonomous driving and intelligent transportation systems.

[0057] Preferably, the raw data in the traffic scene collected by the multimodal sensor is fused to obtain sensor-level fusion features and feature-level fusion features, specifically including:

[0058] Preprocessing the raw data in the traffic scene collected by the multimodal sensor to obtain preprocessed data;

[0059] Align and integrate the preprocessed data in time and space to obtain sensor-level fusion features;

[0060] Extract feature information from preprocessed data;

[0061] The feature information is integrated according to the feature fusion algorithm to obtain feature-level fusion features.

[0062] In this embodiment, by preprocessing the raw data, such as denoising, correction, time synchronization and other operations, the quality of the data can be effectively improved, laying a solid foundation for subsequent feature extraction and fusion. At the same time, the alignment in time and space ensures that the data from different sensors can accurately correspond to the same time and position in the same traffic scene, providing a prerequisite for subsequent feature fusion; sensor-level fusion is fusion at the data level, which directly utilizes the complementarity of different sensors in physical properties. For example, cameras can provide rich color and texture information, while radars and lidars are good at capturing distance and speed information. This fusion method can generate a more comprehensive and accurate scene representation, providing strong support for subsequent target detection and tracking tasks. In addition, sensor-level fusion can also alleviate the impact of single sensor failure or data loss to a certain extent, and improve the robustness of the system; feature extraction is to abstract information useful for target detection and tracking from preprocessed data, such as edges, textures, shapes, etc., and feature-level fusion is to integrate these feature information from different sensors through specific algorithms to generate richer feature representations. This fusion method can not only further improve the quality of features, It can also reveal hidden targets and relationships in the scene through the interaction and complementarity between features. Therefore, feature-level fusion can significantly improve the accuracy, robustness and real-time performance of target detection and tracking. Through the fusion of multimodal data, it can more comprehensively capture target information in traffic scenes and reduce missed detections and false detections due to the limitations of a single sensor. Multimodal fusion can alleviate the impact of environmental changes (such as lighting, weather, etc.) on detection performance to a certain extent, and improve the stability and adaptability of the system in different scenarios. Although multimodal data fusion increases the complexity of data processing, through efficient fusion algorithms and hardware acceleration technology, real-time or near-real-time target detection and tracking can be achieved to meet the needs of practical applications.

[0063] It should be noted that the data after alignment and synchronization at the sensor level can be fused using methods such as weighted averaging, Kalman filtering, particle filtering, etc. to obtain more comprehensive spatial information and more accurate motion state estimation; feature level fusion can use splicing, superposition, weighted sum, etc. to integrate feature information from different sensors to provide a richer and more comprehensive scene description.

[0064] Preferably, the construction of the target detection and tracking algorithm specifically includes:

[0065] Based on the YOLO model, the model structure of the YOLO model is optimized to obtain an optimized YOLO model;

[0066] The optimized YOLO model is trained to obtain a target detection model, wherein the input of the target detection model is sensor-level fusion features and feature-level fusion features, and the output is the position, size, category and state information of the target.

[0067] In this embodiment, by optimizing the structure based on the YOLO model, the detection accuracy and real-time performance of the model can be effectively improved. The YOLO model has been widely used in the field of target detection due to its high speed and high precision. However, facing complex traffic target scenes, the traditional YOLO model may not perform well in specific environments. Optimizing the model structure, such as adding or reducing convolutional layers, adjusting the filter size, introducing residual connections, etc., can better adapt to complex and changeable traffic scenes and improve the accuracy and robustness of target detection. The optimized YOLO model uses sensor-level fusion features and feature-level fusion features as input, which can make full use of multimodal sensors. The complementary advantages between data, this multimodal fusion technology can significantly improve the accuracy of target detection, because different types of sensors (such as cameras, radars, and lidars) have their own advantages and disadvantages in different scenarios, and the fusion of their data can provide more comprehensive and accurate information. At the same time, feature-level fusion further extracts and integrates the key information in these data, making the model more accurate and reliable when detecting targets; the trained target detection model can not only output basic information such as the location, size, and category of the target, but also output the target's state information (such as movement direction, speed, etc.). This comprehensive information output is crucial for subsequent target tracking and traffic management. The close combination of target detection and tracking algorithms can achieve continuous monitoring and trajectory prediction of targets in traffic scenes, providing important data support for autonomous driving and intelligent transportation systems; this technical solution improves the adaptability of target detection algorithms in complex traffic environments by improving the model structure and introducing multimodal fusion technology. Regardless of challenging factors such as lighting changes, occlusion, and weather conditions, the optimized model can maintain high detection accuracy and stability through its strong robustness and adaptability. This adaptability is of great significance to improving the safety and efficiency of the entire transportation system.

[0068] Preferably, the optimization of the model structure of the YOLO model specifically includes:

[0069] Analyze the diversity of target scales in complex traffic scenes and determine the multi-scale feature fusion mechanism of the YOLO model;

[0070] According to the determined multi-scale feature fusion mechanism of the YOLO model, the feature pyramid network of the YOLO model is modified;

[0071] Adjust the number, size, and stride parameters of the convolutional layers in the YOLO model;

[0072] The input of the YOLO model is defined as sensor-level fusion features and feature-level fusion features, and the output is the location, size, category, and state information of the target.

[0073] In this embodiment, in a complex traffic target scene, the scales of targets (such as vehicles, pedestrians, bicycles, etc.) vary greatly. By analyzing the diversity of target scales, it can be determined that the YOLO model needs to introduce a multi-scale feature fusion mechanism to better capture target features of different scales. According to the determined multi-scale feature fusion mechanism, the feature pyramid network in the YOLO model is modified. YOLO is a neural network that uses a multi-layered object detection algorithm to detect objects of different scales. It combines high-level semantic information with low-level detail information through top-down paths and lateral connections, thereby enhancing the model's ability to detect objects of different scales. The feature extraction capability of the model can be further optimized by adjusting the number, size, and step size parameters of the convolutional layers in the YOLO model. The adjustment of these parameters can be verified based on experiments, and the optimal parameter combination can be selected by comparing the model performance under different configurations. The input of the YOLO model is defined as sensor-level fusion features and feature-level fusion features, which means that the model will receive raw data from multiple sensors (such as cameras, radars, lidars, etc.) and integrate these data into more comprehensive perception information through sensor fusion technology. The output of the model is the location, size, category, and state information of the target, which is crucial for subsequent tasks such as target tracking, recognition, and scene understanding.

[0074] It should be noted that by introducing the multi-scale feature fusion mechanism, the optimized YOLO model can better capture and detect targets of different scales in complex traffic target scenes, and improve the accuracy and robustness of detection; adjusting the convolutional layer parameters can further optimize the feature extraction ability of the model, so that the model can more accurately extract useful feature information from the input data, and provide strong support for subsequent target detection tasks; the input of sensor-level and feature-level fusion features enables the model to receive more comprehensive perception information, thereby improving the accuracy of target detection. At the same time, by optimizing the model structure and parameters, the real-time performance of the model can be improved while ensuring accuracy, meeting the real-time detection needs in complex traffic target scenes.

[0075] Preferably, analyzing the diversity of target scales in complex traffic target scenes and determining the multi-scale feature fusion mechanism of the YOLO model specifically includes:

[0076] Collect data sets of complex traffic target scenes;

[0077] Annotate the collected data sets, marking the category, location and scale information of each target;

[0078] Perform statistical analysis on the labeled data set and calculate the scale distribution of different categories of targets;

[0079] According to the scale analysis results, determine the scale range that the YOLO model needs to handle and the degree of dependence of the target detection task on multi-scale features;

[0080] According to the scale range and the degree of dependence, a multi-scale feature fusion mechanism of the YOLO model is designed, and the multi-scale feature fusion mechanism includes a feature pyramid network, and a level-by-level or cross-layer fusion method.

[0081] In this embodiment, by collecting and annotating data sets of complex traffic target scenes, the scale distribution of targets of different categories can be fully understood, which helps to identify and process targets of various scales that are widely present in traffic scenes, such as vehicles in the distance, pedestrians and bicycles in the vicinity, etc.; the statistical analysis results of the scale distribution directly guide the design of the multi-scale feature fusion mechanism of the YOLO model, which ensures that the model can be more flexible and effective in processing targets of different scales, and improves the adaptability of the model to targets of different scales; the designed multi-scale feature fusion mechanism, such as the feature pyramid network and the step-by-step or cross-layer fusion method, can make full use of feature information of different scales, which enables the model to more accurately identify targets during detection, reduces the possibility of missed detection and false detection, and thus improves the accuracy of detection; through reasonable The design of a multi-scale feature fusion mechanism can optimize the allocation and utilization of computing resources while ensuring detection accuracy. For example, it can reduce redundant calculations through cross-layer fusion, or achieve efficient multi-scale feature extraction through a feature pyramid network, thereby improving the overall algorithm's operating efficiency. The optimized processing of targets of different scales enables the YOLO model to better cope with challenges such as lighting changes, occlusions, and deformations when facing complex and changeable traffic scenes. This enhanced robustness ensures the stability and reliability of the model in practical applications. The optimized YOLO model can reduce unnecessary computing overhead while ensuring detection accuracy through a multi-scale feature fusion mechanism, thereby meeting the real-time requirements in the detection of complex target scenes in traffic, which is particularly important for fields such as autonomous driving and intelligent traffic management.

[0082] It should be noted that a variety of data sets of complex traffic target scenes are collected. These data sets should contain images and videos of traffic scenes under different weather, lighting, viewing angles, and occlusion conditions; the collected data sets are accurately annotated to mark the category (such as vehicles, pedestrians, bicycles, etc.), precise location (usually represented by a bounding box), and scale information (such as the width and height of the target) of each target; statistical analysis is performed on the annotated data sets, focusing on the scale distribution characteristics of targets of different categories, which helps to understand the diversity and differences of target scales in complex traffic scenes; according to the scale analysis results, the scale range that the YOLO model needs to handle in the target detection task is determined, and at the same time, the degree of dependence of the task on multi-scale features is evaluated, that is, whether it is necessary to capture the features of small targets and large targets at the same time; based on the scale range and degree of dependence analyzed above, a multi-scale feature fusion mechanism of the YOLO model is designed, which usually includes constructing a feature pyramid network (FPN) that can generate multi-scale feature maps at different levels, and then, the feature maps of different scales are fused by level-by-level fusion or cross-layer fusion to enhance the model's detection ability for targets of different scales.

[0083] Preferably, the multi-target detection method further includes performing scene analysis on the traffic scene, specifically including:

[0084] Based on the sensor-level fusion features and feature-level fusion features, traffic scene information is obtained;

[0085] Based on semantic segmentation technology, the pixels in the traffic scene are segmented into different semantic categories to obtain the segmented traffic scene information;

[0086] Based on scene understanding technology, the targets, actions and relationships in the segmented traffic scene information are analyzed to obtain the scene analysis results.

[0087] In this embodiment, by fusing sensor-level and feature-level information, the advantages of different sensors can be comprehensively utilized to obtain more comprehensive and accurate traffic scene information, which helps to reduce the perception blind spots that may exist in a single sensor in a specific environment and improve the accuracy of overall perception; semantic segmentation technology is used to subdivide pixels in the traffic scene into different semantic categories (such as roads, vehicles, pedestrians, etc.), which not only provides accurate structural information of the scene, but also lays a solid foundation for subsequent scene understanding. This refined segmentation result can more accurately reflect the actual situation of the traffic scene; based on the scene understanding technology, the segmented traffic scene is deeply analyzed, including identifying the targets, actions, relationships, etc. in the scene, which can provide more abundant decision support information for the automatic driving system and the intelligent traffic management system. This information helps the system make more accurate judgments, thereby improving the safety and efficiency of the traffic system; scene analysis technology usually needs to be carried out under high real-time requirements to ensure that the traffic system can respond to complex and changing traffic environments in a timely manner. At the same time, by optimizing algorithms and hardware acceleration, the robustness of the system can be improved while ensuring real-time performance, so that it can adapt to traffic scenes under different environmental conditions.

[0088] It should be noted that semantic segmentation is an important technology in machine vision. It assigns each pixel in the image to a specific category label to achieve a detailed understanding of the scene. In traffic scenes, this means distinguishing different objects or areas such as roads, vehicles, pedestrians, and buildings, providing a basis for subsequent scene analysis and understanding.

[0089] Preferably, the scene understanding technology is used to analyze the targets, actions and relationships in the segmented traffic scene information to obtain scene analysis results, which specifically include:

[0090] In the segmented traffic scene information, identify each independent target object;

[0091] Analyze the motion state of each target, the interactive actions between targets, and the motion trajectory of the targets to obtain the motion analysis results;

[0092] Analyze the spatial relationship between objects in the scene, traffic rules compliance, static spatial relationship and dynamic time dependency between objects to obtain relationship understanding results;

[0093] Integrate the target recognition, action analysis and relationship understanding results to obtain the scene analysis results.

[0094] In this embodiment, in the segmented traffic scene information, firstly, each independent target object, such as vehicles, pedestrians, bicycles, etc., is identified, which depends on the semantic label assigned to each pixel in the previous semantic segmentation step, and the target detection algorithm (such as a detector based on deep learning) is used to further confirm and locate these targets, and obtain their precise positions and bounding boxes; the motion state of each target is analyzed, including dynamic information such as speed, direction, acceleration, etc., which requires combining the target tracking results between consecutive frames, identifying the interactive actions between targets, such as the turning of vehicles, the walking direction of pedestrians, the overtaking or avoidance of vehicles, etc., and using optical flow method, trajectory analysis and other technologies to track the motion trajectory of the target, so as to more accurately analyze the action of the target; analyzing the spatial relationship between the targets in the scene, such as distance, relative position, etc., which helps to understand the potential interaction and mutual influence between the targets, identify the compliance with traffic rules in the scene, such as whether the vehicle obeys the traffic lights, whether the pedestrians walk on the crosswalk, etc., and use advanced technologies such as graph neural network (GNN) to model the complex relationship between targets, including static spatial relationship and dynamic time dependency; the results of target recognition, action analysis and relationship understanding are combined to form an overall understanding of the traffic scene.

[0095] A multi-target detection device for complex traffic scenes, comprising:

[0096] A collection module, used to collect raw data in traffic scenes through multimodal sensors deployed in the traffic environment;

[0097] The fusion module is used to fuse the raw data in the traffic scene collected by the multimodal sensors to obtain sensor-level fusion features and feature-level fusion features;

[0098] A construction module for constructing a target detection and tracking algorithm, wherein the target detection and tracking algorithm is used to process and track multiple targets in a complex traffic scene;

[0099] The analysis module is used to analyze the sensor-level fusion features and the feature-level fusion features according to the target detection and tracking algorithm to obtain multi-target detection and tracking results.

[0100] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned multi-target detection method for complex traffic scenes when executing the computer program.

[0101] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-target detection method for complex traffic scenes.

[0102] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-target detection method for complex traffic scenes, characterized in that: Specifically include: Collect raw data in traffic scenes through multimodal sensors deployed in traffic environments; The raw data in the traffic scene collected by multimodal sensors are fused to obtain sensor-level fusion features and feature-level fusion features; Constructing a target detection and tracking algorithm, wherein the target detection and tracking algorithm is used to process and track multiple targets in complex traffic scenarios; According to the target detection and tracking algorithm, the sensor-level fusion features and the feature-level fusion features are analyzed to obtain multi-target detection and tracking results.

2. A multi-target detection method for complex traffic scenes according to claim 1, characterized in that: The raw data in the traffic scene collected by the multimodal sensor is fused to obtain sensor-level fusion features and feature-level fusion features, which specifically include: Preprocessing the raw data in the traffic scene collected by the multimodal sensor to obtain preprocessed data; Align and integrate the preprocessed data in time and space to obtain sensor-level fusion features; Extract feature information from preprocessed data; The feature information is integrated according to the feature fusion algorithm to obtain feature-level fusion features.

3. The multi-target detection method for complex traffic scenes according to claim 2, characterized in that: The construction of the target detection and tracking algorithm specifically includes: Based on the YOLO model, the model structure of the YOLO model is optimized to obtain an optimized YOLO model; The optimized YOLO model is trained to obtain a target detection model, wherein the input of the target detection model is sensor-level fusion features and feature-level fusion features, and the output is the position, size, category and state information of the target.

4. The multi-target detection method for complex traffic scenes according to claim 3, characterized in that: The optimization of the model structure of the YOLO model specifically includes: Analyze the diversity of target scales in complex traffic scenes and determine the multi-scale feature fusion mechanism of the YOLO model; According to the determined multi-scale feature fusion mechanism of the YOLO model, the feature pyramid network of the YOLO model is modified; Adjust the number, size, and stride parameters of the convolutional layers in the YOLO model; The input of the YOLO model is defined as sensor-level fusion features and feature-level fusion features, and the output is the location, size, category, and state information of the target.

5. The multi-target detection method for complex traffic scenes according to claim 4, characterized in that: The analysis of the diversity of target scales in complex traffic target scenes determines the multi-scale feature fusion mechanism of the YOLO model, specifically including: Collect data sets of complex traffic target scenes; Annotate the collected data sets, marking the category, location and scale information of each target; Perform statistical analysis on the labeled data set and calculate the scale distribution of different categories of targets; According to the scale analysis results, determine the scale range that the YOLO model needs to handle and the degree of dependence of the target detection task on multi-scale features; According to the scale range and the degree of dependence, a multi-scale feature fusion mechanism of the YOLO model is designed, and the multi-scale feature fusion mechanism includes a feature pyramid network, and a level-by-level or cross-layer fusion method.

6. A multi-target detection method for complex traffic scenes according to claim 5, characterized in that: The multi-target detection method further includes performing scene analysis on the traffic scene, specifically including: Based on the sensor-level fusion features and feature-level fusion features, traffic scene information is obtained; Based on semantic segmentation technology, the pixels in the traffic scene are segmented into different semantic categories to obtain the segmented traffic scene information; Based on scene understanding technology, the targets, actions and relationships in the segmented traffic scene information are analyzed to obtain the scene analysis results.

7. A multi-target detection method for complex traffic scenes according to claim 6, characterized in that: The scene understanding technology is used to analyze the targets, actions and relationships in the segmented traffic scene information to obtain scene analysis results, which specifically include: In the segmented traffic scene information, identify each independent target object; Analyze the motion state of each target, the interactive actions between targets, and the motion trajectory of the targets to obtain the motion analysis results; Analyze the spatial relationship between objects in the scene, traffic rules compliance, static spatial relationship and dynamic time dependency between objects to obtain relationship understanding results; Integrate the target recognition, action analysis and relationship understanding results to obtain the scene analysis results.

8. A multi-target detection device for complex traffic scenes, characterized in that: include: A collection module, used to collect raw data in traffic scenes through multimodal sensors deployed in the traffic environment; The fusion module is used to fuse the raw data in the traffic scene collected by the multimodal sensors to obtain sensor-level fusion features and feature-level fusion features; A construction module for constructing a target detection and tracking algorithm, wherein the target detection and tracking algorithm is used to process and track multiple targets in a complex traffic scene; The analysis module is used to analyze the sensor-level fusion features and the feature-level fusion features according to the target detection and tracking algorithm to obtain multi-target detection and tracking results.

9. A computing device, characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the multi-target detection method for complex traffic scenes described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-target detection method for complex traffic scenes described in any one of claims 1 to 7.

Images