Vehicle traffic light sensing abnormity detection method and system and vehicle
By real-time environmental image recognition and tracking obstacles, combining road information and vehicle driving information, using machine learning models to detect traffic light perception abnormalities, solving the shortcomings of the autonomous driving system in identifying abnormal driving scenarios and improving safety and reliability.
Patent Information
- Application Number
- CN202411070276.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2025-01-03
AI Technical Summary
The autonomous driving algorithm has shortcomings in identifying abnormal driving scenarios, especially traffic light perception, which affects safety.
By acquiring real-time environmental images, identifying and tracking obstacles, combining road information and vehicle driving information, using machine learning and deep learning models to detect traffic light perception anomalies.
Real-time detection of traffic light perception abnormalities is achieved, improving the safety and reliability of the autonomous driving system, and reducing development and testing costs.
Smart Images
Figure CN119251794A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to a method, system and vehicle for detecting abnormalities in vehicle traffic light perception. Background Art
[0002] Autonomous driving technology is undergoing a transformation from algorithm-driven to data-driven, in which big data and intelligent models play a key role. Sensor data not only promotes the continuous optimization of perception modules, but also enables the identification of new unknown objects. At the same time, human driving behavior data is crucial for refining the commonalities and characteristics of driving strategies. These data support the decision-making planning module to generate vehicle movements that are closer to human driving habits.
[0003] In the process of developing and OTA upgrading autonomous driving algorithms, real-world driving data is crucial to algorithm iteration. Since most data samples reflect normal driving scenarios, it is difficult to identify abnormal driving scenarios, which affects the safety of autonomous driving.
[0004] In view of the above problems, those skilled in the art have been seeking solutions. Summary of the invention
[0005] The technical problem to be solved by the present application is to provide a method, system and vehicle for detecting abnormalities in vehicle traffic light perception in view of the defects of the above-mentioned prior art.
[0006] In order to achieve the above objectives, this application is implemented through the following technical solutions:
[0007] The present application provides a method for detecting abnormal vehicle traffic light perception, comprising:
[0008] S1, obtaining a real-time environmental image around the vehicle;
[0009] S2, identifying and tracking objects around the vehicle according to the real-time environment image to obtain motion data of obstacles;
[0010] S3, acquiring road information and vehicle driving information according to the real-time environment image;
[0011] S4. Determine vehicle traffic light perception abnormality based on the movement data of the obstacle, the road information and the vehicle driving information.
[0012] Optionally, the road information includes traffic light positions, traffic light display colors, traffic light display numbers, lane lines, and traffic signs;
[0013] The vehicle driving information includes vehicle speed, steering angle, lane, and forward direction.
[0014] Optionally, step S2 includes:
[0015] Preprocessing the real-time environment image;
[0016] Performing feature extraction on the preprocessed real-time environment image to obtain feature information;
[0017] The feature information is analyzed using a machine learning and / or deep learning model to obtain an obstacle identification result.
[0018] Optionally, step S2 further includes:
[0019] The obstacle recognition result is subjected to motion tracking to obtain motion data of the obstacle.
[0020] Optionally, step S4 includes:
[0021] Comparing the motion data of the obstacle at a preset number of frames to obtain the road traffic conditions;
[0022] Determine vehicle traffic light perception abnormality based on the road traffic condition, the road information and the vehicle driving information.
[0023] The present application also provides a detection system based on the above method, comprising: a sensor module, a data processing module, and a communication module;
[0024] The sensor module is used to obtain real-time environmental images around the vehicle and obtain vehicle driving information;
[0025] The data processing module is used to identify and track objects around the vehicle according to the real-time environment image to obtain motion data of obstacles, obtain road information according to the real-time environment image, and determine the abnormality of vehicle traffic light perception according to the motion data of obstacles, the road information and the vehicle driving information;
[0026] The communication module is used to realize information communication between the sensor module and the data processing module.
[0027] Optionally, the data processing module is also used to preprocess the real-time environment image, perform feature extraction on the preprocessed real-time environment image to obtain feature information, and analyze the feature information using machine learning and / or deep learning models to obtain obstacle recognition results.
[0028] Optionally, the data processing module is further used to perform motion tracking on the obstacle identification result to obtain motion data of the obstacle.
[0029] Optionally, the data processing module is also used to compare the motion data of the obstacle of a preset number of frames to obtain road traffic conditions, and determine vehicle traffic light perception abnormalities based on the road traffic conditions, the road information and the vehicle driving information.
[0030] The present application also provides a vehicle, comprising the system as described above.
[0031] The present application provides a method, system and vehicle for detecting vehicle traffic light perception anomalies, which can avoid abnormal scenarios by detecting and mining traffic light perception anomalies in real time, and improve the safety and reliability of the autonomous driving system.
[0032] In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present application is described in detail below with reference to the accompanying drawings and specific implementation methods.
[0034] Figure 1 It is a flow chart of a detection method provided in one embodiment of the present application;
[0035] Figure 2 It is a schematic diagram of the structure of a detection system provided in one embodiment of the present application. DETAILED DESCRIPTION
[0036] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0037] Figure 1 This is a flow chart of the detection method provided in one embodiment of the present application. Figure 1 The present application provides a method for detecting abnormal perception of vehicle traffic lights, comprising:
[0038] S1. Acquire real-time environment images around the vehicle.
[0039] In an autonomous driving system, image acquisition is the first step in obtaining environmental information. Onboard cameras and sensors are used to continuously capture high-resolution image data that can reflect the dynamic environment around the vehicle. To ensure the quality of the data, multiple cameras are usually used to cover different perspectives, including front view, side view, and rear view.
[0040] S2. Identify and track objects around the vehicle based on real-time environmental images to obtain motion data of obstacles.
[0041] Optionally, step S2 includes:
[0042] Preprocess the real-time environment images;
[0043] Perform feature extraction on the preprocessed real-time environment image to obtain feature information;
[0044] The feature information is analyzed using machine learning and / or deep learning models to obtain obstacle identification results.
[0045] In this embodiment, the obstacles may be living things or non-living things with movement ability.
[0046] Image preprocessing is a key step to improve the accuracy of obstacle detection. In addition to basic denoising and enhancement, preprocessing may also include contrast adjustment, color correction, edge enhancement and other steps. These operations help highlight the outline and features of obstacles, laying a solid foundation for subsequent feature extraction and recognition.
[0047] The feature extraction stage aims to identify key information related to obstacle recognition from the preprocessed image. These features may include edges, corners, textures, contours, etc. Through advanced image processing techniques such as filters and conversion algorithms, the features of obstacles can be effectively extracted from complex backgrounds.
[0048] During the obstacle recognition phase, the system uses machine learning or deep learning models to analyze the extracted features. These models are trained with a large amount of data and can accurately classify and identify different types of obstacles. In addition, the algorithm also evaluates the location and movement trajectory of the obstacle, providing an important basis for obstacle avoidance decisions.
[0049] Optionally, step S2 further includes:
[0050] The obstacle recognition results are motion tracked to obtain the motion data of the obstacle.
[0051] After obstacle identification, the post-processing step optimizes and confirms the identification results. This includes eliminating false positives, merging duplicate detections, and tracking the position changes of obstacles. In addition, post-processing also includes using historical data and time series analysis to predict the future state of obstacles, thereby providing the autonomous driving system with more accurate obstacle avoidance decision-making basis.
[0052] After neural network detection, the online detection module receives data from the camera sensor to obtain the type, position, speed and direction of each object, and broadcasts the identified traffic scene element information through the topic communication mechanism in Neta OS FrameWork.
[0053] S3. Acquire road information and vehicle driving information based on the real-time environment image.
[0054] Optionally, the road information includes the location of traffic lights, the color displayed by traffic lights, the numbers displayed by traffic lights, lane lines, and traffic signs; the vehicle driving information includes the vehicle speed, steering angle, lane, and direction of travel.
[0055] Combining the perception and recognition results, IMU inertial navigation and vehicle chassis information, we monitor the corresponding topic channels to obtain traffic lights, lane lines, traffic signs and other elements, and then transform the coordinate system of each target from the camera coordinate system to the vehicle body coordinate system.
[0056] S4. Determine the abnormality of vehicle traffic light perception based on the movement data of the obstacle, the road information and the vehicle driving information.
[0057] Optionally, step S4 includes:
[0058] Compare the motion data of obstacles at a preset number of frames to obtain the road traffic conditions;
[0059] Determine vehicle traffic light perception abnormalities based on road conditions, road information and vehicle driving information.
[0060] In this step, the obstacle is a vehicle located in the lane. The traffic conditions of the traffic lights in each lane are determined based on the vehicle motion data of several frames before and after, for example, 10 frames. Then, the current driving state of the vehicle is determined by combining the vehicle speed, steering angle, turn signal and other information. For example, the vehicle is in an accelerating, decelerating or stopping state. The traffic information of the traffic lights at the current moment is compared to find abnormal scenes. In another embodiment, the obstacle can also be a pedestrian at a traffic light intersection. The traffic conditions of the traffic lights at this intersection are determined based on the pedestrian motion data of several frames before and after, for example, 10 frames. Then, the current driving state of the vehicle is determined based on the driving information of the vehicle. The traffic conditions of the traffic lights and the current driving state of the vehicle are combined to determine whether the vehicle is in a traffic light perception abnormal scene.
[0061] In summary, mining abnormal traffic light scenarios requires message communication, perception results, and chassis information. Through the above technologies and information, it is possible to mine abnormal traffic light perception scenarios during normal vehicle driving, thereby preventing vehicle accidents.
[0062] In another embodiment, in order to effectively mine and identify abnormal scenarios in autonomous driving, a reinforcement learning method can be used to improve the system's ability to respond to abnormal scenarios through continuous exploration and learning.
[0063] Figure 2 is a schematic diagram of the structure of a detection system provided in an embodiment of the present application, see Figure 2 The present application also provides a detection system based on the above method, including: a sensor module, a data processing module, and a communication module.
[0064] The sensor module is used to obtain real-time environmental images around the vehicle and obtain vehicle driving information. In one embodiment, the sensor module includes sensors such as vehicle chassis, camera, Li DAR, I MU inertial navigation, etc., and assists in detecting abnormal traffic light perception by fusing information obtained by different sensors, with good accuracy and reliability.
[0065] The data processing module is used to identify and track objects around the vehicle based on real-time environmental images to obtain motion data of obstacles, obtain road information based on real-time environmental images, and determine vehicle traffic light perception abnormalities based on the motion data of obstacles, road information and vehicle driving information.
[0066] The communication module is used to realize information communication between the sensor module and the data processing module.
[0067] In one embodiment, the communication module is the Neta OS FrameWork framework, which is a runtime framework designed specifically for autonomous driving scenarios. It is based on a centralized computing model and has the advantages of low latency and high throughput. A decentralized service discovery mechanism is performed through broadcasting and other means. In the Neta OS FrameWork framework, message communication can be completed through nodemanager (node manager) and wr itemanager (write manager).
[0068] Optionally, the data processing module is also used to preprocess the real-time environment image, perform feature extraction on the preprocessed real-time environment image to obtain feature information, and analyze the feature information using a machine learning and / or deep learning model to obtain an obstacle recognition result.
[0069] Optionally, the data processing module is further used to perform motion tracking on the obstacle recognition result to obtain motion data of the obstacle.
[0070] Optionally, the data processing module is also used to compare the motion data of obstacles of a preset number of frames to obtain road traffic conditions, and determine vehicle traffic light perception abnormalities based on road traffic conditions, road information and vehicle driving information.
[0071] The present application also provides a vehicle, comprising the system as described above.
[0072] The solution provided in this application has the following advantages:
[0073] 1. Improve the efficiency of abnormal scene data acquisition: In traditional data collection, abnormal scene data is difficult to collect due to low probability. The present invention improves the efficiency of abnormal scene data acquisition through real-time detection and mining, ensuring that the algorithm can cover various driving situations more comprehensively.
[0074] 2. Real-time anomaly detection and response: By utilizing the efficient Neta OS FrameWok framework and real-time message communication mechanism, the present invention can detect and respond to abnormal scenarios in real time during vehicle driving, thereby improving the safety and reliability of the autonomous driving system.
[0075] 3. Reduce development and testing costs: Through effective data mining and abnormal scenario detection, the present invention can reduce the dependence on actual abnormal scenarios during the development and testing process, reduce related costs and accelerate technology iteration.
[0076] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these changes and variations. The technical features of the above-described embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0077] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for detecting abnormality in vehicle traffic light perception, characterized in that: include: S1, obtaining a real-time environmental image around the vehicle; S2, identifying and tracking objects around the vehicle according to the real-time environment image to obtain motion data of obstacles; S3, acquiring road information and vehicle driving information according to the real-time environment image; S4. Determine vehicle traffic light perception abnormality based on the movement data of the obstacle, the road information and the vehicle driving information.
2. The method according to claim 1, characterized in that The road information includes the location of traffic lights, the color displayed by traffic lights, the number displayed by traffic lights, lane lines, and traffic signs; The vehicle driving information includes vehicle speed, steering angle, lane, and forward direction.
3. The method according to claim 2, characterized in that Step S2 includes: Preprocessing the real-time environment image; Performing feature extraction on the preprocessed real-time environment image to obtain feature information; The feature information is analyzed using a machine learning and / or deep learning model to obtain an obstacle identification result.
4. The method according to claim 3, characterized in that Step S2 also includes: The obstacle recognition result is subjected to motion tracking to obtain motion data of the obstacle.
5. The method according to claim 4, characterized in that Step S4 includes: Comparing the motion data of the obstacle at a preset number of frames to obtain the road traffic conditions; Determine vehicle traffic light perception abnormality based on the road traffic condition, the road information and the vehicle driving information.
6. A detection system based on the method according to any one of claims 1 to 5, characterized in that: include: Sensor module, data processing module, communication module; The sensor module is used to obtain real-time environmental images around the vehicle and obtain vehicle driving information; The data processing module is used to identify and track objects around the vehicle according to the real-time environment image to obtain motion data of obstacles, obtain road information according to the real-time environment image, and determine the abnormality of vehicle traffic light perception according to the motion data of obstacles, the road information and the vehicle driving information; The communication module is used to realize information communication between the sensor module and the data processing module.
7. The system according to claim 6, characterized in that The data processing module is also used to preprocess the real-time environment image, perform feature extraction on the preprocessed real-time environment image to obtain feature information, and analyze the feature information using a machine learning and / or deep learning model to obtain an obstacle recognition result.
8. The system according to claim 7, characterized in that The data processing module is also used to perform motion tracking on the obstacle recognition result to obtain motion data of the obstacle.
9. The system according to claim 8, characterized in that The data processing module is also used to compare the motion data of the obstacle of a preset number of frames to obtain the road traffic condition, and determine the vehicle traffic light perception abnormality based on the road traffic condition, the road information and the vehicle driving information.
10. A vehicle, characterized in that: Comprising a system as claimed in any one of claims 6-9.