Automatic driving method and system based on infrared polarization image sensor
By combining infrared polarization image sensors and lidar sensors, using machine learning models and Kalman filters to process data, building a three-dimensional model of the surrounding environment of the vehicle, the problem of low data accuracy in the autonomous driving system is solved and more accurate Acquisition of environmental information and safe driving, timely warnings have improved the experience and safety of autonomous driving.
Patent Information
- Application Number
- CN202410914579.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When data collection and data collection and integration of sensors in the existing autonomous driving system, the data accuracy is not ideal due to environmental factors, which affects the sense of autonomous driving experience.
Using a combination of infrared polarization image sensors and lidar sensors, data is optimized through machine learning models, and data fusion is used to process using Kalman filters to build a three-dimensional model of the surrounding environment of the vehicle, and road structures, obstacle distribution, Information on traffic vehicles and pedestrians, update data in real time to monitor environmental changes, and plan vehicle paths and driving strategies.
It improves the accuracy of data processing and integration, achieves more accurate acquisition of vehicle surrounding environment information, ensures safe driving of the vehicle, and predicts potential risks through intelligent early warning modules to avoid accidents, and improves the accuracy and timeliness of early warning.
Smart Images

Figure CN118876993A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and more specifically, to an autonomous driving method and system based on an infrared polarization image sensor. Background Art
[0002] With the development of autonomous driving technology, safe and reliable perception systems are crucial to the realization of advanced driver assistance systems (ADAS) and autonomous driving. As an emerging perception technology, infrared polarization image sensors have the advantage of providing high-quality images in complex environments, so they have received widespread attention in autonomous driving.
[0003] The document with the prior art publication number CN109515448B provides an automatic driving sensor arrangement method and structure, which realizes the interaction between the automatic driving vehicle and the surrounding vehicles and road information, and the detection of static objects and other organisms such as pedestrians around the vehicle through sensors; provides a method for detecting the environment at the bottom of the vehicle during the vehicle starting phase to ensure the safety of the automatic driving vehicle during the starting phase; provides a method for detecting organisms with the help of infrared sensors, detecting the surrounding environment of the road with cameras, and completing the interaction between vehicles and vehicles and roads with information interaction receiving devices, and integrating the information between various systems to ensure the safe driving of vehicles on the road; provides a method for confirming the relative position and positioning between each other based on the interaction between vehicles and between vehicles and roads; provides a method for sending and receiving the status information of the current vehicle and the status information of other vehicles in real time based on information interaction equipment; ensures that vehicles do not collide with each other during driving.
[0004] Although the above-mentioned prior art solutions can achieve the relevant beneficial effects through the structure of the prior art, the accuracy of the data is not ideal due to environmental factors when the sensors collect data and fuse the collected data, which in turn affects the automatic driving system. In view of this, we propose an automatic driving method and system based on infrared polarization image sensor. Summary of the invention
[0005] 1. Technical issues to be solved
[0006] The purpose of this application is to provide an autonomous driving method and system based on an infrared polarization image sensor, which solves the technical problem in the prior art that when sensors collect data and fuse collected data, the data accuracy is not ideal due to environmental factors, which affects the autonomous driving experience, and achieves the technical effect of improving the accuracy of data processing and fusion, thereby improving the autonomous driving experience.
[0007] 2. Technical solution
[0008] The present application provides an automatic driving method based on an infrared polarization image sensor, comprising the following steps:
[0009] S1. Collecting information about the vehicle's surrounding environment;
[0010] S2. Preprocessing the collected vehicle surrounding environment information;
[0011] S3. Build a machine learning model, train the model and extract features according to task requirements;
[0012] S4, fusing the data optimized by the machine learning model through the Kalman filter;
[0013] S5. construct a three-dimensional model of the vehicle's surrounding environment based on the fused data and analyze it to extract the required vehicle's surrounding environment information;
[0014] S6. Plan the path and driving strategy of the vehicle according to the extracted vehicle surrounding environment information, and update the collected data in real time to monitor the changes of the vehicle surrounding environment.
[0015] By adopting the above technical solution, the collected data is optimized through the machine learning model, and the data optimized by the machine learning model is fused and processed by the Kalman filter. The Kalman filter filters and predicts the sensor observations, and the predicted new observations are generated to effectively smooth the sensor observations, eliminate noise and drift in the sensor observations, and improve the accuracy of the prediction, so as to obtain more accurate information about the vehicle's surrounding environment; based on the fused data, a three-dimensional model of the vehicle's surrounding environment is constructed, and through the analysis of the three-dimensional model of the vehicle's surrounding environment, environmental information such as road structure, obstacle distribution, traffic vehicles, and pedestrian movement direction and speed are extracted, and according to the environmental information, the vehicle's path and driving strategy are planned, and the collected data is updated in real time, and the changes in the surrounding environment are monitored in real time. According to the real-time data and environmental conditions, combined with the vehicle's dynamic characteristics and environmental conditions, intelligent speed control, lane keeping and obstacle avoidance operations are performed to ensure the safe driving of the vehicle.
[0016] As an optional solution of the technical solution of this application document, the vehicle surrounding environment information at least includes road structure, obstacle distribution, traffic vehicles, pedestrians, and movement direction and speed.
[0017] As an optional solution to the technical solution of this application document, the vehicle's surrounding environment information is collected through an infrared polarization image sensor and a lidar sensor.
[0018] By adopting the above technical solution, the infrared polarization image sensor is used to capture the road, vehicles, pedestrians, traffic signs and other information in the vehicle's surrounding environment in real time, and the laser radar sensor is used to obtain the distance, speed and shape of objects in the vehicle's surrounding environment.
[0019] Infrared polarization image sensors use the characteristics of infrared polarization light reflected or emitted by the surface of an object to obtain image information. Infrared polarization image sensors can sense biological and other obstacles around the vehicle. They have the advantages of high sensitivity, strong anti-interference ability, and strong night vision ability. They have higher spatial resolution than traditional infrared cameras and can effectively improve the perception ability and accuracy of infrared image capture in low-light or no-light environments. In autonomous driving systems, infrared polarization image sensors and lidar sensors are installed in important positions such as the front, side and roof of the vehicle to provide all-round coverage around the vehicle to avoid blind spots.
[0020] As an optional solution to the technical solution of this application document, the method for constructing a machine learning model is:
[0021] Automatically extract features from data through deep learning algorithms;
[0022] Divide the data into training set, validation set and test set according to the proportion. The training set is used to train the model, the validation set is used to adjust the hyperparameters, and the test set is used to finally evaluate the performance of the model.
[0023] Randomly extract data from the training set or validation set and divide it into two groups: one group is used to train the model, and the other group is used to validate the model;
[0024] Divide the data into several batches, use the first batch of data to train the model and monitor and evaluate the performance of the model through the validation set data;
[0025] Test the performance of the model using the data in the test set.
[0026] By adopting the above technical solution, a machine learning model is built based on the preprocessed data, and the model is trained through a deep learning algorithm according to task requirements such as target detection, classification and tracking, and effective image features are extracted from the data, thereby optimizing the quality of the data and the accuracy of subsequent fusion; if the performance of the model on the validation set is lower than expected, adjust the model hyperparameters and train the model again until the model meets the requirements.
[0027] As an optional solution of the technical solution of this application document, the data volume of the training set accounts for 80-90% of the total data set, the data volume of the validation set accounts for 10-20% of the total data set, and the other data sets are divided into test sets.
[0028] As an optional solution to the technical solution of this application document, during the automatic driving of the vehicle, the potential risks of the vehicle are evaluated through an intelligent early warning module.
[0029] By adopting the above technical solution, the intelligent warning module can evaluate the potential risks of the vehicle, thereby predicting possible problems and issuing warnings in advance to avoid accidents. Warnings can be sent to the driver or the autonomous driving system through sound, vision or touch, so that the driver can slow down in advance or take other avoidance measures.
[0030] As an optional solution of the technical solution of this application document, the intelligent warning module communicates with other vehicles and infrastructure.
[0031] By adopting the above technical solution, more traffic information such as the driving status of other vehicles, road conditions and traffic signals can be obtained through vehicle-to-vehicle and vehicle-to-infrastructure communications, thereby further improving the accuracy and timeliness of early warnings.
[0032] The present application also discloses an automatic driving system based on an infrared polarization image sensor, including a data acquisition module, a data processing module, a data fusion module, a monitoring decision module and a warning interaction module;
[0033] Data acquisition module, used to capture and collect information about the vehicle's surrounding environment;
[0034] Data processing module, used for processing collected data, building machine learning models and training models;
[0035] The data fusion module is used to fuse the processed data from different sensors to obtain the final distance, speed and direction information;
[0036] Monitoring and decision-making module, used for data update monitoring and planning of vehicle paths and driving strategies;
[0037] The early warning interaction module is used to evaluate and warn of potential risks of vehicles.
[0038] As an optional solution of the technical solution of this application document, the early warning interaction module is also used to communicate with other vehicles and infrastructure.
[0039] By adopting the above technical solutions, the warning interaction module communicates with other vehicles and infrastructure to achieve collaborative driving and traffic flow optimization.
[0040] As an optional solution of the technical solution of this application document, the data acquisition module collects data through an infrared polarization image sensor and a lidar sensor, and polarizers are installed in front of the mirrors of the infrared polarization image sensor and the lidar sensor.
[0041] By adopting the above technical solution, when the polarizer is combined with the infrared polarization image sensor, the direction of the incident light can be adjusted, thereby changing the intensity and quality of the infrared radiation received by the infrared polarization image sensor; when the polarizer is combined with the lidar sensor, the laser beam of the lidar sensor is irradiated to the object, and the light in the set polarization direction can pass through the lidar sensor, while the light from other directions is blocked. By detecting the polarization state of the reflected light, different objects and obstacles can be distinguished, thereby improving the accuracy and resolution of the information collected by the lidar sensor.
[0042] 3. Beneficial effects
[0043] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
[0044] 1. This application optimizes the collected data through a machine learning model. The data optimized by the machine learning model is fused and processed by a Kalman filter, which can effectively eliminate the noise and drift in the sensor observation values and improve the accuracy of the prediction, thereby achieving more accurate information about the vehicle's surrounding environment. A three-dimensional model of the vehicle's surrounding environment is constructed based on the fused data, and through the analysis of the three-dimensional model of the vehicle's surrounding environment, environmental information such as road structure, obstacle distribution, traffic vehicles, and pedestrian movement direction and speed are extracted, and the vehicle's path and driving strategy are planned based on the environmental information. At the same time, the collected data is updated in real time, thereby monitoring changes in the surrounding environment. According to real-time data and environmental conditions, combined with the vehicle's dynamic characteristics and environmental conditions, intelligent speed control, lane keeping, obstacle avoidance and other operations are performed to ensure safe driving of the vehicle.
[0045] 2. This application uses an intelligent early warning module to assess the potential risks of the vehicle, thereby predicting possible problems and issuing warnings in advance to avoid accidents.
[0046] 3. The vehicle-to-vehicle and vehicle-to-infrastructure communication connections of this application can obtain more traffic information such as the driving status of other vehicles, road conditions and traffic signals, thereby further improving the accuracy and timeliness of early warnings, and realizing collaborative driving and traffic flow optimization.
[0047] 4. The present application uses a polarizing plate setting. When the polarizing plate is combined with an infrared polarization image sensor, the direction of the incident light can be adjusted, thereby changing the intensity and quality of the infrared radiation received by the infrared polarization image sensor; when the polarizing plate is combined with a lidar sensor, the laser beam of the lidar sensor irradiates the object, and the light in the set polarization direction can pass through the lidar sensor, while the light from other directions is blocked. By detecting the polarization state of the reflected light, different objects and obstacles can be distinguished, thereby improving the accuracy and resolution of the information collected by the lidar sensor. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 This is a flow chart of an autonomous driving method based on an infrared polarization image sensor disclosed in a preferred embodiment of the present application. DETAILED DESCRIPTION
[0049] The present application is further described in detail below in conjunction with the accompanying drawings.
[0050] Reference Figure 1 The embodiment of the present application discloses an autonomous driving method based on an infrared polarization image sensor, which uses the infrared polarization image sensor to capture information such as roads, vehicles, pedestrians, traffic signs, etc. in the vehicle's surrounding environment in real time, and uses a lidar sensor to obtain information such as the distance, speed, and shape of objects in the vehicle's surrounding environment.
[0051] Infrared polarization image sensors use the characteristics of infrared polarization light reflected or emitted by the surface of an object to obtain image information. Infrared polarization image sensors can sense creatures (pedestrians, animals, etc.) and other obstacles around the vehicle. They have the advantages of high sensitivity, strong anti-interference ability, and strong night vision ability. They have higher spatial resolution than traditional infrared cameras and can effectively improve the perception ability and accuracy of infrared image capture in low-light or no-light environments. LiDAR sensors obtain information such as the distance, speed, and shape of objects in the vehicle's surrounding environment by emitting laser beams and detecting and processing the returned reflected light.
[0052] In the autonomous driving system, infrared polarization image sensors and lidar sensors are installed in important locations such as the front, sides and roof of the vehicle to provide all-round coverage around the vehicle and avoid blind spots.
[0053] In order to further improve the accuracy of information collection of the infrared polarization image sensor and the lidar sensor, polarizing plates are installed in front of the mirrors of the infrared polarization image sensor and the lidar sensor.
[0054] When the polarizer is combined with the infrared polarization image sensor, the direction of the incident light can be adjusted, thereby changing the intensity and quality of the infrared radiation received by the infrared polarization image sensor. When the polarizer is combined with the lidar sensor, the laser beam of the lidar sensor is irradiated to the object, and the light in the set polarization direction can pass through the lidar sensor, while the light in other directions is blocked. By detecting the polarization state of the reflected light, different objects and obstacles can be distinguished, thereby improving the accuracy and resolution of the information collected by the lidar sensor.
[0055] The infrared images collected by the infrared polarization image sensor are preprocessed by grayscale, denoising, and normalization, and the data obtained by the lidar sensor are preprocessed by denoising to obtain high-quality data. Grayscale is to convert the information of multiple wavelengths contained in the initially captured infrared image into a grayscale image of the same wavelength by linearly transforming the pixel values, thereby simplifying subsequent image processing and reducing the amount of data; denoising is to eliminate the influence of contrast on the infrared image during the grayscale process by filtering methods such as median filtering, mean filtering or bilateral filtering, thereby improving image quality and the accuracy of subsequent processing; normalization is to use the Min-Max normalization method to convert the image from a grayscale image to a form more suitable for computer vision processing to reduce the difference of the image, reduce recognition problems at different scales, and improve the stability and robustness of the algorithm.
[0056] The machine learning model is constructed based on the preprocessed data. The model is trained based on the task requirements such as target detection, classification and tracking through the deep learning algorithm, and effective features are extracted from the data to optimize the data quality and the accuracy of the later fusion. The method of constructing the machine learning model is:
[0057] Automatically extract features from data based on deep learning algorithms; divide data into training set, validation set and test set according to a certain ratio. The training set is used to train the model, the validation set is used to adjust hyperparameters, and the test set is used to finally evaluate the performance of the model. The data volume of the training set accounts for 80-90% of the total data set, the data volume of the validation set accounts for 10-20% of the total data set, and the rest of the data set is divided into test sets;
[0058] Randomly extract data from the training set or validation set and divide it into two groups: one for training the model and the other for validating the model; divide the data into several batches, use the data from the first batch to train the model, and monitor and evaluate the performance of the model through the data from the validation set. If the performance of the model on the validation set is lower than expected, adjust the model's hyperparameters such as neuron size, activation function, loss function, and train the model again until the model meets the requirements. Finally, test the performance of the model through the data from the test set.
[0059] The data optimized by the machine learning model is fused and processed through the Kalman filter to obtain the final distance, speed and direction information.
[0060] When the Kalman filter fuses the data after the machine learning model is optimized, it first defines the state vector (the value of the variable to be estimated, including the position and speed of the target) and the observation vector (the distance, angle, pixel position of the target in the image, and polarization information); after the state vector and observation vector are defined, the state equation and observation equation are established, and the data after the machine learning model optimization is used as the observation input of the Kalman filter, and the parameters of the initialization Kalman filter are set, including the initial state estimation, the initial prediction error covariance matrix, the process noise covariance matrix, and the observation noise covariance matrix, and then the Kalman filter is iterated and fused. Finally, the state estimation after the Kalman filter fusion process is output as the result for subsequent target tracking, navigation, etc. The Kalman filter performs filtering and prediction on the sensor observation value, and the predicted new observation value can effectively smooth the sensor observation value, eliminate the noise and drift in the sensor observation value, and improve the accuracy of the prediction.
[0061] Based on the fused data, a three-dimensional model of the vehicle's surrounding environment is constructed, and by analyzing the three-dimensional model of the vehicle's surrounding environment, environmental information such as road structure, obstacle distribution, traffic vehicles, pedestrian movement direction and speed are extracted.
[0062] The vehicle plans its path and driving strategy based on environmental information such as road structure, obstacle distribution, traffic vehicles, pedestrian movement direction and speed extracted from the three-dimensional environmental model. At the same time, it updates the collected data in real time and monitors changes in the surrounding environment in real time. Based on the real-time data and environmental conditions, combined with the vehicle's dynamic characteristics and environmental conditions, it performs intelligent speed control, lane keeping, obstacle avoidance and other operations to ensure safe driving of the vehicle.
[0063] During the process of automatic driving, the intelligent warning module evaluates the potential risks of the vehicle, predicts possible problems and issues warnings in advance to avoid accidents. Warnings can be sent to the driver or the automatic driving system through sound, vision or touch, so that the driver can slow down in advance or take other avoidance measures.
[0064] The intelligent warning module can also communicate with other vehicles and infrastructure to carry out collaborative driving and traffic flow optimization. Through vehicle-to-vehicle and vehicle-to-infrastructure communication, more traffic information such as the driving status of other vehicles, road conditions and traffic signals can be obtained, thereby further improving the accuracy and timeliness of warnings.
[0065] The embodiment of the present application also discloses an autonomous driving system based on an infrared polarization image sensor, including a data acquisition module, a data processing module, a data fusion module, a monitoring decision module, and a warning interaction module.
[0066] Data acquisition module, used to capture and collect information about the vehicle's surrounding environment;
[0067] Data processing module, used for processing collected data, building machine learning models and training models;
[0068] The data fusion module is used to fuse the processed data from different sensors to obtain the final distance, speed and direction information;
[0069] Monitoring and decision-making module, used for data update monitoring and planning of vehicle paths and driving strategies;
[0070] The early warning interaction module is used to evaluate and warn of potential risks of vehicles.
[0071] The warning interaction module is also used to communicate with other vehicles and infrastructure for collaborative driving and traffic flow optimization.
Claims
1. An automatic driving method based on an infrared polarization image sensor, characterized in that: The following steps are involved: S1. Collecting information about the vehicle's surrounding environment; S2. Preprocessing the collected vehicle surrounding environment information; S3. Build a machine learning model, train the model and extract features according to task requirements; S4, fusing the data optimized by the machine learning model through the Kalman filter; S5. construct a three-dimensional model of the vehicle's surrounding environment based on the fused data and analyze it to extract the required vehicle's surrounding environment information; S6. Plan the path and driving strategy of the vehicle according to the extracted vehicle surrounding environment information, and update the collected data in real time to monitor the changes of the vehicle surrounding environment.
2. The automatic driving method based on infrared polarization image sensor according to claim 1, characterized in that: The vehicle's surrounding environment information includes at least road structure, obstacle distribution, traffic vehicles, pedestrians, and movement direction and speed.
3. The automatic driving method based on infrared polarization image sensor according to claim 1, characterized in that: The vehicle's surrounding environment information is collected through infrared polarization image sensors and lidar sensors.
4. The automatic driving method based on infrared polarization image sensor according to claim 1, characterized in that: The method to build a machine learning model is: Automatically extract features from data through deep learning algorithms; Divide the data into training set, validation set and test set according to the proportion. The training set is used to train the model, the validation set is used to adjust the hyperparameters, and the test set is used to finally evaluate the performance of the model. Randomly extract data from the training set or validation set and divide it into two groups: one group is used to train the model, and the other group is used to validate the model; Divide the data into several batches, use the first batch of data to train the model and monitor and evaluate the performance of the model through the validation set data; Test the performance of the model using the data in the test set.
5. The automatic driving method based on infrared polarization image sensor according to claim 4, characterized in that: The training set data accounts for 80-90% of the total data set, the validation set data accounts for 10-20% of the total data set, and the rest of the data set is divided into test sets.
6. The automatic driving method based on infrared polarization image sensor according to claim 1, characterized in that: During the vehicle's autonomous driving process, the intelligent warning module is used to assess the vehicle's potential risks.
7. The automatic driving method based on infrared polarization image sensor according to claim 6, characterized in that: The intelligent warning module communicates with other vehicles and infrastructure.
8. An automatic driving system based on an infrared polarization image sensor, used to implement the automatic driving method based on an infrared polarization image sensor according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, data processing module, data fusion module, monitoring decision module and early warning interaction module; Data acquisition module, used to capture and collect information about the vehicle's surrounding environment; Data processing module, used for processing collected data, building machine learning models and training models; The data fusion module is used to fuse the processed data from different sensors to obtain the final distance, speed and direction information; Monitoring and decision-making module, used for data update monitoring and planning of vehicle paths and driving strategies; The early warning interaction module is used to evaluate and warn of potential risks of vehicles.
9. The automatic driving system based on infrared polarization image sensor according to claim 8, characterized in that: The warning interaction module is also used to communicate with other vehicles and infrastructure.
10. The automatic driving system based on infrared polarization image sensor according to claim 8, characterized in that: The data acquisition module acquires data through an infrared polarization image sensor and a laser radar sensor, and polarizers are installed in front of the mirrors of the infrared polarization image sensor and the laser radar sensor.
Citation Information
Cited By
Intelligent vehicle data acquisition method and system based on big data artificial intelligence analysis
CN119380552A
