Automatic calibration method for intelligent driving camera and radar

Abstract

The invention discloses an automatic calibration method for an intelligent driving camera and a radar, relates to the technical field of intelligent driving, and solves the technical problem that the driving safety and the intelligent degree of a system are not high due to the fact that obstacles with fuzzy boundaries cannot be accurately recognized in the process of intelligent driving in the prior art. The method comprises the following steps: acquiring vehicle-mounted sensor data in a vehicle driving process; setting a response priority based on the type recognition result, the obstacle type and the property of the obstacle; planning a driving path of the vehicle based on the response priority and the motion state of the obstacle; and the vehicle is monitored and adjusted in real time. According to the invention, through the strategy based on the deformation threshold value, the obstacle which is difficult to recognize can be intelligently judged, so that the intelligent driving system can effectively manage the driving risk according to the property of the obstacle and the potential danger degree under various complex road conditions, and the driving safety and the driving comfort can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent driving, and in particular is an automatic calibration method for an intelligent driving camera and a radar. Background Art

[0002] The development of intelligent driving technology enables vehicles to navigate and avoid obstacles autonomously without the need for human driving intervention. Among them, accurate identification and calibration of roadblocks is one of the key steps to ensure safe driving. With the advancement of sensor technology, such as lidar, cameras, ultrasonic sensors, etc., intelligent driving systems can obtain rich environmental information. There are many obstacles on the road, such as pedestrians, cars, animals, garbage bags, etc., but some obstacles are difficult to accurately judge, such as damaged garbage bags. Due to the blurred boundaries during movement, the vehicle cannot accurately identify them, resulting in low driving safety and low intelligence of the system.

[0003] The present invention provides an automatic calibration method for an intelligent driving camera and a radar to solve the above technical problems. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an automatic calibration method for an intelligent driving camera and a radar, which is used to solve the technical problem that the intelligent driving in the prior art cannot accurately identify obstacles with blurred boundaries during movement, thereby resulting in low driving safety and low intelligence level of the system.

[0005] To achieve the above object, a first aspect of the present invention provides an automatic calibration method for an intelligent driving camera and a radar, comprising:

[0006] S1: Acquire vehicle-mounted sensor data during vehicle driving;

[0007] S2: Identify obstacle types in the vehicle-mounted sensor data based on a deep learning algorithm to obtain type recognition results; obstacle types include people, animals, vehicles, and road signs, and type recognition results include success and failure. Obstacles with successful type recognition results have a certain obstacle type;

[0008] S3: Based on the shape recognition algorithm, obstacles whose type recognition results are failed are identified to obtain the properties of the obstacles; wherein the properties of the obstacles include hardness and flexibility;

[0009] S5: Setting a response priority based on the type recognition result, the obstacle type and the nature of the obstacle; wherein the response priority includes a high priority and a low priority;

[0010] S6: Plan the vehicle's driving path based on the response priority and motion status of the obstacles;

[0011] S7: Real-time monitoring and adjustment of vehicles.

[0012] Preferably, the vehicle-mounted sensors include cameras, laser radars and millimeter-wave radars.

[0013] Preferably, the identifying the obstacle type in the vehicle-mounted sensor data based on the deep learning algorithm includes:

[0014] A1: extract the vehicle sensor data and preprocess it to obtain preprocessed data;

[0015] A2: Select the convolutional neural network model as the deep learning model and use the sample image dataset to train the deep learning model. After the training is completed, a deep learning model is obtained whose input is the obstacle image and output is the obstacle type.

[0016] A3: Classify obstacles according to the deep learning model to obtain the type of obstacles.

[0017] The present invention can usually perform inference and classification in a short time through the deep learning model, and quickly and accurately identify the type of obstacles. The system can take appropriate driving strategies in time, such as avoidance or deceleration, to improve the safety of intelligent driving. The intelligent driving system can effectively acquire and process obstacle data in the surrounding environment, identify different types of obstacles and formulate corresponding response strategies according to their priorities, which is beneficial to improving driving safety and the intelligence level of the system.

[0018] Preferably, the identifying of obstacles with a type recognition failure result based on a shape recognition algorithm includes:

[0019] B1: Obtain the deformation area of ​​obstacles in adjacent time periods based on shape recognition algorithm;

[0020] B2: Calculate the deformation rate based on the deformation area of ​​the obstacle in adjacent time periods;

[0021] B3: Determine whether the deformation rate is greater than the set deformation threshold; if yes, mark the property of the obstacle as flexible; if no, mark the property of the obstacle as hard.

[0022] The present invention can make intelligent judgments on obstacles that are difficult to identify through a strategy based on deformation thresholds, so that the intelligent driving system can still effectively identify and handle obstacles under various complex road conditions, and effectively manage driving risks according to the nature and potential danger level of the obstacles, which is conducive to improving driving safety and driving comfort.

[0023] Preferably, the deformation rate is calculated based on the deformation area of ​​the obstacle in adjacent time periods:

[0024] C1: Extract the shape and area A of the obstacle at the t-1th time point t-1 , the shape and area of ​​the obstacle at the tth time point A t ; Wherein, t is a positive integer;

[0025] C2: By formula Calculate the deformation rate DR.

[0026] Preferably, the setting of the response priority based on the type recognition result, the obstacle type and the nature of the obstacle includes:

[0027] Extract type recognition results, obstacle types and obstacle properties;

[0028] Determine whether the type recognition result is successful; if yes, set the response priority based on the obstacle type; if no, set the response priority based on the nature of the obstacle.

[0029] Preferably, setting the response priority based on the obstacle type includes:

[0030] Extract the obstacle type corresponding to each obstacle; set the corresponding response priority for each obstacle type based on the experience of the technicians.

[0031] Preferably, setting the response priority based on the property of the obstacle includes:

[0032] Extract the nature of the obstacle; determine whether the nature of the obstacle is hard; if yes, mark the response priority as high priority; if no, mark the response priority as low priority.

[0033] Preferably, the planning of the vehicle's driving path based on the response priority and motion state of the obstacle includes:

[0034] F1: Extract the response priority and motion state of the obstacle; the motion state includes position, velocity and acceleration;

[0035] F2: Determine whether the obstacle response priority is high priority; if yes, the vehicle adopts an emergency driving strategy and jumps to F3; if no, the vehicle drives normally and jumps to F1; the emergency driving strategy includes avoidance and deceleration;

[0036] F3: Predict the motion state of the obstacle based on the physical tracking algorithm to obtain the motion trajectory of the obstacle;

[0037] F4: Plan the vehicle's driving path based on the movement trajectory of obstacles.

[0038] Preferably, the real-time monitoring and adjustment of the vehicle includes:

[0039] G1: Obtain and fuse onboard sensor data to generate accurate understanding of the surrounding environment;

[0040] G2: monitor the vehicle status and surrounding environment changes;

[0041] G3: Analyze the vehicle’s driving data to obtain potential improvement points and corresponding optimization strategies.

[0042] The present invention acquires data about the surrounding environment, including information about roads, obstacles, vehicle status, etc., processes and fuses the raw data acquired by sensors, and generates an accurate understanding of the surrounding environment. This includes tasks such as obstacle detection, road sign detection, lane line detection, and real-time tracking of surrounding vehicles and pedestrians, and adjusting decision-making and control strategies at any time, which can ensure the safety and reliability of the vehicle during intelligent driving.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention can usually perform inference and classification in a short time through the deep learning model, and quickly and accurately identify the type of obstacles. The system can take appropriate driving strategies in time, such as avoidance or deceleration, to improve the safety of intelligent driving. The intelligent driving system can effectively acquire and process obstacle data in the surrounding environment, identify different types of obstacles and formulate corresponding response strategies according to their priorities, which is conducive to improving driving safety and the intelligence level of the system.

[0045] 2. The present invention can make intelligent judgments on obstacles that are difficult to identify through a strategy based on deformation thresholds, so that the intelligent driving system can still effectively identify and handle obstacles under various complex road conditions, and effectively manage driving risks according to the nature and potential danger level of the obstacles, which is conducive to improving driving safety and driving comfort.

[0046] 3. The present invention acquires data about the surrounding environment, including information about roads, obstacles, vehicle status, etc., processes and fuses the raw data acquired by sensors, and generates an accurate understanding of the surrounding environment. This includes tasks such as obstacle detection, road sign detection, lane line detection, and real-time tracking of surrounding vehicles and pedestrians, and adjusting decision-making and control strategies at any time, which can ensure the safety and reliability of the vehicle during intelligent driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is an overall flow chart of the intelligent driving automatic calibration method of the present invention;

[0049] Figure 2 A flow chart for identifying the nature of obstacles in the present invention;

[0050] Figure 3 The present invention is a flow chart for planning the driving path of a vehicle. DETAILED DESCRIPTION

[0051] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1-Figure 3 The first aspect of the present invention provides an automatic calibration method for an intelligent driving camera and a radar, comprising:

[0053] S1: Acquire vehicle-mounted sensor data during vehicle driving;

[0054] S2: Identify obstacle types in the vehicle-mounted sensor data based on a deep learning algorithm to obtain type recognition results; obstacle types include people, animals, vehicles, and road signs, and type recognition results include success and failure. Obstacles with successful type recognition results have a certain obstacle type;

[0055] S3: Based on the shape recognition algorithm, obstacles whose type recognition results are failed are identified to obtain the properties of the obstacles; wherein the properties of the obstacles include hardness and flexibility;

[0056] S5: Setting a response priority based on the type recognition result, the obstacle type and the nature of the obstacle; wherein the response priority includes a high priority and a low priority;

[0057] S6: Plan the vehicle's driving path based on the response priority and motion status of the obstacles;

[0058] S7: Real-time monitoring and adjustment of vehicles.

[0059] In this embodiment, the vehicle-mounted sensors include cameras, laser radars, and millimeter-wave radars.

[0060] For example,

[0061] In this embodiment, the obstacle type in the vehicle sensor data is identified based on the deep learning algorithm, including:

[0062] A1: extract the vehicle sensor data and preprocess it to obtain preprocessed data;

[0063] A2: Select the convolutional neural network model as the deep learning model and use the sample image dataset to train the deep learning model. After the training is completed, a deep learning model is obtained whose input is the obstacle image and output is the obstacle type.

[0064] A3: Classify obstacles according to the deep learning model to obtain the type of obstacles.

[0065] Exemplarily, the vehicle-mounted sensor data is extracted and preprocessed to obtain preprocessed data; wherein the preprocessing method includes eliminating abnormal data and filling missing data;

[0066] Select the convolutional neural network model as the deep learning model, use 80% of the sample image dataset as training data and 20% as test data;

[0067] The deep learning model is trained using the training data; the trained deep learning model is tested using the test data, and the deep learning model is adjusted according to the test results; after the training is completed, a deep learning model is obtained whose input is the obstacle image and whose output is the obstacle type.

[0068] The present invention can usually perform inference and classification in a short time through the deep learning model, and quickly and accurately identify the type of obstacles. The system can take appropriate driving strategies in time, such as avoidance or deceleration, to improve the safety of intelligent driving. The intelligent driving system can effectively acquire and process obstacle data in the surrounding environment, identify different types of obstacles and formulate corresponding response strategies according to their priorities, which is beneficial to improving driving safety and the intelligence level of the system.

[0069] In this embodiment, obstacles with failed type recognition results are identified based on the shape recognition algorithm, including:

[0070] B1: Obtain the deformation area of ​​obstacles in adjacent time periods based on shape recognition algorithm;

[0071] B2: Calculate the deformation rate based on the deformation area of ​​the obstacle in adjacent time periods;

[0072] B3: Determine whether the deformation rate is greater than the set deformation threshold; if yes, mark the property of the obstacle as flexible; if no, mark the property of the obstacle as hard.

[0073] Exemplarily, the deformation threshold e=10% and the deformation rate DR=13% are set; since the deformation rate is greater than the set deformation threshold, the property of the obstacle is marked as flexible.

[0074] The present invention can make intelligent judgments on obstacles that are difficult to identify through a strategy based on deformation thresholds, so that the intelligent driving system can still effectively identify and handle obstacles under various complex road conditions, and effectively manage driving risks according to the nature and potential danger level of the obstacles, which is conducive to improving driving safety and driving comfort.

[0075] In this embodiment, the deformation rate is calculated based on the deformation area of ​​the obstacle in adjacent time periods:

[0076] C1: Extract the shape and area A of the obstacle at the t-1th time point t-1 , the shape and area of ​​the obstacle at the tth time point A t ; Wherein, t is a positive integer;

[0077] C2: By formula Calculate the deformation rate DR.

[0078] For example, set t=20, and assume that the shape and area of ​​the obstacle at the 19th time point is A t-1 =100cm 2 , the shape and area of ​​the obstacle at the 20th time point A t =87cm 2 ; The deformation rate DR is calculated by the formula to be 13%.

[0079] In this embodiment, the response priority is set based on the type recognition result, obstacle type and obstacle property, including:

[0080] Extract type recognition results, obstacle types and obstacle properties;

[0081] Determine whether the type recognition result is successful; if yes, set the response priority based on the obstacle type; if no, set the response priority based on the nature of the obstacle.

[0082] In this embodiment, the response priority is set based on the obstacle type, including:

[0083] Extract the obstacle type corresponding to each obstacle; set the corresponding response priority for each obstacle type based on the experience of the technicians.

[0084] Exemplarily, the obstacle types corresponding to the obstacles include people, animals, vehicles and road signs, and the response priorities of people, animals and vehicles are set to high priorities, and the response priority of road signs is set to low priorities.

[0085] In this embodiment, the response priority is set based on the nature of the obstacle, including:

[0086] Extract the nature of the obstacle; determine whether the nature of the obstacle is hard; if yes, mark the response priority as high priority; if no, mark the response priority as low priority.

[0087] Exemplarily, assuming that the obstacle is a plastic bag, the obstacle property of the plastic bag is flexibility, so the response priority of the plastic bag is set to a low priority.

[0088] In this embodiment, the vehicle's driving path is planned based on the response priority and motion state of the obstacle, including:

[0089] F1: Extract the response priority and motion state of the obstacle; the motion state includes position, velocity and acceleration;

[0090] F2: Determine whether the obstacle response priority is high priority; if yes, the vehicle adopts an emergency driving strategy and jumps to F3; if no, the vehicle drives normally and jumps to F1; the emergency driving strategy includes avoidance and deceleration;

[0091] F3: Predict the motion state of the obstacle based on the physical tracking algorithm to obtain the motion trajectory of the obstacle;

[0092] F4: Plan the vehicle's driving path based on the movement trajectory of obstacles.

[0093] Exemplarily, the response priority of the current obstacle is set to a high priority, at which time the vehicle will adopt an emergency driving strategy to avoid traffic accidents; the vehicle predicts the motion state of the obstacle based on a physical tracking algorithm to obtain the motion trajectory of the obstacle; and the vehicle's driving path is planned according to the motion trajectory of the obstacle.

[0094] In this embodiment, real-time monitoring and adjustment of the vehicle include:

[0095] G1: Obtain and fuse onboard sensor data to generate accurate understanding of the surrounding environment;

[0096] G2: monitor the vehicle status and surrounding environment changes;

[0097] G3: Analyze the vehicle’s driving data to obtain potential improvement points and corresponding optimization strategies.

[0098] The present invention acquires data about the surrounding environment, including information about roads, obstacles, vehicle status, etc., processes and fuses the raw data acquired by sensors, and generates an accurate understanding of the surrounding environment. This includes tasks such as obstacle detection, road sign detection, lane line detection, and real-time tracking of surrounding vehicles and pedestrians, and adjusting decision-making and control strategies at any time, which can ensure the safety and reliability of the vehicle during intelligent driving.

[0099] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An automatic calibration method for intelligent driving cameras and radars, characterized in that: include: S1: Acquire vehicle-mounted sensor data during vehicle driving; S2: Identify obstacle types in the vehicle-mounted sensor data based on a deep learning algorithm to obtain type recognition results; obstacle types include people, animals, vehicles, and road signs, and type recognition results include success and failure. Obstacles with successful type recognition results have a certain obstacle type; S3: Based on the shape recognition algorithm, obstacles whose type recognition results are failed are identified to obtain the properties of the obstacles; wherein the properties of the obstacles include hardness and flexibility; S5: Setting a response priority based on the type recognition result, the obstacle type and the nature of the obstacle; wherein the response priority includes a high priority and a low priority; S6: Plan the vehicle's driving path based on the response priority and motion status of the obstacles; S7: Real-time monitoring and adjustment of vehicles.

2. The automatic calibration method for intelligent driving camera and radar according to claim 1, characterized in that: The vehicle-mounted sensors include cameras, laser radars and millimeter-wave radars.

3. The automatic calibration method for intelligent driving camera and radar according to claim 1, characterized in that: The method of identifying obstacle types in vehicle-mounted sensor data based on a deep learning algorithm includes: A1: extract the vehicle sensor data and preprocess it to obtain preprocessed data; A2: Select the convolutional neural network model as the deep learning model and use the sample image dataset to train the deep learning model. After the training is completed, a deep learning model is obtained whose input is the obstacle image and output is the obstacle type. A3: Classify obstacles according to the deep learning model to obtain the type of obstacles.

4. The automatic calibration method for a smart driving camera and radar according to claim 1, characterized in that: The step of identifying obstacles with a failed type recognition result based on a shape recognition algorithm includes: B1: Obtain the deformation area of ​​obstacles in adjacent time periods based on shape recognition algorithm; B2: Calculate the deformation rate based on the deformation area of ​​the obstacle in adjacent time periods; B3: Determine whether the deformation rate is greater than the set deformation threshold; if yes, mark the property of the obstacle as flexible; if no, mark the property of the obstacle as hard.

5. The automatic calibration method of a smart driving camera and radar according to claim 4, characterized in that: The deformation rate is calculated based on the deformation area of ​​the obstacle in adjacent time periods: C1: Extract the shape and area A of the obstacle at the t-1th time point t-1 , the shape and area of ​​the obstacle at the tth time point A t ; Wherein, t is a positive integer; C2: By formula Calculate the deformation rate DR.

6. The automatic calibration method for intelligent driving camera and radar according to claim 1, characterized in that: The setting of the response priority based on the type recognition result, the obstacle type and the nature of the obstacle includes: Extract type recognition results, obstacle types and obstacle properties; Determine whether the type recognition result is successful; if yes, set the response priority based on the obstacle type; if no, set the response priority based on the nature of the obstacle.

7. The automatic calibration method for a smart driving camera and radar according to claim 6, characterized in that: The setting of response priority based on obstacle type includes: Extract the obstacle type corresponding to each obstacle; set the corresponding response priority for each obstacle type based on the experience of the technicians.

8. The automatic calibration method for a smart driving camera and radar according to claim 6, characterized in that: The setting of the response priority based on the nature of the obstacle includes: Extract the nature of the obstacle; determine whether the nature of the obstacle is hard; if yes, mark the response priority as high priority; if no, mark the response priority as low priority.

9. The automatic calibration method for intelligent driving camera and radar according to claim 1, characterized in that: The planning of the vehicle's driving path based on the response priority and motion state of the obstacle includes: F1: Extract the response priority and motion state of the obstacle; the motion state includes position, velocity and acceleration; F2: Determine whether the obstacle response priority is high priority; if yes, the vehicle adopts an emergency driving strategy and jumps to F3; if no, the vehicle drives normally and jumps to F1; the emergency driving strategy includes avoidance and deceleration; F3: Predict the motion state of the obstacle based on the physical tracking algorithm to obtain the motion trajectory of the obstacle; F4: Plan the vehicle's driving path based on the movement trajectory of obstacles.

10. The automatic calibration method for a smart driving camera and radar according to claim 1, characterized in that: The real-time monitoring and adjustment of the vehicle includes: G1: Obtain and fuse onboard sensor data to generate accurate understanding of the surrounding environment; G2: monitor the vehicle status and surrounding environment changes; G3: Analyze the vehicle’s driving data to obtain potential improvement points and corresponding optimization strategies.

Images