Automatic driving sensing system and automatic driving vehicle

Abstract

The invention relates to an automatic driving perception system and an automatic driving vehicle. The system is used for being installed on a vehicle for environment perception. The system comprises a camera module which is arranged on the vehicle in a surrounding manner and is used for fully covering the peripheral view of the vehicle; the infrared module is used for sensing the advancing direction of the vehicle when the vehicle runs at night; the laser radar module and the millimeter wave radar module are used for sensing surrounding obstacle information in an all-weather manner; the camera module, the infrared module, the laser radar module and the millimeter wave radar module act together, and environment information obtained through sensing is subjected to data processing to form a multi-source heterogeneous stereoscopic perception system. According to the automatic driving sensing system, multiple sensing modes are combined to form multi-source heterogeneous three-dimensional sensing, the requirements of various special scenes can be met, and the vehicle safety is improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to autonomous driving perception systems and autonomous driving vehicles. Background Art

[0002] With the rapid development of technology in the field of trunk logistics, the turning point of commercialization of autonomous driving is approaching, and the potential demand for L4 autonomous driving on highways is becoming stronger. However, in the application of autonomous driving heavy trucks on highways, its autonomous driving perception system faces a series of technical difficulties.

[0003] Among them, in order to ensure the safety of driving, the perception distance of the autonomous driving system of the autonomous driving heavy truck has become a key indicator. The maximum perception distance of the autonomous driving perception system must reach more than 1000m to ensure sufficient time for reaction and decision-making; and the precise perception distance must reach more than 250m to achieve accurate grasp of the road conditions ahead.

[0004] However, the requirements for autonomous driving perception have encountered many challenges in practical applications. First, due to the heavy load and high driving speed, the vibration during driving is more severe, and this single sensor is often difficult to achieve accurate and stable perception; secondly, commercial vehicles have extremely high requirements for cost reduction, efficiency improvement and safety, requiring autonomous driving vehicles to not only work normally during the day, but also maintain stable perception and driving capabilities in special scenarios such as night and dense fog. This requires that the perception problems in special scenarios must be solved at the same time.

[0005] Therefore, in the application of autonomous driving heavy trucks in highway scenarios, there are still problems with perception distance, accurate perception, and perception in special scenarios. Summary of the invention

[0006] Based on this, it is necessary to provide an autonomous driving perception system and autonomous driving vehicle that can improve the perception distance, accurately perceive the distance and solve the perception problems in special scenarios during the autonomous driving process, thereby improving vehicle safety.

[0007] An automatic driving perception system is used to be installed on a vehicle for environmental perception, including a camera module, which is arranged on the vehicle in a surround manner and is used for full coverage of the vehicle's circumferential field of view; an infrared module, which is used to perceive the vehicle's forward direction when the vehicle is driving at night; a laser radar module and a millimeter-wave radar module, which are used to perceive surrounding obstacle information around the clock; the camera module, infrared module, laser radar module and millimeter-wave radar module work together to form a multi-source heterogeneous three-dimensional perception system after the perceived environmental information is processed by a data processing unit.

[0008] In one embodiment, the structure of the camera module includes: at least one front-view camera arranged at the front of the vehicle in the forward direction, at least one side front-view camera arranged at the side of the vehicle in the forward direction, and at least one side rear-view camera arranged at the side of the vehicle in the forward direction; the side front-view camera is located at the side and senses toward the side front, and the side rear-view camera is located at the side and senses toward the side rear. The sensing range of the side front-view camera and the side rear-view camera is the direction of all circumferences of the vehicle except the sensing range of the front-view camera; the camera module recognizes and classifies various objects, and this recognition capability helps the autonomous driving plan a safe driving path; in addition, the camera module helps control the vehicle to stay in the center of the lane by identifying the lane lines on the road, reducing the risk of accidents caused by lane deviation.

[0009] In one embodiment, the front-view camera includes a front-view long-range camera, a front-view medium-range camera and a front-view close-range camera; the front-view long-range camera and the front-view medium-range camera are installed at the top of the front of the vehicle in the forward direction to facilitate long-range perception; the front-view close-range camera is installed at the bottom of the front of the vehicle in the forward direction.

[0010] In one embodiment, the structure of the infrared module includes: at least one infrared camera arranged at the front of the vehicle in the direction of travel; the infrared camera is used to detect heat sources in the direction of travel of the vehicle at night. The infrared module is not limited by light conditions and works normally at night, in low light or in bad weather, so that the self-driving car can still maintain accurate perception of the surrounding environment in a complex and changing environment, thereby ensuring driving safety.

[0011] In one of the embodiments, the structure of the laser radar module includes: at least one forward long-range laser radar arranged at the front position of the vehicle in the forward direction, and at least one side laser radar arranged at the side position of the vehicle in the forward direction.

[0012] In one embodiment, a forward blind spot laser radar is also added at the bottom of the front of the vehicle in the forward direction, and the forward blind spot laser radar is used to sense the position of the driver's blind spot in the forward direction of the vehicle. Through the synergistic effect of the forward long-range laser radar, the side laser radar and the forward blind spot laser radar, the laser radar can detect the vehicle in all directions and provide comprehensive and accurate perception information, including obstacle distance and location information, to help the autonomous driving system better identify potential dangers and risks and optimize the decision-making and planning of the autonomous driving system.

[0013] In one embodiment, the structure of the millimeter wave radar module includes: at least one forward millimeter wave radar arranged at the front position of the vehicle in the forward direction, and at least one side angle radar arranged at the side position of the vehicle in the forward direction. Through the forward millimeter wave radar, the distance and speed difference with the vehicle in front can be monitored in real time, so as to automatically adjust the vehicle speed and maintain a safe distance, which not only improves driving comfort, but also reduces the risk of accidents caused by driver fatigue or inattention; through the side angle radar, obstacles on both sides of the vehicle are detected to help the autonomous driving car understand the situation around the vehicle, so as to make the correct lane change or steering decision.

[0014] In one of the embodiments, there are multiple lateral angle radars, and the multiple lateral angle radars are symmetrically distributed on both sides of the vehicle's forward direction.

[0015] An autonomous driving vehicle comprises: a vehicle body and an autonomous driving perception system; the autonomous driving perception system is arranged on the vehicle body.

[0016] The above-mentioned autonomous driving perception system combines the advantages of different perception methods. The camera module is irreplaceable in environmental semantic recognition, and has obvious advantages in recognition accuracy and cost, achieving the effect of full coverage of the field of view; the infrared module detects heat sources in the direction of the vehicle's advance during night driving to meet the needs of night driving; the lidar module and millimeter-wave radar module make up for the defect of low camera visibility in special scenes, realize all-round and multi-angle obstacle detection, and improve the adaptability of the scene; after integrating multiple perception modules with different advantages, a multi-source heterogeneous three-dimensional perception is formed as a whole to meet the needs of various special scenes and improve vehicle safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 is a schematic diagram of an overall system framework in an embodiment of the present invention.

[0018] FIG. 2 is a schematic diagram of an installation arrangement in an embodiment of the present invention.

[0019] Figure 3 It is a side view of Figure 2.

[0020] Figure 4 FIG. 4 is a distribution diagram of the field of view of a camera module in one embodiment of the present invention.

[0021] Figure 5 FIG. 4 is a distribution diagram of the field of view of the infrared module in one embodiment of the present invention.

[0022] Figure 6 4 is a diagram showing the field of view distribution of a laser radar module in one embodiment of the present invention.

[0023] Figure 7 FIG. 4 is a diagram showing the field of view distribution of a millimeter wave radar module in an embodiment of the present invention.

[0024] Figure 8 FIG. 4 is a diagram showing the overall system field of view distribution in one embodiment of the present invention.

[0025] Description of reference numerals:

[0026] 1. Camera module; 2. Infrared module; 3. LiDAR module; 4. Millimeter wave radar module;

[0027] 101, front long-range camera; 102, front mid-range camera; 103, side front camera; 104, side rear camera; 105, front short-range camera;

[0028] 201. Infrared camera;

[0029] 301, forward long-range laser radar; 302, side laser radar; 303, forward blind spot laser radar;

[0030] 401. Forward millimeter-wave radar; 402. Side angle radar. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0032] In the description of the present application, it should be understood that if the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. appear, the orientation or position relationship indicated by these terms is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0033] In addition, if the terms "first" or "second" appear, these terms are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, if the term "plurality" appears, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0034] In this application, unless otherwise clearly specified and limited, if the terms "installed", "connected", "connected", "fixed" and the like appear, these terms should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0035] In the present application, unless otherwise clearly specified and limited, if there is a description that a first feature is "above" or "below" a second feature, etc., or similar descriptions appear, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "above" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0036] It should be noted that if an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be a central element. If an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. If any, the terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used in this application are for illustrative purposes only and do not represent the only implementation method.

[0037] Refer to Figure 1, which shows a framework schematic diagram of an autonomous driving perception system in an embodiment of the present application. The autonomous driving perception system provided by an embodiment of the present application includes a camera module 1, an infrared module 2, a lidar module 3 and a millimeter-wave radar module 4; the camera module is arranged on the vehicle in a surrounding manner to fully cover the vehicle's circumferential field of view; the infrared module is used to perceive the vehicle's forward direction when the vehicle is driving at night; the lidar module and the millimeter-wave radar module are used to perceive surrounding obstacle information around the clock; the camera module, infrared module, lidar module and millimeter-wave radar module work together to form a multi-source heterogeneous three-dimensional perception system after the perceived environmental information is processed by a data processing unit.

[0038] In some embodiments, a data processing unit is also included, which is used to fuse the environmental information perceived by the camera module 1, the infrared module 2, the lidar module 3 and the millimeter wave radar module 4 to form a unified environmental perception model.

[0039] Combined with Figure 2 and Figure 3 As shown in Figure 2 and Figure 3 The schematic diagram of the arrangement of the automatic driving perception system in an embodiment of the present application on the vehicle is shown. In some embodiments, the camera module 1 includes: at least one front view camera arranged at the front position of the vehicle in the forward direction, at least one side front view camera 103 arranged at the side position of the vehicle in the forward direction, and at least one side rear view camera 104 arranged at the side position of the vehicle in the forward direction; the side front view camera 103 is located at the side and perceives towards the side front, and the side rear view camera 104 is located at the side and perceives towards the side rear; the perception range of the side front view camera 103 and the side rear view camera 104 is the direction of the entire circumference of the vehicle except the perception range of the front view camera.

[0040] The front-view camera includes a front-view long-range camera 101 , a front-view medium-range camera 102 , and a front-view short-range camera 105 .

[0041] In a specific embodiment, a front long-range camera 101 and a front medium-range camera 102 are arranged at the top of the vehicle in the forward direction, a front short-range camera 105 is arranged at the bottom of the vehicle in the forward direction, and a pair of side rear-view cameras 104 and a pair of side front-view cameras 103 are arranged on the left and right sides of the vehicle in a symmetrical distribution, and sense towards the side rear and side front respectively; Figure 4 , Figure 4 A distribution diagram of the field of view of the camera module 1 in a specific embodiment is shown.

[0042] In actual operation, in a specific embodiment, the camera module 1 is used to identify and classify various objects, such as other vehicles, pedestrians and obstacles. This recognition capability helps the autonomous driving to plan a safe driving path and avoid collisions with obstacles or other vehicles. In addition, the camera module 1 can help control the vehicle to stay in the center of the lane by identifying lane lines on the road. This is especially important on highways or urban roads, and can ensure that the autonomous driving car drives in accordance with traffic regulations and reduce the risk of accidents caused by deviation from the lane. The camera module 1 can also detect and identify traffic signals, signs and road signs to ensure that the autonomous driving car follows traffic regulations.

[0043] See Figure 2 and Figure 3 In some embodiments, the infrared module 2 includes at least one infrared camera 201 disposed at the front of the vehicle in the forward direction; the infrared camera 201 is used to detect heat sources in the forward direction of the vehicle during nighttime driving.

[0044] In a specific embodiment, an infrared camera 201 is set at the bottom of the vehicle in the direction of travel. The infrared camera 201 can detect people or animals in the direction of travel of the vehicle when driving at night to avoid accidents. Figure 5 , Figure 5 The figure shows a distribution diagram of the field of view of the infrared module 2 in a specific embodiment.

[0045] In actual operation, in a specific embodiment, the infrared camera 201 uses infrared technology to perform imaging and has the ability to work in all-weather environments. Compared with the cameras in the camera module 1, the infrared camera 201 is not restricted by light conditions and can work normally at night, in low light or in bad weather (such as rain, snow, fog and haze), so that the autonomous driving car can still maintain accurate perception of the surrounding environment in a complex and changeable environment, thereby ensuring driving safety. At the same time, the infrared camera 201 is highly sensitive to living things. Since the human body and animals emit specific infrared radiation, the infrared camera 201 can accurately detect the existence of these living things, so that the infrared camera 201 has significant advantages in detecting pedestrians and animals in autonomous driving, especially at night or in bad weather conditions, when the visible light camera is difficult to function, the infrared camera 201 can still reliably provide perception information.

[0046] See Figure 2 and Figure 3 In some embodiments, the laser radar module 3 includes at least one forward long-range laser radar 301 arranged at the front position of the vehicle in the forward direction, and at least one side laser radar 302 arranged at the side position of the vehicle in the forward direction; and a forward blind spot laser radar 303 is also added at the bottom of the front position of the vehicle in the forward direction, and the forward blind spot laser radar 303 is used to sense the driver's blind spot in the forward direction of the vehicle.

[0047] In a specific embodiment, a forward long-range laser radar 301 is arranged at the top of the front of the vehicle in the forward direction, a forward blind laser radar 303 is arranged at the bottom of the front of the vehicle in the forward direction, and a pair of side laser radars 302 are symmetrically arranged on both sides of the front of the vehicle in the forward direction; Figure 6 , Figure 6 A distribution diagram of the field of view of the laser radar module 3 in a specific embodiment is shown.

[0048] In actual work, although the forward long-range laser radar 301 can detect long-distance obstacles in front, due to the limitations of its installation position and field of view, blind spots are often formed on both sides of the vehicle and in the close range. These blind spots may cause the autonomous driving system to be unable to perceive surrounding obstacles in a timely manner in certain scenarios, thereby increasing the risk of accidents. The cooperation of the side laser radar 302 and the forward blind laser radar 303 can be used to supplement these blind spots, provide more comprehensive and accurate perception information, and provide accurate obstacle distance and location information, which can help the autonomous driving system better identify potential dangers and risks and optimize the decision-making and planning of the autonomous driving system; in addition, for example, the laser radar can use a multi-beam laser radar to improve the perception accuracy of the three-dimensional information of surrounding obstacles.

[0049] See Figure 2 and Figure 3 In some embodiments, the millimeter-wave radar module 4 includes at least one forward millimeter-wave radar 401 disposed at the front position of the vehicle in the forward direction, and at least one lateral angle radar 402 disposed at the side position of the vehicle in the forward direction; further, there are multiple lateral angle radars 402, and the multiple lateral angle radars 402 are symmetrically distributed on both sides of the vehicle in the forward direction.

[0050] In a specific embodiment, a forward millimeter wave radar 401 is arranged at the bottom of the front of the vehicle in the forward direction, a pair of side angle radars 402 are arranged at the front position of the side of the vehicle, and another pair of side angle radars 402 are arranged at the rear position of the side of the vehicle, for a total of four side angle radars 402; Figure 7 , Figure 7 A distribution diagram of the field of view of the millimeter wave radar module 4 in a specific embodiment is shown.

[0051] In actual operation, in a specific embodiment, the forward millimeter-wave radar 401 can detect obstacles in front of the vehicle in real time and accurately measure the distance between them and the vehicle; this can help the autonomous vehicle determine when to slow down or stop to avoid collision with obstacles in front; through the forward millimeter-wave radar 401, the autonomous vehicle can monitor the distance and speed difference with the vehicle in front in real time, so as to automatically adjust the vehicle speed and maintain a safe distance, which can not only improve driving comfort, but also reduce the risk of accidents caused by driver fatigue or inattention; the side angle radar 402 is used to detect obstacles on both sides of the vehicle, such as vehicles, pedestrians or shoulders in adjacent lanes, etc., which can help the autonomous vehicle better understand the situation around the vehicle and make correct lane change or steering decisions; during the lane change process, the side angle radar 402 can also monitor the situation of the adjacent lanes in real time to determine whether there is enough space for lane change; at the same time, during high-speed driving, the side angle radar 402 can continuously monitor the dynamic environment on the side of the vehicle to avoid collision between the vehicle and roadside obstacles, which is of great significance to improving the safety of autonomous vehicles.

[0052] See also Figure 8 , Figure 8 The overall field of view distribution of the autonomous driving perception system in some embodiments is shown.

[0053] In some embodiments, the data processing unit uses a deep learning algorithm to analyze and identify environmental information perceived by the perception module to improve the accuracy and reliability of environmental perception.

[0054] In some embodiments, a communication module may also be provided, and the communication module is used to transmit the perceived environmental information to the vehicle's automatic driving control system in real time to achieve real-time decision-making and path planning.

[0055] In some embodiments, the camera module 1 , the infrared module 2 , the lidar module 3 , and the millimeter wave radar module 4 are calibrated regularly to ensure the accuracy of the perception data.

[0056] An automatic driving vehicle comprises a vehicle body and the automatic driving perception system described above, wherein the automatic driving perception system is arranged on the vehicle body. The automatic driving perception system described above utilizes the advantages of different perception modes. The camera module 1 is irreplaceable in environmental semantic recognition, has obvious advantages in recognition accuracy and cost, and can achieve the effect of full coverage of the field of vision; the infrared module 2 detects the heat source in the vehicle's forward direction during night driving to meet the needs of night driving; the laser radar module 3 and the millimeter wave radar module 4 make up for the defect of low camera visibility in special scenes, realize all-round and multi-angle obstacle detection, and improve the adaptability of the scene; finally, the environmental information perceived by the camera module 1, the infrared module 2, the laser radar module 3 and the millimeter wave radar module 4 are fused and processed by the data processing unit to form a unified environmental perception model. After integrating multiple perception modules with different advantages, a multi-source heterogeneous stereoscopic perception is formed as a whole, which meets the needs of various special scenes and can improve the safety of the vehicle.

[0057] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. An automatic driving perception system, used for being installed on a vehicle to perceive the environment, characterized in that: include: A camera module is disposed on the vehicle in a surrounding manner and is used to fully cover the peripheral field of view of the vehicle; Infrared module, used to sense the vehicle's direction of travel when the vehicle is driving at night; LiDAR module and millimeter-wave radar module are used to sense surrounding obstacle information around the clock; The camera module, infrared module, laser radar module and millimeter wave radar module work together to form a multi-source heterogeneous three-dimensional perception system through a data processing unit to sense the environmental information.

2. The autonomous driving perception system according to claim 1, characterized in that: The structure of the camera module includes: at least one front view camera arranged at the front position of the vehicle in the forward direction, at least one side front view camera arranged at the side position of the vehicle in the forward direction, and at least one side rear view camera arranged at the side position of the vehicle in the forward direction.

3. The autonomous driving perception system according to claim 2, characterized in that: The side front view camera is located on the side and senses towards the side front, and the side rear view camera is located on the side and senses towards the side rear; The sensing range of the side front view camera and the side rear view camera is all directions around the vehicle except the sensing range of the front view camera.

4. The autonomous driving perception system according to claim 2, characterized in that: The front-view camera includes a front-view long-range camera, a front-view medium-range camera and a front-view short-range camera; The front-view long-range camera and the front-view medium-range camera are mounted on the top of the front of the vehicle in the forward direction, so as to facilitate long-distance perception; The front-view short-range camera is mounted at the bottom of the front of the vehicle in the forward direction.

5. The autonomous driving perception system according to claim 1, characterized in that: The structure of the infrared module includes: at least one infrared camera arranged at the front position of the vehicle in the forward direction; The infrared camera is used for detecting heat sources in the direction of vehicle travel during nighttime driving.

6. The autonomous driving perception system according to claim 1, characterized in that: The structure of the laser radar module includes: at least one forward long-range laser radar arranged at the front position of the vehicle in the forward direction, and at least one side laser radar arranged at the side position of the vehicle in the forward direction.

7. The autonomous driving perception system according to claim 6, characterized in that: A forward blind spot-filling laser radar is also provided at the bottom of the front of the vehicle in the forward direction, and the forward blind spot-filling laser radar is used to sense that the forward direction of the vehicle is located in the driver's blind spot.

8. The autonomous driving perception system according to claim 1, characterized in that: The structure of the millimeter wave radar module includes: at least one forward millimeter wave radar arranged at the front position of the vehicle in the forward direction, and at least one side angle radar arranged at the side position of the vehicle in the forward direction.

9. The autonomous driving perception system according to claim 8, characterized in that: There are multiple lateral angle radars, which are symmetrically distributed on both sides of the vehicle's forward direction.

10. An automatic driving vehicle, characterized in that: include: A vehicle body and an autonomous driving perception system as described in any one of claims 1 to 9; The autonomous driving perception system is arranged on the vehicle body.

Images