Automatic emergency braking system and method for vehicle

Abstract

The invention discloses an automatic emergency braking system and method for a vehicle. The driving safety and system redundancy of the vehicle can be greatly improved. Comprising a sensor module used for collecting environment data around a vehicle when the vehicle runs; the model building module is used for building a prediction model of the vehicle; and the control module is used for carrying out collision detection on the vehicle and the environmental data based on the prediction model, and braking the vehicle when judging that the vehicle is collided.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to an automatic emergency braking system and method for a vehicle. Background Art

[0002] Currently, with the development of vehicle automation and intelligence, intelligent driving of vehicles has also attracted much attention. In traditional vehicle safety assurance systems, a single-line laser sensor (such as lidar) is usually relied on to detect the surrounding environment (including obstacles), and then to identify, prevent, and avoid collision risks (such as decelerating to stop, emergency stopping). However, a single-line laser sensor often has the following defects in actual use: it is difficult to effectively process noise in complex environments, and its noise processing ability is insufficient; when the sensor fails, the safety assurance fails, and the system redundancy is insufficient, etc. Although multi-line laser sensors have been used for auxiliary design currently, these problems have not been completely solved yet. Summary of the Invention

[0003] Technical Problems to be Solved by the Invention

[0004] This application is formed to solve the above technical problems, and its purpose is to provide an automatic emergency braking system and method for a vehicle, which can greatly improve the driving safety of the vehicle and the system redundancy.

[0005] Technical Solutions for Solving the Technical Problems

[0006] This application provides an automatic emergency braking system for a vehicle, including: a sensor module, which collects environmental data around the vehicle when the vehicle is driving; a model construction module, which constructs a prediction model of the vehicle; and a control module, which performs collision detection on the vehicle and the environmental data based on the prediction model, and brakes the vehicle when it is determined that a collision will occur.

[0007] Preferably, the sensor module includes a plurality of sensors, and the plurality of sensors are installed on the vehicle in a manner that covers all directions and all angles of the vehicle.

[0008] Preferably, the plurality of sensors are cameras, lidars, laser sensors, ultrasonic sensors, and / or combinations thereof.

[0009] Preferably, the sensor module includes at least one single-line lidar and at least one multi-line lidar installed on the vehicle.

[0010] Preferably, the model construction module constructs successively: a basic model of the vehicle; an extended model including a safety zone; and the prediction model that dynamically adjusts the safety zone according to the driving state of the vehicle.

[0011] Preferably, in the prediction model, different safety levels are assigned to the safety zones, and the safety levels are dynamically adjusted according to the driving state of the vehicle.

[0012] Preferably, the control module includes: a data processing unit that processes the environmental data to obtain information about obstacles; an attitude prediction unit that predicts the attitude of the vehicle in the next n seconds in combination with the prediction model; and a collision determination unit that determines whether the vehicle will collide with the obstacle in the next n seconds in combination with the attitude and the information about the obstacle.

[0013] Preferably, in the data processing unit, the data of multiple sensors are fused, and the types and contours of objects in the surrounding environment are identified as the information about the obstacles.

[0014] Preferably, the attitude prediction unit, when the vehicle is in the autonomous driving mode, predicts the attitude of the vehicle in the next n seconds based on the control amount of the vehicle in the next n seconds and in combination with the prediction model; when the vehicle is in the manual driving mode, predicts the attitude of the vehicle in the next n seconds based on the real-time state information of the vehicle's driving and assuming that the real-time state information remains unchanged in the next n seconds and in combination with the prediction model.

[0015] Preferably, n is 1 to 5.

[0016] Preferably, the collision determination unit outputs a braking instruction if it determines that the vehicle will collide with the obstacle in the next n seconds, and outputs an instruction to continue the current driving if it determines that the vehicle will not collide with the obstacle in the next n seconds.

[0017] Preferably, the collision determination unit detects whether the attitude of the vehicle overlaps with the obstacle in the environment. If there is an overlap, it is determined that a collision will occur; if there is no overlap, it is determined that no collision will occur.

[0018] Preferably, the control module further includes a risk assessment unit that divides different risk levels based on the determination result of the collision determination unit and the attitude of the vehicle in the next n seconds.

[0019] Preferably, the vehicle is a straddle carrier, and the straddle carrier includes: a main body that can move on the ground and has an accommodation space inside; a grasping part installed in the accommodation space of the main body.

[0020] Preferably, the prediction model is composed of a main body model constructed based on the main body and a grasping part model constructed based on the grasping part.

[0021] Preferably, the main body model and the grasping part model have safety zones that can be independently adjusted.

[0022] Preferably, in the prediction model, the safety area of the grasping part model includes a safety area in the horizontal direction and a safety area in the vertical direction. When the grasping part is in the automatic grasping and placing mode, based on the control amount of the grasping part in the next n seconds and in combination with the prediction model, predict the attitude of the grasping part in the horizontal and vertical directions in the next n seconds; when the grasping part is in the manual grasping and placing mode, based on the real-time state information of the operation of the grasping part and assuming that the real-time state information remains unchanged in the next n seconds, and in combination with the prediction model, predict the attitude of the grasping part in the horizontal and vertical directions in the next n seconds.

[0023] Preferably, n is 1 to 5.

[0024] This application also provides an automatic emergency braking method for a vehicle, including: an environment perception step of collecting environment data around the vehicle when the vehicle is traveling; a model construction step of constructing a prediction model of the vehicle; and a vehicle control step of performing a collision detection on the vehicle and the environment data based on the prediction model, and braking the vehicle when it is determined that a collision will occur.

[0025] Preferably, in the environment perception step, when perceiving an object, use sensors provided on the vehicle, and the sensors include at least one single-line lidar and at least one multi-line lidar.

[0026] Preferably, in the model construction step, construct in sequence: a basic model of the vehicle; an extended model including a safety area; and the prediction model that dynamically adjusts the safety area according to the driving state of the vehicle.

[0027] Preferably, the vehicle control step includes: a data processing step of processing the environment data in the environment perception step to obtain information about obstacles; an attitude prediction step of predicting the attitude of the vehicle in the next n seconds in combination with the prediction model in the model construction step; and a collision determination step of determining whether the vehicle will collide with the obstacle in the next n seconds in combination with the attitude and the information about the obstacle.

[0028] Preferably, in the data processing step, fuse the environment data in the environment perception step, perform classification and recognition on the fused environment data, and obtain the types and contours of objects in the surrounding environment as the information about the obstacles.

[0029] Preferably, in the data processing step, use a deep learning model and a clustering analysis algorithm to perform classification and recognition on the fused data.

[0030] Preferably, in the attitude prediction step, when the vehicle is in the autonomous driving mode, based on the control amount of the vehicle in the next n seconds and in combination with the prediction model, predict the attitude of the vehicle in the next n seconds; when the vehicle is in the manual driving mode, based on the real-time state information of the vehicle running and assuming that the real-time state information remains unchanged in the next n seconds, and in combination with the prediction model, predict the attitude of the vehicle in the next n seconds.

[0031] Preferably, in the collision determination step, if it is determined that the vehicle will collide with the obstacle in the next n seconds, output a braking instruction; if it is determined that the vehicle will not collide with the obstacle in the next n seconds, output an instruction to continue the current driving.

[0032] Preferably, in the collision determination step, detect whether the attitude of the vehicle overlaps with the obstacle in the environment. If there is an overlap, it is determined that a collision will occur; if there is no overlap, it is determined that no collision will occur.

[0033] Preferably, it further includes a risk assessment step. In the risk assessment step, based on the judgment result of the collision determination step and the attitude of the vehicle in the next n seconds, different risk levels are divided.

[0034] Advantages of the Invention

[0035] According to the automatic emergency braking system and method for a vehicle of the present application, by constructing an adaptive straddle carrier model, a fusion algorithm for multi-line lidar, a deep learning point cloud filtering algorithm, and a vehicle future attitude prediction technology, even in a complex environment, whether in the autonomous driving or manual driving mode, effective collision detection and prevention can be carried out on the straddle carrier, realizing the identification, prevention, and avoidance of collision risks, ensuring the safety of the vehicle during driving or operation, and at the same time improving the system redundancy. Description of the Drawings

[0036] Figure 1 It is a structural block diagram showing an automatic emergency braking system for a vehicle according to an embodiment of the present application.

[0037] Figure 2 It shows the basic model, extended model, and prediction model of the vehicle.

[0038] Figure 3 It is a flowchart showing an automatic emergency braking method for a vehicle according to an embodiment of the present application.

[0039] Figure 4 It is a schematic structural diagram showing a straddle carrier as an embodiment of the present application.

[0040] Figure 5 It shows the basic model, extended model, and prediction model of the straddle carrier.

[0041] Symbol Explanation:

[0042] 10 - Automatic Emergency Braking System; 20 - Communication System; 30 - Autopilot System; 110 - Sensor Module; 120 - Model Building Module; 130 - Control Module; 131 - Data Processing Unit; 132 - Attitude Prediction Unit; 133 - Collision Judgment Unit. Detailed Implementation Manner

[0043] Hereinafter, the present application will be further described in conjunction with the following embodiments. It should be understood that the following embodiments are only used to illustrate the present application and do not limit the present application. The same reference signs throughout the specification denote the same components. In addition, unless otherwise defined, all terms (including technical and scientific terms) used in this specification have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs. And, commonly used terms defined in the dictionary, unless otherwise specifically defined, should not be interpreted abnormally or overly. In addition, for the known functions mentioned in this specification, although the corresponding structures for implementing these functions may not be disclosed due to space limitations, it is default that such structures are already provided.

[0044] A vehicle according to an embodiment of the present application includes: an Automatic Emergency Braking System (hereinafter also referred to as the AEB system) 10, a Communication System 20, and an Autopilot System 30. The Communication System 20 is used to connect the vehicle with the external environment, between vehicles, and between various components inside the vehicle and transmit communication.

[0045] Figure 1 is a structural block diagram showing the Automatic Emergency Braking System 10. Hereinafter, refer to Figure 1 Specifically described. The Automatic Emergency Braking System of a vehicle according to an embodiment of the present application includes: a Sensor Module 110, a Model Building Module 120, and a Control Module 130. Specifically, the Sensor Module 110 collects environmental data around the vehicle when the vehicle is running. The Model Building Module 120 builds a prediction model of the vehicle. The Control Module 130 performs collision detection on the vehicle and the environmental data based on the prediction model, and brakes the vehicle when it is determined that a collision will occur.

[0046] More specifically, the sensor module 110 includes a plurality of sensors, which are installed on the vehicle in a way that covers all directions and angles of the vehicle. The plurality of sensors can be, for example, cameras, lidars, laser sensors, ultrasonic sensors, and / or combinations thereof, etc. The sensor module 110 is capable of sensing a specific area of an object, that is, a sensing area, which is also the surrounding environment of the vehicle. When a sensor such as a lidar is applied as the sensor module 110, the sensor module 110 emits signals in all directions centered on the vehicle. When the signals encounter an object, they are reflected and then received by the sensor module 110. The sensor module 110 transmits the signals that are emitted and reflected back, as well as time, etc., to the control module 130. In addition, when a sensor such as a camera is applied as the sensor module 110, an image of the corresponding sensing area is captured and transmitted to the control module 130.

[0047] As mentioned above, in the AEB system, it is far from enough to rely only on a single sensor to detect the collision risk. Therefore, in this embodiment, the sensor module 110 includes at least one single-line lidar and at least one multi-line lidar installed on the vehicle. Among them, the single-line lidar performs 1D scanning and can cover a 2D plane when rotating, and the multi-line lidar performs 2D or 3D point cloud sensing. More specifically, the single-line lidar is mostly used to extract line segment or boundary features, and the multi-line lidar is mostly used to extract three-dimensional geometric features such as planes and curvatures. However, this is only an example here, and the specific application is not limited to this. In addition, the sensor module 110 can also detect the vehicle's own information, such as the vehicle's speed, acceleration, etc. However, in this application, the sensor module 110 only needs to be able to sense the surrounding environment (including objects in the environment) at least, and no specific requirements are made for other functions.

[0048] Figure 2 The basic model, extended model, and prediction model of the vehicle are shown. Hereinafter, with reference to Figure 2 Details are described. In addition, Figure 2 In, for the sake of simplifying the illustration, only the vehicle and its surrounding safety zones are schematically shown in a two-dimensional plane. However, it should be understood that both the vehicle and its surrounding safety zones are three-dimensional structures.

[0049] The model construction module 120 constructs the prediction model of the vehicle and sends it to the attitude prediction unit 132. Specifically, the model construction module 120 constructs: the basic model, extended model, and prediction model of the vehicle. More specifically, the model construction module 120 constructs the basic model of the vehicle, constructs an extended model including the safety zone based on the basic model, and constructs a prediction model that dynamically adjusts the safety zone based on the extended model and according to the vehicle's driving state.

[0050] Figure 2Among them, (a) schematically shows the basic model of the vehicle. According to the basic vehicle information (such as length, width, height, etc.), the geometric shape of the vehicle (simplified vehicle model) is defined by a rectangle. Figure 2 Among them, (b) schematically shows the extended model of the vehicle. The extended model is constructed on the basis of the basic model according to vehicle information, environmental information, transportation requirements, etc. The extended model includes a safety zone (shown as a gray area in the figure) surrounding the vehicle. The safety zone moves along with the vehicle when it moves and serves as a buffer space. Its width can be determined, for example, as 0.2 meters to 1 meter, etc. For example, when the vehicle has a large self-weight, large inertia and a long braking distance, the safety zone is set larger. Another example is that when the vehicle has a high height and a high center of gravity, the safety zone is set larger, and so on. Figure 2 Among them, (c) schematically shows the prediction model of the vehicle. According to the vehicle speed or environmental conditions, the range of the safety zone is dynamically adjusted on the basis of the extended model. For example, when the vehicle is driving at a high speed or has a large acceleration, a larger front safety zone is required. At this time, the rear safety zone and the side safety zones can be appropriately reduced. When turning, a larger side safety zone is required, and at this time, the front safety zone and the rear safety zone can be appropriately reduced. At the same time, for example, different color depths can also be set for the safety zone to represent different safety levels. For example, the closer to the vehicle, the darker the color of the safety zone, indicating less safety.

[0051] In this embodiment, the model construction module 120 receives the driving data of the vehicle via the communication system 20, and dynamically updates the shape, size and color (i.e., safety level) of the safety zone based on the received driving data. In other words, the prediction model (specifically, the safety zone in the prediction model) is updated in real time according to the vehicle motion state, etc. The motion data can include, for example, the vehicle dimensions, chassis model, etc. pre-input, and can also include the vehicle speed, wheel speed, acceleration, operation state of the joystick, vehicle state, and even weather conditions, etc. from the vehicle body sensors. It can also include position information, destination information, etc. from GPS, etc. The specific type of the driving data can be determined according to actual needs and will not be elaborated here.

[0052] Next, continue to refer to Figure 1 The control module 130 will be described in detail. Specifically, the control module 130 includes: a data processing unit 131 that processes the environmental data collected by the sensor module 110 to obtain information about obstacles in the surrounding environment; an attitude prediction unit 132 that predicts the attitude of the vehicle in the next n seconds in combination with the prediction model of the model construction module 120; and a collision determination unit 133 that determines whether the vehicle will collide with an obstacle (an object in the surrounding environment) in the next n seconds in combination with the attitude and the obstacle information.

[0053] The data processing unit 131 receives inputs from the sensor module 110, generates a three-dimensional point cloud map of the vehicle's surrounding environment using multi-sensor data fusion technology, applies a deep learning model to identify objects in the environment, such as vehicles, pedestrians, obstacles, etc., and outputs structured environment information, such as the position information, speed, size, and category of surrounding objects. Details are as follows.

[0054] First, in this embodiment, the sensor module 110 includes a single-line lidar and a multi-line lidar. Therefore, the data processing unit 131 needs to fuse the sensor data from the sensor module 110 to combine the characteristics of the two devices to make up for the deficiencies of a single sensor, achieve more accurate and comprehensive environmental perception, and improve the system robustness and redundancy. For example, through external parameter calibration, the relative position and attitude of the two lidars are obtained, and the data of the single-line lidar and the multi-line lidar are unified in a world coordinate system (such as the vehicle body coordinate system or the global coordinate system). Generally speaking, since the single-line lidar has a higher sampling frequency, the time can be synchronized through GPS or timestamps. Then, the 2D point cloud data collected by the single-line lidar and the 3D point cloud data collected by the multi-line lidar are fused through methods such as projection, stitching, or weighted fusion. Furthermore, key features (such as edges, planes, curved surfaces, etc.) can be extracted from the fused point cloud data and matched and optimized (such as ICP or NDT).

[0055] Next, using the trained deep learning model, it receives the input of the fused point cloud data and outputs object category information in the environment to the collision determination unit 133. Specifically, the deep learning model globally filters and optimizes the fused point cloud, such as classification and denoising. Points that cannot be classified and recognized are removed as noise points, and the recognized points are re-encoded to determine the object type (such as vehicles, pedestrians, buildings, etc.) and the precise contour or boundary (i.e., the surrounding environment), and sent to the collision determination unit 133. Among them, the deep learning model, as an existing open-source model, can be based on existing frameworks (such as PyTorch or TensorFlow, etc.) and network structures for point cloud processing (such as PointNet, PointNet++, KPConv, etc.).

[0056] In this embodiment, the deep learning model performs category prediction on each input point, and the classification categories include: noise points, suspected points, and valid points. For example, points classified as noise points are directly filtered (such as obvious outliers or abnormal points, etc.). Points classified as valid points are re-encoded to further determine the specific type of the obstacle, such as pedestrians, vehicles, etc. as dynamic obstacles, and road piles, walls, trees, etc. as static obstacles. Points that cannot match the characteristics of known obstacles are marked as suspected points for further analysis. At this time, in order to prevent the suspected points representing obstacles from being misjudged as noise points and removed, error prevention measures can be adopted. For example, clustering fallback can be used, and clustering algorithms such as DBSCAN are used to re-cluster the point cloud, extract and analyze the features of small-scale point cloud clusters, and determine whether they may be obstacles. For example, if the number and shape of the point cloud meet certain threshold conditions (such as height, width), the cluster is retained; for example, point features can also be used to analyze the geometric features of the point cloud (such as density, surface normal vector) and the connection relationship between adjacent points, and determine whether the height of the point cloud conforms to the physical characteristics of the obstacle, and whether the distance between points constitutes continuity, etc.; for example, time series verification can also be used, combined with continuous frame point cloud data, to perform temporal analysis on suspected noise points. If the suspected points persist in multiple time frames, they may be valid obstacles. Thus, the present application introduces a hybrid strategy such as clustering-deep learning to enhance the discrimination ability of suspected points, ensure that the misjudgment risk of a single method is minimized, and guarantee the safety and robustness of the system.

[0057] The attitude prediction unit 132 predicts the attitude of the vehicle in the next n seconds, and the attitude includes position, direction, angle, speed, etc. At this time, the vehicle has two driving modes: autonomous driving mode and manual driving mode.

[0058] When the vehicle is in the autonomous driving mode, the autonomous driving system 30 combines the current state, control instructions, planning goals, and dynamic model of the vehicle, etc., plans the driving trajectory in the next n seconds, calculates the corresponding control quantity output in the next n seconds, and sends it to the attitude prediction unit 132. Among them, the control quantity includes but is not limited to: direction control quantities such as wheel rotation angle; speed control quantities such as acceleration, deceleration, and target speed; and trajectory control quantities such as lateral offset and heading angle. The attitude prediction unit 132 receives the control quantity input from the autonomous driving system 30, and receives the prediction model constructed by the model construction module 120 (specifically, a prediction model updated in real time). By combining the control quantity in the next n seconds with the prediction model of the vehicle, the attitude of the vehicle including the safety zone in the next n seconds can be predicted. And the predicted attitude of the vehicle including the safety zone in the next n seconds is sent to the collision determination unit 133.

[0059] When the vehicle is in the manual driving mode, that is, when it is operated by a human at this time, the attitude of the vehicle's driving state information changes constantly, and it usually needs to be detected in real time by sensors. The attitude prediction unit 132 receives the real-time state information of the vehicle's driving (such as real-time information such as wheel rotation angle, wheel speed, and motor speed detected by sensors) through the communication system 20, and assumes that these state information remain unchanged within the next n seconds. Using the kinematic model or dynamic model of the vehicle, and combining the prediction model from the model construction module 120 (specifically, a prediction model updated in real time), it predicts the attitude of the vehicle including the safety zone within the next n seconds.

[0060] Furthermore, the above-mentioned n seconds can be, for example, 1 to 5 seconds. Due to the inertia of the vehicle, the changes in its acceleration, speed, circulation box, etc. are usually small and will not fluctuate violently within a short period of time. Therefore, it is reasonable to assume that the signal remains unchanged within the next 1 to 5 seconds. Generally speaking, the operating frequency of the vehicle body sensor is 50 Hz, that is, the signal is updated every 0.02 seconds. Therefore, the state information of the vehicle's driving is refreshed every 0.02 seconds, and thus the attitude of the vehicle including the safety zone within the next n seconds is re-estimated every 0.02 seconds. Since the frequency of information refreshing is high, the prediction frequency is also high, which can basically be understood as real-time prediction. In addition, within this specific short time window, the interference of environmental factors (such as terrain undulation, wind force, etc.) on the vehicle dynamics can also be ignored. Thus, while ensuring the accuracy of the prediction results, the data complexity can be effectively reduced.

[0061] The collision judgment unit 133 receives the attitude input from the attitude prediction unit 132 and also receives the environmental information from the data processing unit 131, and judges whether the vehicle will collide with the objects in the surrounding environment within the next n seconds. Specifically, the following are input into the collision judgment unit 133: the predicted attitude of the vehicle within the next n seconds and the environmental point cloud data. Among them, the attitude is the future attitude of the vehicle after the safety zone in the model has been superimposed, and the environmental point cloud data can also be data of the point cloud area that may interact with the vehicle further screened according to the future trajectory of the vehicle. Thus, the collision judgment unit 133 performs collision detection on the future attitude of the vehicle and the point cloud based on the input. And according to the judgment result, it outputs corresponding control instructions, such as parking, decelerating, continuing to drive, etc. If it is determined that the vehicle will collide with the objects in the surrounding environment within the next n seconds, for example, it can send control instructions such as decelerating, emergency braking, or steering to avoid to the autonomous driving system 30, and issue an alarm prompt if necessary. If it is determined that the vehicle will not collide with the objects in the surrounding environment within the next n seconds, for example, it can send an instruction to continue the current driving to the autonomous driving system 30.

[0062] Further, the control module 130 may further include a risk assessment unit (not shown). The risk assessment unit receives the judgment result from the collision judgment unit, evaluates the risk level of the collision risk according to the judgment result and the vehicle attitude, and makes corresponding behavior decisions according to the risk level and outputs corresponding control instructions. For example, when the environmental obstacle overlaps with the darker part of the safety area of the vehicle's future attitude, it indicates a high collision risk, that is, a high risk level, and the control module 130 issues an alarm and a forced braking instruction.

[0063] Figure 3 is a flowchart showing an automatic emergency braking method for a vehicle according to an embodiment of the present application. The following will be described with reference to Figure 3 in detail.

[0064] According to the automatic emergency braking method for a vehicle of the present invention, the following steps may be included:

[0065] An environment perception step S11 of collecting environmental data around the vehicle when the vehicle is traveling; a model construction step of constructing a prediction model of the vehicle; and a vehicle control step of performing collision detection on the vehicle and the environmental data based on the prediction model and braking the vehicle when it is determined that a collision will occur.

[0066] In the environment perception step, when perceiving an object, sensors provided on the vehicle are used, and the sensors include at least one single-line lidar and at least one multi-line lidar.

[0067] In the model construction step, the following are constructed: a basic model of the vehicle (step S21); an extended model including a safety area (step S22); and a prediction model for dynamically adjusting the safety area according to the vehicle driving state (step S23).

[0068] The vehicle control step includes: a data processing step S31 of processing the data collected in the environment perception step to obtain environmental information around the vehicle; an attitude prediction step S32 of predicting the attitude of the vehicle in the next n seconds in combination with the prediction model constructed in the model construction step S2; and a collision judgment step S33 of judging whether the vehicle will collide with an object in the surrounding environment in the next n seconds in combination with the attitude and environmental information.

[0069] In the data processing step S31, the data collected in the environment perception step is fused, and the fused data is classified and recognized by using a deep learning model to obtain the object type and contour or boundary in the surrounding environment.

[0070] Further, a deep learning model is used, and an error prevention measure is adopted to classify and recognize the fused data.

[0071] In the attitude prediction step S32, when the vehicle is in the autonomous driving mode, the control quantity instruction for the next n seconds output by the autonomous driving system is combined in the prediction model constructed in the model construction step S2 to generate the attitude of the vehicle in the next n seconds.

[0072] In the attitude prediction step S32, when the vehicle is in the manual driving mode, the real-time state information of the vehicle's travel output by the communication system is combined in the prediction model constructed in the model construction step S2, and it is assumed that the state information remains unchanged within the next n seconds, to generate the attitude of the vehicle in the next n seconds.

[0073] In the collision judgment step S33, collision detection is performed on the speculated attitude of the vehicle in the next n seconds and the object types, contours or boundaries in the surrounding environment obtained, to determine whether a collision will occur. When it is determined that a collision will occur, braking measures are taken. Specifically, during the collision detection, it is judged whether the future attitude of the vehicle will overlap with the contour of the environmental object. If there is an overlap, it is determined that the vehicle will collide with the object in the surrounding environment in the next n seconds. If there is no overlap, it is determined that the vehicle will not collide with the object in the surrounding environment in the next n seconds.

[0074] In addition, a risk assessment step may also be included, based on the judgment result of the collision judgment step S33 and the attitude of the vehicle in the next n seconds, different risk levels are divided. Different braking measures are taken according to different risk levels.

[0075] <Embodiment>

[0076] The above takes a conventional vehicle as an example to illustrate the system and method of an embodiment of the present application, but the present application is not limited thereto. Hereinafter, the present application will be further described by taking an unconventional vehicle as an example.

[0077] As an unconventional-shaped vehicle, a straddle carrier can be cited. The intelligent driving straddle carrier is applied to places such as ports, container terminals, large workshops, and logistics centers, and is used for automatically handling or stacking containers, goods or other materials in the above places. Therefore, the safety guarantee during the intelligent driving process of the straddle carrier is crucial, especially the safety guarantee of the container and spreader during the process of changing the spreader height of the straddle carrier or when the straddle carrier is in the state of carrying a container.

[0078] Figure 4 It is a schematic structural diagram showing a straddle carrier as an embodiment of the present application. The following is combined with Figure 4 to illustrate the basic structure of the straddle carrier 1.

[0079] The straddle carrier 1 mainly includes: a main body 11 that can move on the ground and has an accommodation space inside; and a grasping part 12 installed in the accommodation space of the main body 11. The straddle carrier uses a control module to make the sensor module detect the surrounding environment, and makes the driving module travel to a specified position to accommodate a transport vehicle (which may carry a container) in the accommodation space inside the main body 11, and the grasping part 12 located above the transport vehicle (which may carry a container) grasps or places the container.

[0080] In this embodiment, the main body 11 is in the shape of a gantry, and adopts a wide gauge design (also known as a wide-body structure, WS: Wide-Structure) and a low center of gravity design, thereby having sufficient lateral stability to prevent the vehicle from tilting or rolling over when carrying heavy containers. Specifically, the main body 11 mainly includes: multiple legs 111 and multiple wheels 112 installed at the bottom of the multiple legs 111. Among them, the multiple legs 111 are arranged in two parallel rows to form a portal frame structure, which is used to support the weight of the vehicle and provide stability, and an accommodation space for accommodating a transport vehicle (which may carry a container) is formed inside. The multiple wheels 112 are arranged in two rows and correspondingly provided below the legs 111, and disperse and bear the weight of the whole vehicle while moving. In addition, as Figure 1 shown, the four legs 111 enclose a rectangular accommodation space, and the eight wheels 112 are correspondingly arranged at the bottom of the legs 111. However, the number, configuration, etc. of the legs 111 and the wheels 112 are only examples, and the present application does not make any restrictions.

[0081] The grasping part 12 is slidably installed in the main body 11 relative to the main body 11 in the accommodation space. More specifically, the grasping part 12 is installed in the accommodation space between the legs 111 of the main body 11 in such a way that it can slide relative to the main body 11 at least in the height direction and / or the vehicle length direction (i.e., the specified track), and mainly includes a grasping device such as a spreader that can grasp and lift the container.

[0082] The straddle carrier 1 also includes an automatic emergency braking system, an automatic driving system, a communication system, etc. In addition, multiple sensors installed at various parts of the straddle carrier 1 constitute a sensor module (not shown), and the installation positions of the multiple sensors are not specifically limited. In addition, Figure 1 the state shown is the empty container state where the straddle carrier does not grasp the container. Although the illustration of the straddle carrier in the state of carrying the container is omitted, those skilled in the art can understand it.

[0083] Figure 5 The basic model, extended model and prediction model of the straddle carrier are shown. As Figure 5As shown, the basic model, extended model and prediction model of the straddle carrier are all composed of a main body model constructed based on the main body 11 and a grabbing part model constructed based on the grabbing part 12. These models are generated based on the basic vehicle information of the straddle carrier 1 and the vehicle status information detected in real time, and are stored in the system of the straddle carrier 1 for ready use.

[0084] Figure 5 (a) is a basic model of a straddle carrier, and from left to right are a basic model of a main body 11, a basic model of a grabbing part 12, and a basic model of a straddle carrier combining the main body and the grabbing part. The order of the models is the same and will not be described in detail later.

[0085] Figure 5 (b) is an extended model of the straddle carrier. On the basis of the basic model, independent safety zones are added to the main body and the gripping part. When the main body moves, the safety zones of the main body and the gripping part move accordingly. When the gripping part moves relative to the main body, the safety zone of the gripping part moves accordingly relative to the safety zone of the main body. In addition, the safety zone of the gripping part includes the horizontal safety zone and the vertical safety zone. In other words, for the gripping part, collision prediction during the lifting and lowering of the spreader is equally important.

[0086] Figure 5 (c) in the figure is the prediction model of the straddle carrier. On the basis of the extended model, independent dynamic safety zones are added to the main body and the gripping part, respectively. In this embodiment, the safety zones of the main body and the gripping part can be adjusted in a linked manner or independently according to specific conditions. For example, the safety zones of the main body and the gripping part can be dynamically adjusted according to the vehicle state, the spreader state or the environmental conditions, or the safety zones of the main body or the gripping part can be adjusted separately. For example, when the vehicle speed is low, the safety zone range of the main body and the gripping part can be appropriately reduced. When the height of the spreader after grabbing the container is high, the center of gravity of the straddle carrier as a whole becomes higher, and the safety zone range of the main body and the gripping part can be appropriately increased to avoid rollover caused by short-distance forced braking. When the container contains fragile items, the safety zone range of the main body and the gripping part is increased, the collision judgment threshold is increased, and sufficient braking distance is reserved to avoid damage to the items in the container caused by short-distance forced braking.

[0087] also, Figure 5 This is a diagram of a straddle carrier and its model viewed from above, showing the horizontal safety zone of the gripping part. Since the principle is basically the same, the illustration and description of the vertical safety zone are omitted. By setting a fully wrapped safety zone and dynamic adjustment on the gripping part, the future posture and collision risk of the gripping part in the horizontal and vertical directions can be predicted by using the model of the gripping part. The following further describes the automatic emergency braking method of the straddle carrier 1.

[0088] The straddle carrier 1 receives task instructions from the vehicle management system or the vehicle dispatching system via the communication system, conducts path planning through the autonomous driving system and travels according to the preset path, or the operator manually drives the straddle carrier 1 to perform the task.

[0089] Next, during the driving process, sensors are used to detect the surrounding environment in real time, and obtain the object categories and boundary information in the environment.

[0090] Synchronously, the prediction model is updated in real time through the vehicle motion state and the spreader state, etc. When driving autonomously, according to the control amount of autonomous driving, combined with the latest prediction model, the attitude of the straddle carrier in the next 2 seconds (including the main body part and the grasping part) is calculated. When driving manually, assuming that the state information of the vehicle motion and the spreader motion remains unchanged in the next 2 seconds, combined with the latest prediction model, the attitude of the straddle carrier in the next 2 seconds (including the main body part and the grasping part) is calculated.

[0091] Next, collision detection is performed on the calculated attitude and the detected object categories and boundaries, that is, it is detected whether the safety area in the future attitude overlaps with the obstacles in the surrounding environment. If there is an overlap, it is determined that the straddle carrier will collide with the objects in the surrounding environment in the next 2 seconds, and a deceleration braking instruction is sent to the autonomous driving system, and if necessary, a prompt is given to the operator that emergency braking has been performed. If there is no overlap, it is determined that the straddle carrier will not collide with the objects in the surrounding environment in the next 2 seconds, and an instruction to continue the current driving is sent to the autonomous driving system.

[0092] In summary, the present application proposes an automatic emergency braking system and method based on point cloud data processing and short-term motion prediction. Through an adaptive vehicle model, a fusion algorithm of a multi-line lidar, a deep learning point cloud filtering algorithm, and a vehicle future attitude prediction technology, collision prevention is carried out on the vehicle in the autonomous driving or manual driving mode, aiming to improve the safety and redundancy of the vehicle under different operation modes and load states.

[0093] In addition, a computer-readable storage medium may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. The computer program includes computer program code. The computer program code may be in the form of source code, object code, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0094] In addition, any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed. This should be understood by those skilled in the art to which the embodiments of the present application belong.

[0095] In addition, those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0096] Finally, it should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An automatic emergency braking system for a vehicle, comprising: A sensor module collects environmental data around the vehicle when the vehicle is traveling; A model building module, building a prediction model of the vehicle; as well as The control module performs collision detection between the vehicle and the environmental data based on the prediction model, and brakes the vehicle when it is determined that a collision is likely.

2. The automatic emergency braking system according to claim 1, characterized in that: The sensor module includes a plurality of sensors, and the plurality of sensors are installed on the vehicle in a manner of covering all directions and angles of the vehicle.

3. The automatic emergency braking system according to claim 2, characterized in that: The plurality of sensors are cameras, lidars, laser sensors, ultrasonic sensors and / or combinations thereof.

4. The automatic emergency braking system according to claim 1, characterized in that: The sensor module includes at least one single-line laser radar and at least one multi-line laser radar installed on the vehicle.

5. The automatic emergency braking system according to claim 1, characterized in that: The model building module sequentially builds: a basic model of the vehicle; an extended model including a safety zone; and the prediction model for dynamically adjusting the safety zone according to the driving state of the vehicle.

6. The automatic emergency braking system according to claim 5, characterized in that: In the prediction model, different safety levels are divided for the safety zone, and the safety level is dynamically adjusted according to the driving status of the vehicle.

7. The automatic emergency braking system according to claim 1, characterized in that: The control module comprises: A data processing unit, processing the environmental data to obtain information about obstacles; A posture prediction unit, combining the prediction model to predict the posture of the vehicle in the next n seconds; and The collision judgment unit combines the posture and the information of the obstacle to judge whether the vehicle will collide with the obstacle in the next n seconds.

8. The automatic emergency braking system according to claim 7, characterized in that: In the data processing unit, the data from the multiple sensors are fused, and the type and outline of objects in the surrounding environment are identified as the information of the obstacle.

9. The automatic emergency braking system according to claim 7, characterized in that: The posture prediction unit, When the vehicle is in the automatic driving mode, based on the control amount of the vehicle in the next n seconds, combined with the prediction model, predicting the posture of the vehicle in the next n seconds; When the vehicle is in manual driving mode, based on the real-time state information of the vehicle and assuming that the real-time state information remains unchanged in the next n seconds, the posture of the vehicle in the next n seconds is predicted in combination with the prediction model.

10. The automatic emergency braking system according to claim 7, characterized in that: The n is 1-5.

11. The automatic emergency braking system according to claim 7, characterized in that: The collision determination unit, If it is determined that the vehicle will collide with the obstacle in the next n seconds, a braking command is output; if it is determined that the vehicle will not collide with the obstacle in the next n seconds, a command to continue the current driving is output.

12. The automatic emergency braking system according to claim 11, characterized in that: The collision determination unit detects whether the posture of the vehicle overlaps with the obstacle in the environment, and determines that a collision will occur if the posture overlaps with the obstacle, and determines that a collision will not occur if the posture does not overlap with the obstacle.

13. The automatic emergency braking system according to claim 7, characterized in that: The control module also includes a risk assessment unit, The risk assessment unit classifies the vehicle into different risk levels based on the judgment result of the collision judgment unit and the posture of the vehicle in the next n seconds.

14. The automatic emergency braking system according to any one of claims 1 to 13, characterized in that: The vehicle is a straddle carrier, The straddle carrier comprises: a main body which can move on the ground and has a receiving space inside; and a grabbing part installed in the receiving space of the main body.

15. The automatic emergency braking system according to claim 14, characterized in that: The prediction model is composed of a main body model constructed based on the main body and a grasping part model constructed based on the grasping part.

16. The automatic emergency braking system according to claim 15, characterized in that: The main body model and the gripping part model have independently adjustable safety zones.

17. The automatic emergency braking system according to claim 16, characterized in that: In the prediction model, the safety zone of the gripping part model includes a safety zone in a horizontal direction and a safety zone in a vertical direction. When the gripping unit is in the automatic gripping and releasing mode, based on the control amount of the gripping unit in the next n seconds, combined with the prediction model, predict the posture of the gripping unit in the horizontal direction and the vertical direction in the next n seconds; When the gripping unit is in the manual gripping and releasing mode, based on the real-time status information of the gripping unit operation and assuming that the real-time status information remains unchanged in the next n seconds, combined with the prediction model, the posture of the gripping unit in the horizontal and vertical directions in the next n seconds is predicted.

18. The automatic emergency braking system according to claim 17, characterized in that: The n is 1-5.

19. An automatic emergency braking method for a vehicle, comprising: An environmental perception step, collecting environmental data around the vehicle while the vehicle is traveling; A model building step, building a prediction model for the vehicle; as well as The vehicle control step performs collision detection between the vehicle and the environmental data based on the prediction model, and brakes the vehicle when it is determined that a collision is likely.

20. The automatic emergency braking method according to claim 19, characterized in that: In the environment perception step, The vehicle uses sensors to sense objects in the environment as the environmental data. The sensor includes at least one single-line laser radar and at least one multi-line laser radar.

21. The automatic emergency braking method according to claim 19, characterized in that: The model building steps include: Build the basic model of the vehicle; Based on the basic model, construct an extended model including a safety zone; and Based on the extended model, the prediction model is constructed, and the prediction model includes a safety zone dynamically adjusted according to the driving state of the vehicle.

22. The automatic emergency braking method according to claim 19, characterized in that: The vehicle control step comprises: A data processing step, obtaining information about obstacles based on the environmental data sensed in the environmental sensing step; a posture prediction step, combining the prediction model constructed in the model construction step to predict the future posture of the vehicle; and The collision judgment step combines the future posture and the information of the obstacle to judge whether the vehicle will collide with the obstacle.

23. The automatic emergency braking method according to claim 22, characterized in that: In the data processing step, the environmental data in the environmental perception step is fused, the fused environmental data is classified and identified, and the type and outline of objects in the surrounding environment are obtained as the information of the obstacle.

24. The automatic emergency braking method according to claim 23, characterized in that: In the data processing step, a deep learning model and a clustering analysis algorithm are used to classify and identify the fused environmental data.

25. The automatic emergency braking method according to claim 22, characterized in that: In the posture prediction step, When the vehicle is in an automatic driving mode, predicting a future posture of the vehicle based on a control amount of the vehicle and the prediction model; When the vehicle is in a manual driving mode, the future posture of the vehicle is predicted based on the real-time state quantity of the vehicle and the prediction model, wherein the real-time state quantity remains unchanged for a period of time in the future.

26. The automatic emergency braking method according to claim 22, characterized in that: In the collision determination step, If it is determined that the vehicle will collide with the obstacle in the future, a braking instruction is output; if it is determined that the vehicle will not collide with the obstacle in the future, an instruction to continue the current driving is output.

27. The automatic emergency braking method according to claim 26, characterized in that: In the collision determination step, It is detected whether the posture of the vehicle overlaps with the obstacle in the environment, and if so, it is determined that a collision will occur, and if not, it is determined that no collision will occur.

28. The automatic emergency braking method according to claim 22, characterized in that: It also includes a risk assessment step, In the risk assessment step, different risk levels are divided based on the judgment result of the collision judgment step and the future posture of the vehicle.

Images