A method and device for automatic emergency braking redundant decision-making based on neural network

Abstract

The present invention relates to a method and device for automatic emergency braking redundant decision-making based on a neural network, the method comprising: respectively acquiring the data of a radar and a visual sensor installed on the own vehicle in real time, and performing target fusion on them; based on the data of the real-time visual sensor and the pedestrian target, using the trained first neural network to issue a first braking decision to the own vehicle; based on the target fused data, identifying pedestrians and vehicles therein, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; fusing the first braking decision, the second braking decision and the third braking decision, and issuing a braking decision to the braking system of the own vehicle. The present invention combines the traditional decision algorithm with the decision algorithm based on the neural network, the two sets of decision algorithms are independent and parallel, and the braking demand is comprehensively judged, which can reduce the false triggering of AEBS and increase the robustness and reliability of the algorithm system.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle assisted driving, and in particular relates to a neural network-based automatic emergency braking redundant decision-making method and device. Background Art

[0002] The automatic emergency braking system is an assisted driving technology. When the object in front of the vehicle meets the collision conditions and the driver's behavior cannot avoid the collision, the AEBS (Advanced Emergency Braking System) will control the vehicle to brake automatically to reduce the harm of the collision.

[0003] At present, most automatic emergency braking decisions are based on a set of rule-based physics strategies (Newton's first law and / or second law). First, the collision time is calculated according to the formula, then the most dangerous target is selected based on the collision time, and finally the collision time of this target and other parameters are used to determine whether to issue deceleration for braking. However, the sensor is not always absolutely accurate after identifying the target and converting it into position and speed information. Therefore, subsequent decisions will also be inaccurate due to measurement errors and problems in judging the timing of braking. The decision-making of neural networks is end-to-end, without a clear measurement process, and is completely based on probability. Combining it with traditional decision-making algorithms can improve the accuracy of decisions. Summary of the invention

[0004] In order to improve the accuracy and redundancy of emergency braking decision-making in autonomous driving technology, a first aspect of the present invention provides an automatic emergency braking redundant decision-making method based on a neural network, comprising:

[0005] The data of the radar and visual sensor installed on the own vehicle are respectively acquired in real time and target fused; based on the real-time visual sensor data and pedestrian targets, a first braking decision is issued to the own vehicle using a trained first neural network; pedestrians and vehicles are identified based on the target fused data, and a second braking decision and a third braking decision are respectively made according to the pedestrians and vehicles through target selection and TTC algorithm; the first braking decision, the second braking decision and the third braking decision are fused, and a braking decision is issued to the braking system of the own vehicle.

[0006] In some embodiments of the present invention, the first neural network is trained by the following method: obtaining data of the vehicle at different braking triggers and its corresponding visual images, and constructing a training data set based thereon; using the visual images in the training data set as samples and the corresponding braking decisions as labels, and training the first neural network until its error is lower than a threshold and tends to be stable, thereby obtaining a trained first neural network.

[0007] Furthermore, the first neural network includes multiple convolutional layers and multiple fully connected layers, and the convolution kernel size of each convolutional layer is 5*5 and the step size is 3.

[0008] In some embodiments of the present invention, the identifying of pedestrians and vehicles based on the target fused data, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm include: identifying pedestrians and vehicles based on the target fused data: if the target is a pedestrian, then: calculating the collision time between each target and the own vehicle using a collision algorithm, and automatically making a second braking decision; or making a second braking decision using a second neural network completed through training; if the target is a vehicle, calculating the deceleration required for braking of the own vehicle according to the collision algorithm or kinematic model, and making a third braking decision based on it.

[0009] Furthermore, the second neural network is trained using the same training method as the first neural network.

[0010] In the above-mentioned embodiment, the real-time acquisition of the data of the radar and visual sensor installed on the vehicle and the target fusion thereof include: clustering the acquired radar point cloud data, and classifying the targets according to the target reflectivity, size and speed to obtain one or more first targets; using the target recognition algorithm to recognize the image acquired by the visual sensor to obtain one or more second targets; matching each first target with each second target, and merging and fusing the same target.

[0011] The second aspect of the present invention provides an automatic emergency braking redundant decision-making device based on a neural network, including: an acquisition module, used to respectively acquire in real time the data of the radar and visual sensor installed on the own vehicle, and perform target fusion on them; a decision module, used to issue a first braking decision to the own vehicle based on the real-time visual sensor data and pedestrian targets using a trained first neural network; identify pedestrians and vehicles based on the target fused data, and make a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; a fusion module, used to fuse the first braking decision, the second braking decision and the third braking decision, and issue a braking decision to the braking system of the own vehicle.

[0012] Furthermore, the decision module includes: a first decision unit, which is used to issue a first braking decision to the vehicle itself based on the data of the real-time visual sensor and the pedestrian target by using the trained first neural network; a second decision unit, which is used to identify the pedestrians and vehicles based on the target fused data, and make the second braking decision and the third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm.

[0013] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic emergency braking redundant decision method based on neural network provided in the first aspect of the present invention.

[0014] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the automatic emergency braking redundant decision method based on a neural network provided in the first aspect of the present invention is implemented.

[0015] The beneficial effects of the present invention are:

[0016] The present invention discloses a redundant decision method and device for automatic emergency braking based on a neural network, the method comprising: respectively acquiring the data of a radar and a visual sensor installed on the own vehicle in real time, and performing target fusion on them; based on the data of the real-time visual sensor and the pedestrian target, using the trained first neural network to issue a first braking decision to the own vehicle; identifying pedestrians and vehicles based on the target fused data, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; fusing the first braking decision, the second braking decision and the third braking decision, and issuing a braking decision to the braking system of the own vehicle. The present invention combines the traditional decision algorithm with the decision algorithm based on the neural network, the two sets of decision algorithms are independent and parallel, and the braking demand is comprehensively judged, which can reduce the false triggering of AEBS and increase the robustness and reliability of the algorithm system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A basic flow chart of a neural network-based automatic emergency braking redundant decision method in some embodiments of the present invention;

[0018] Figure 2 A schematic diagram of a specific flow chart of a neural network-based automatic emergency braking redundant decision method in some embodiments of the present invention;

[0019] Figure 3 is a schematic diagram of the structure of a first neural network in some embodiments of the present invention;

[0020] Figure 4 is one of the geometrical schematic diagrams of the kinematic model in some embodiments of the present invention;

[0021] Figure 5 The second geometrical schematic diagram of the kinematic model in some embodiments of the present invention;

[0022] Figure 6 The third geometrical schematic diagram of the kinematic model in some embodiments of the present invention;

[0023] Figure 7 FIG4 is a fourth geometrical schematic diagram of a kinematic model in some embodiments of the present invention;

[0024] Figure 8 It is a structural schematic diagram of an automatic emergency braking redundant decision-making device based on a neural network in some embodiments of the present invention;

[0025] Fig. 9 It is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. DETAILED DESCRIPTION

[0026] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0027] refer to Figure 1 In a first aspect of the present invention, a neural network-based automatic emergency braking redundant decision method is provided, comprising: S100. acquiring real-time data from a radar and a visual sensor installed on the own vehicle respectively, and performing target fusion on the data; S200. issuing a first braking decision to the own vehicle using a trained first neural network based on the real-time visual sensor data and pedestrian targets; identifying pedestrians and vehicles based on the target fused data, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; S300. fusing the first braking decision, the second braking decision and the third braking decision, and issuing a braking decision to the braking system of the own vehicle.

[0028] It can be understood that the decision of the neural network is end-to-end, and there is no clear measurement process, nor a clear decision-making process for judging the braking timing. It is a behavior based entirely on statistical probability; combining it with the traditional decision algorithm can improve the accuracy of the decision as a whole, avoid false triggering with a greater probability, and thus reduce the probability of false triggering. The braking decision involved in this embodiment includes but is not limited to determining the driving parameters of the own vehicle or the target vehicle such as the moment of braking, the speed at the moment of braking, the acceleration at the moment of braking, the duration of braking, the maximum braking deceleration or the following time. That is: steering manipulation, speed control, and acceleration control used to avoid collision with the vehicle in front are prohibited. Specifically, steering manipulation / steering manipulation suppression, speed control / speed control suppression, acceleration / deceleration control operations are performed in order to avoid collision, but the above operations are not performed in order to avoid collision with the vehicle in front. Based on the fusion information and collision detection of one or more different types of sensors, the posture information of the target vehicle is used to perform steering manipulation, speed control, and acceleration control of the own vehicle. In contrast, when the sensor or fusion information is inaccurate or insufficient, new steering maneuvers cannot be controlled based on the correct vehicle posture when the sensor detects an abnormality, so there is a possibility of performing dangerous steering maneuvers, resulting in unexpected vehicle movement and collision with obstacles such as other vehicles.

[0029] refer to Figure 2 In step S100 of the following embodiment, the real-time acquisition of the data of the radar and visual sensor installed on the vehicle and the target fusion thereof include: S100. Clustering the acquired radar point cloud data, and classifying the targets according to the target reflectivity, size and speed to obtain one or more first targets; S200. Using the target recognition algorithm to recognize the image acquired by the visual sensor to obtain one or more second targets; S300. Matching each first target with each second target, and merging and fusing the same target.

[0030] It should be understood that the visual sensor is the direct source of information for the entire machine vision system, and is mainly composed of one or two graphic sensors, and sometimes equipped with a light projector and other auxiliary equipment. The main function of the visual sensor is to obtain enough of the most original images to be processed by the machine vision system. The image sensor can use a laser scanner, a linear array and an area array CCD camera, or a TV camera, a digital camera, or a visual sensor module on an intelligent terminal device. The radar involved in this embodiment can be divided into over-the-horizon radar, microwave radar, millimeter-wave radar, and laser radar according to the radar frequency band; according to the angular tracking method, there are single-pulse radar, cone scanning radar, and hidden cone scanning radar.

[0031] In step S200 of some embodiments of the present invention, the first neural network is trained by the following method: obtaining data of the vehicle at different braking triggers and its corresponding visual images, and constructing a training data set based thereon; using the visual images in the training data set as samples and the corresponding braking decisions as labels, training the first neural network until its error is lower than a threshold and tends to be stable, thereby obtaining a trained first neural network. It is necessary and explained that the sample image generally refers to an image containing a pedestrian.

[0032] refer to Figure 3 Specifically, the network structure adopts 5 convolutional layers and 3 fully connected layers. The convolution kernel size of each convolutional layer is 5*5, and the step length is 3step. The last layer of the fully connected layer is divided into two outputs after normalization. The ideal situation is that when A is set to 1 and B is set to 0 during training, there is a braking request; when A is set to 0 and B is set to 1, there is no braking request (normalization has been performed); when A>0.9 is actually processed, the braking request is considered to be true. The complete training method is:

[0033] 1. Data acquisition: A large number of road tests will be conducted before mass production. The data of pedestrian braking triggered by traditional algorithms will be recorded, and the triggering results will be corrected based on later simulations and manual screening to obtain a large number of training sets.

[0034] 2. Classification of training sets: The training sets are divided into two categories, the first category is the data when the braking request for pedestrians is triggered, and the second category is when the braking request is not triggered;

[0035] 3. Training: The input is the camera data in the training set, and the output is whether to brake (compared with the true value of the training set to optimize the network parameters);

[0036] 4. Verification: Verify based on the test set to adjust the loss function, activation function and learning rate.

[0037] refer to Figure 2 In step S200 of some embodiments of the present invention, the pedestrians and vehicles are identified based on the target fused data, and the second braking decision and the third braking decision are made respectively according to the pedestrians and vehicles through target selection and TTC algorithm, including: S201. Pedestrians and vehicles are identified based on the target fused data: if the target is a pedestrian, then: the collision algorithm is used to calculate the collision time between each target and the vehicle itself, and the second braking decision is automatically made; or the second neural network completed through training is used to make the second braking decision; S202. If the target is a vehicle, the collision algorithm is used to make the third braking decision.

[0038] See also Figures 4 to 7,Specifically, the kinematic model includes a vehicle kinematic model with the rear axle as the origin, a vehicle kinematic model with the center of mass, a front-wheel drive vehicle kinematic model and Ackerman steering geometry;

[0039] 1. Vehicle motion model with the rear axle as the origin: The vehicle model in autonomous driving can be simplified as a rigid body structure moving on a two-dimensional plane. The state of the vehicle at any time is q = (x, y, θ). The origin of the vehicle coordinates is located at the center of the rear axle, and the coordinate axis is parallel to the vehicle body. s represents the speed of the vehicle, φ represents the steering angle (left is positive, right is negative), and L represents the distance between the front and rear wheels. If the steering angle remains unchanged, the vehicle will circle in place with a radius of ρ. Vehicle motion model: In a very short time Δt, it can be approximately considered that the vehicle moves in the direction of the vehicle body. x d y Indicates that in d t The distance the vehicle moves on the x-axis and y-axis within a certain period of time.

[0040] 2. Vehicle kinematic model centered on the center of mass

[0041] Among them, point A is the front wheel, point B is the rear wheel, C is the center of mass of the vehicle, O is the intersection of OA and OB, which is the instantaneous rolling center of the vehicle, and line segments OA and OB are perpendicular to the directions of the two rolling wheels respectively; β is the slip angle (TireSlip Angle), which refers to the angle between the vehicle's moving direction and the direction indicated by the wheel rim; ψ is the heading angle (HeadingAngle), which refers to the angle between the vehicle body and the X-axis.

[0042] 3. When the vehicle is front-wheel-only, we can assume that the transfer fails and re-upload to cancel the constant value of 0. At the same time, since we assume that the car is front-wheel-drive, we assume that the steering wheel angle is equal to the front wheel angle. At this time, the vehicle kinematics formula is as follows:

[0043]

[0044] x, y, V, ψ, v are pose parameters.

[0045] 4. Ackerman Turning Geometry is a geometry designed to solve the problem that the inner and outer steering wheel paths point to different centers when a vehicle turns. In a single-vehicle model, the left / right front wheel deflection angles are assumed to be the same when turning. Although the two angles are usually roughly equal, they are not actually the same. Usually, the inner tire has a larger turning angle. When the rear axle center is taken as the reference point, the red line in the figure is the turning radius R. Therefore, the difference in the steering angles of the two front wheels is proportional to the square of the average steering angle. When a vehicle designed according to Ackerman steering geometry turns along a curve, the equal cranks of the four-bar linkage are used to make the steering angle of the inner wheel approximately 2 to 4 degrees larger than that of the outer wheel, so that the centers of the four wheel paths roughly intersect the instantaneous turning center on the extension line of the rear axle, allowing the vehicle to turn smoothly.

[0046] It can be understood that the collision algorithm TTC (Time-To-Collision) TTC is defined as: "The time required for a collision if the ego vehicle and the target continue to move at their current speed and the same path". In the study of traffic conflict technology, TTC has been proven to be an effective means of measuring the severity of traffic conflicts and distinguishing critical behaviors from normal behaviors. The results of some studies point to the direct use of TTC as a clue for traffic decision-making. The prediction of future interactions between the ego vehicle and the target involves creating predicted trajectories for the subject vehicle and all vehicles that may interact to see if a collision may occur. In the TTC algorithm, vehicles are treated as two-dimensional planes. Each vehicle is represented by a rectangle located at a specific position in the plane. Each vehicle has a speed and acceleration, and both speed and acceleration are vectors. Each "subject" vehicle will interact with nearby vehicles, and there is no leading vehicle or following vehicle. The actions of the subject vehicle follow three rules: 1. Follow the vehicle in front; 2. Avoid collision; 3. Adjust the intensity of the action taken based on the value of TTC.

[0047] Furthermore, the second neural network is trained using the same training method as the first neural network.

[0048] In step S400 of some embodiments of the present invention, the fusion method is as follows: the decision making method of pedestrians is adopted; and for vehicles and pedestrians, as long as one of them meets the braking condition, braking is performed.

[0049] In summary, the overall decision-making process of this embodiment first performs target recognition based on the original data of the radar camera and classifies the target type. Generally speaking, due to the relationship between the target speed, size and reflectivity, the target recognition algorithm is more accurate in identifying vehicles, but has a weaker ability and low accuracy in identifying pedestrians. Therefore, for vehicles, the decision-making process is only based on traditional decision-making algorithms, while for pedestrians, a combination of traditional decision-making algorithms and a neural network-based decision-making algorithm on the right is used, and finally a fusion output of braking requests and acceleration requests is performed.

[0050] Example 2

[0051] refer to Figure 8 According to a second aspect of the present invention, a redundant decision-making device 1 for automatic emergency braking based on a neural network is provided, comprising: an acquisition module 11, for respectively acquiring data from a radar and a visual sensor installed on the own vehicle in real time, and performing target fusion on the data; a decision module 12, for issuing a first braking decision to the own vehicle using a trained first neural network based on the real-time visual sensor data and pedestrian targets; identifying pedestrians and vehicles based on the target fused data, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; a fusion module 13, for fusing the first braking decision, the second braking decision and the third braking decision, and issuing a braking decision to the braking system of the own vehicle.

[0052] Furthermore, the decision module 12 includes: a first decision unit, which is used to issue a first braking decision to the vehicle itself based on the data of the real-time visual sensor and the pedestrian target by using the trained first neural network; a second decision unit, which is used to identify the pedestrians and vehicles based on the target fused data, and make the second braking decision and the third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm.

[0053] Example 3

[0054] refer to Fig. 9 According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic emergency braking redundant decision method based on a neural network according to the first aspect of the present invention.

[0055] The electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0056] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Fig. 9 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0057] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus, or a device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0058] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more computer programs. When the one or more programs are executed by the electronic device, the electronic device:

[0059] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, Python, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0060] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A redundant decision-making method for automatic emergency braking based on neural network, It is characterized in that include: Acquire the data from the radar and visual sensors installed on the vehicle in real time and fuse them into targets; Based on the data of the real-time visual sensor and the pedestrian target, the trained first neural network is used to issue a first braking decision to the vehicle itself; based on the target fused data, pedestrians and vehicles are identified, and according to the pedestrians and vehicles, the second braking decision and the third braking decision are made respectively through target selection and TTC algorithm; The first braking decision, the second braking decision and the third braking decision are integrated to issue a braking decision to the braking system of the own vehicle; the integration method is as follows: the decision on pedestrians is made by using the method of and; and for vehicles and pedestrians, as long as one of them meets the braking condition, braking is performed; The identifying of pedestrians and vehicles based on the target fused data, and making a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm include: Identify pedestrians and vehicles based on the fused data of the target: If the target is a pedestrian, then: use the TTC algorithm to calculate the collision time between each target and the vehicle itself, and automatically make a second braking decision; If the target is a vehicle, the deceleration required for braking the own vehicle is calculated according to the TTC algorithm, and the third braking decision is made based on it.

2. The automatic emergency braking redundant decision-making method based on a neural network according to claim 1, It is characterized in that The first neural network is trained by the following method: Obtain vehicle data at different braking triggering times and their corresponding visual images, and construct a training dataset based on them; The visual images in the training data set are used as samples and the corresponding braking decisions are used as labels to train the first neural network until its error is lower than a threshold and tends to be stable, thereby obtaining a trained first neural network.

3. The automatic emergency braking redundant decision-making method based on neural network according to claim 2, It is characterized in that The first neural network includes multiple convolutional layers and multiple fully connected layers, and the convolution kernel size of each convolutional layer is 5*5 and the step size is 3.

4. The automatic emergency braking redundant decision method based on a neural network according to any one of claims 1 to 3, It is characterized in that The real-time acquisition of data from the radar and visual sensor installed on the vehicle and target fusion thereof include: Clustering the acquired radar point cloud data, and classifying the targets according to their reflectivity, size, and speed to obtain one or more first targets; Using a target recognition algorithm to recognize the image acquired by the visual sensor, and obtaining one or more second targets; Each first target is matched with each second target, and the same targets are merged and fused.

5. A neural network-based automatic emergency braking redundant decision-making device using the neural network-based automatic emergency braking redundant decision-making method according to any one of claims 1 to 4, It is characterized in that include: The acquisition module is used to acquire the data of the radar and visual sensors installed on the vehicle in real time and perform target fusion; A decision module is used to issue a first braking decision to the vehicle based on the data of the real-time visual sensor and the pedestrian target by using the trained first neural network; identify pedestrians and vehicles based on the target fused data, and make a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm; The fusion module is used to fuse the first braking decision, the second braking decision and the third braking decision, and issue a braking decision to the braking system of the own vehicle.

6. The automatic emergency braking redundant decision-making device based on neural network according to claim 5, It is characterized in that The decision module comprises: A first decision unit is used to issue a first braking decision to the own vehicle based on the data of the real-time visual sensor and the pedestrian target by using the trained first neural network; The second decision unit is used to identify pedestrians and vehicles based on the target fused data, and make a second braking decision and a third braking decision respectively according to the pedestrians and vehicles through target selection and TTC algorithm.

7. An electronic device, include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic emergency braking redundant decision-making method based on a neural network as described in any one of claims 1 to 4.

8. A computer readable medium having a computer program stored thereon, in, When the computer program is executed by a processor, the neural network-based automatic emergency braking redundant decision-making method according to any one of claims 1 to 4 is implemented.

Images