Automatic emergency braking control methods, systems, equipment, media and procedures products
By integrating acoustic and wireless sensing data into the traditional AEB system, multimodal fusion target perception information is generated, which solves the problem of trigger accuracy in non-line-of-sight blind spot scenarios of the traditional AEB system and improves the active safety performance of the vehicle.
Patent Information
- Application Number
- CN202611524491.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-29
- Publication Date
- 2026-11-10
AI Technical Summary
Traditional AEB systems have low accuracy in triggering automatic emergency braking in non-line-of-sight blind spots caused by obstacles, resulting in poor active safety performance of vehicles.
By acquiring acoustic signal information and channel state information within the vehicle's non-line-of-sight blind spot, acoustic sensing data and wireless sensing data are generated and fused to generate multimodal fused target sensing information, and an automatic emergency braking control command is output.
It improves the reliability of target detection and perception results in non-line-of-sight blind spot scenarios, enhances the triggering accuracy and control stability of automatic emergency braking, and improves the active safety performance of vehicles.
Smart Images

Figure CN123061328A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of active safety control technology for vehicles, and in particular to an automatic emergency braking control method, system, device, medium, and program product. Background Technology
[0002] With the rapid popularization of intelligent connected vehicles and the continuous improvement of active safety requirements in the New Car Assessment Program (NCAP), the Automatic Emergency Braking (AEB) system has become a standard safety feature in passenger cars, commercial vehicles, etc., playing an irreplaceable role in effectively avoiding or mitigating collisions between vehicles and pedestrians or non-motorized vehicles.
[0003] Currently, traditional AEB systems typically rely on line-of-sight sensors, including visible light cameras, millimeter-wave radar, lidar, and ultrasonic radar. Specifically, traditional AEB systems detect and locate obstacles such as pedestrians and non-motorized vehicles based on data collected by these line-of-sight sensors, and then execute automatic emergency braking control based on the target detection and location results.
[0004] However, the inventors discovered that when using the traditional AEB system for automatic emergency braking control in scenarios with non-visual blind spots caused by obstacles, there is a problem of low accuracy in triggering automatic emergency braking, resulting in poor active safety performance of the vehicle. Summary of the Invention
[0005] This invention provides an automatic emergency braking control method, system, device, medium, and program product to solve the problem that the accuracy of automatic emergency braking triggering is low when using the above-mentioned traditional AEB system for braking control in non-line-of-sight blind spot scenarios caused by obstacles, resulting in poor vehicle active safety performance.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] In a first aspect, the present invention provides an automatic emergency braking control method, comprising: acquiring multimodal environmental information within a non-line-of-sight blind spot corresponding to the vehicle, the multimodal environmental information including acoustic signal information and channel state information; generating acoustic sensing data corresponding to a first collision target within the non-line-of-sight blind spot based on the acoustic signal information, and generating wireless sensing data corresponding to a second collision target within the non-line-of-sight blind spot based on the channel state information; in response to the first collision target and the second collision target being the same collision target, fusing the acoustic sensing data and the wireless sensing data to generate multimodal fused target sensing information; and outputting an automatic emergency braking control command based on the multimodal fused target sensing information, the acoustic sensing data, and the wireless sensing data, and executing the automatic emergency braking control command.
[0008] In one possible implementation, the multimodal environment information also includes visual perception information. The acoustic perception data and wireless perception data are fused to generate multimodal fused target perception information, including: generating target acoustic scene confidence and target wireless scene confidence based on visual perception information and acoustic signal information; and fusing the acoustic perception data and wireless perception data based on the target acoustic scene confidence and target wireless scene confidence to generate multimodal fused target perception information.
[0009] In one possible implementation, the acoustic sensing data includes an initial acoustic confidence level corresponding to a first collision target, and the wireless sensing data includes an initial wireless confidence level corresponding to a second collision target. Based on the target acoustic scene confidence level and the target wireless scene confidence level, the acoustic sensing data and the wireless sensing data are fused to generate multimodal fused target sensing information. This includes: correcting the initial acoustic confidence level and the initial wireless confidence level respectively to obtain corrected acoustic confidence level and corrected wireless confidence level; fusing the corrected acoustic confidence level and the corrected wireless confidence level based on the target acoustic scene confidence level and the target wireless scene confidence level to generate fused acoustic confidence level and fused wireless confidence level; and fusing the acoustic sensing data and the wireless sensing data based on the fused acoustic confidence level and the fused wireless confidence level to generate multimodal fused target sensing information.
[0010] In one possible implementation, the acoustic sensing data further includes the acoustic lateral distance between the first collision target and the vehicle, and the acoustic lateral velocity corresponding to the first collision target; the wireless sensing data further includes the wireless lateral distance between the second collision target and the vehicle, and the wireless lateral velocity corresponding to the second collision target. Based on the fused acoustic confidence and fused wireless confidence, the acoustic sensing data and the wireless sensing data are fused to generate multimodal fused target sensing information, including: fusing the acoustic lateral distance and the wireless lateral distance based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral distance; fusing the acoustic lateral velocity and the wireless lateral velocity based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral velocity; and fusing the corrected acoustic confidence and the corrected wireless confidence based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target confidence.
[0011] In one possible implementation, generating a target acoustic scene confidence score and a target wireless scene confidence score based on visual perception information and acoustic signal information includes: determining the target driving scene type of the vehicle based on visual perception information and acoustic signal information; determining the target acoustic scene confidence score based on the target driving scene type based on a first mapping relationship between the driving scene type and the acoustic scene confidence score; and calculating the target wireless scene confidence score based on the target acoustic scene confidence score.
[0012] In one possible implementation, an automatic emergency braking control command is output based on multimodal fusion of target perception information, acoustic perception data, and wireless perception data. This includes: determining whether the collision target is valid based on the multimodal fusion of target perception information, acoustic perception data, and wireless perception data; if the collision target is valid, then outputting an automatic emergency braking control command based on the multimodal fusion of target perception information; if the collision target is invalid, then not outputting an automatic emergency braking control command.
[0013] In one possible implementation, an automatic emergency braking control command is output based on multimodal fusion target perception information, including: calculating a first collision time when the colliding target reaches a first collision point based on the multimodal fusion target lateral distance and multimodal fusion target lateral velocity in the multimodal fusion target perception information, wherein the first collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the colliding target; determining the first target collision time based on the first collision time, a second collision time, and the multimodal fusion target confidence level in the multimodal fusion target perception information, wherein the second collision time is the collision time when the vehicle reaches the first collision point; and outputting an automatic emergency braking control command based on the first target collision time.
[0014] In one possible implementation, an automatic emergency braking control command is output based on the first target collision time, including: determining the target risk level based on the first target collision time according to a second mapping relationship between the collision time and the risk level; and outputting the automatic emergency braking control command based on the target risk level.
[0015] In one possible implementation, the first target collision time is determined based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information. This includes: obtaining an intermediate collision time based on the first collision time and the second collision time; and correcting the intermediate collision time based on the multimodal fusion target confidence, the road surface adhesion coefficient, and the scene correction coefficient to obtain the first target collision time.
[0016] In one possible implementation, the acoustic sensing data further includes the acoustic motion direction of the first collision target, and the wireless sensing data further includes the wireless motion direction corresponding to the second collision target. Based on the multimodal fusion target sensing information, the acoustic sensing data, and the wireless sensing data, it is determined whether the collision target is valid, including: determining whether the confidence level of the multimodal fusion target in the multimodal fusion target sensing information is greater than a preset confidence threshold, and determining whether the acoustic motion direction is consistent with the wireless motion direction; if the confidence level of the multimodal fusion target is greater than the preset confidence threshold, and the acoustic motion direction is consistent with the wireless motion direction, then the collision target is determined to be valid; if the confidence level of the multimodal fusion target is less than or equal to the preset confidence threshold, or the acoustic motion direction is inconsistent with the wireless motion direction, then the collision target is determined to be invalid.
[0017] In one possible implementation, after generating acoustic sensing data and wireless sensing data, the automatic emergency braking control method further includes: performing time alignment processing on the acoustic sensing data and wireless sensing data respectively to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data; and performing spatial alignment processing on the time-aligned acoustic sensing data and time-aligned wireless sensing data respectively based on the vehicle's on-board coordinate system to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
[0018] In one possible implementation, the acoustic signal information includes time-domain waveform information, frequency-domain frequency information, and acoustic signal quality feature information. Based on the acoustic signal information, acoustic perception data corresponding to the first collision target within the non-line-of-sight occlusion blind zone is generated, including: extracting the occlusion-penetrating acoustic signal corresponding to the first collision target based on the frequency-domain frequency information; calculating the propagation delay difference corresponding to the first collision target based on the occlusion-penetrating acoustic signal; obtaining the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference; estimating the acoustic lateral motion velocity corresponding to the first collision target based on the time-domain waveform information of the occlusion-penetrating acoustic signal; and determining the initial acoustic confidence level corresponding to the first collision target based on the acoustic signal quality feature information.
[0019] In one possible implementation, the channel state information includes signal amplitude fluctuations, signal phase changes, and multipath interference information. Based on the channel state information, wireless sensing data corresponding to the second collision target within the non-line-of-sight blind zone is generated, including: constructing a motion index corresponding to the second collision target based on signal amplitude fluctuations and signal phase changes; obtaining the wireless lateral distance and wireless lateral motion speed corresponding to the second collision target based on the motion index; and determining the initial wireless confidence level corresponding to the second collision target based on signal phase changes and multipath interference information.
[0020] In one possible implementation, the automatic emergency braking control method further includes: in response to the fact that the first collision target and the second collision target are not the same collision target, determining the collision time of the second target based on acoustic sensing data and wireless sensing data; and outputting an automatic emergency braking control command based on the collision time of the second target.
[0021] In one possible implementation, determining the collision time of the second target based on acoustic sensing data and wireless sensing data includes: calculating a third collision time for the first collision target to reach the second collision point based on the acoustic lateral distance and acoustic lateral velocity corresponding to the first collision target in the acoustic sensing data, where the second collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the first collision target; calculating a fourth collision time for the second collision target to reach the third collision point based on the wireless lateral distance and wireless lateral velocity corresponding to the second collision target in the wireless sensing data, where the third collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the second collision target; and taking the smaller of the third and fourth collision times to obtain the collision time of the second target.
[0022] In one possible implementation, during the execution of the automatic emergency braking control command, the automatic emergency braking control method further includes: acquiring the vehicle's attitude information, including yaw rate and sideslip angle; determining whether the yaw rate is less than or equal to a yaw rate threshold and whether the sideslip angle is less than or equal to a sideslip angle threshold; if the yaw rate is less than or equal to the yaw rate threshold and the sideslip angle is less than or equal to the sideslip angle threshold, then the automatic emergency braking control command is continuously executed; if the yaw rate is greater than the yaw rate threshold or the sideslip angle is greater than the sideslip angle threshold, then a stability protection mechanism is activated.
[0023] In a second aspect, the present invention provides an automatic emergency braking control device, comprising:
[0024] The acquisition module is used to acquire multimodal environmental information within the non-line-of-sight blind spot corresponding to the vehicle. This multimodal environmental information includes acoustic signal information and channel state information.
[0025] The generation module is used to generate acoustic sensing data corresponding to the first collision target in the non-line-of-sight occlusion blind zone based on acoustic signal information, and to generate wireless sensing data corresponding to the second collision target in the non-line-of-sight occlusion blind zone based on channel state information.
[0026] The fusion processing module is used to fuse acoustic sensing data and wireless sensing data in response to the first collision target and the second collision target being the same collision target, and generate multimodal fused target sensing information.
[0027] The control module is used to output automatic emergency braking control commands based on multimodal fusion target perception information, acoustic perception data and wireless perception data, and to execute the automatic emergency braking control commands.
[0028] In one possible implementation, the multimodal environment information also includes visual perception information. The fusion processing module is specifically used to: generate target acoustic scene confidence and target wireless scene confidence based on visual perception information and acoustic signal information; and perform fusion processing on acoustic perception data and wireless perception data based on target acoustic scene confidence and target wireless scene confidence to generate multimodal fused target perception information.
[0029] In one possible implementation, the acoustic sensing data includes an initial acoustic confidence level corresponding to the first collision target, and the wireless sensing data includes an initial wireless confidence level corresponding to the second collision target. The fusion processing module is further configured to: correct the initial acoustic confidence level and the initial wireless confidence level respectively to obtain corrected acoustic confidence level and corrected wireless confidence level; perform fusion processing on the corrected acoustic confidence level and corrected wireless confidence level based on the target acoustic scene confidence level and the target wireless scene confidence level to generate fused acoustic confidence level and fused wireless confidence level; and perform fusion processing on the acoustic sensing data and the wireless sensing data based on the fused acoustic confidence level and the fused wireless confidence level to generate multimodal fused target sensing information.
[0030] In one possible implementation, the acoustic sensing data further includes the acoustic lateral distance between the first collision target and the vehicle and the acoustic lateral velocity corresponding to the first collision target; the wireless sensing data further includes the wireless lateral distance between the second collision target and the vehicle and the wireless lateral velocity corresponding to the second collision target; the fusion processing module is further configured to: fuse the acoustic lateral distance and the wireless lateral distance based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral distance; fuse the acoustic lateral velocity and the wireless lateral velocity based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral velocity; and fuse the corrected acoustic confidence and the corrected wireless confidence based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target confidence.
[0031] In one possible implementation, the fusion processing module is further configured to: determine the target driving scenario type of the vehicle based on visual perception information and acoustic signal information; determine the target acoustic scenario confidence based on the target driving scenario type based on a first mapping relationship between the driving scenario type and the acoustic scenario confidence; and calculate the target wireless scenario confidence based on the target acoustic scenario confidence.
[0032] In one possible implementation, the control module is specifically used to: determine whether the collision target is valid based on the multimodal fusion target perception information, acoustic perception data, and wireless perception data; if the collision target is valid, output an automatic emergency braking control command based on the multimodal fusion target perception information; if the collision target is invalid, do not output an automatic emergency braking control command.
[0033] In one possible implementation, the control module is further configured to: calculate the first collision time of the colliding target reaching the first collision point based on the lateral distance and lateral velocity of the multimodal fusion target in the multimodal fusion target perception information, wherein the first collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the colliding target; determine the first target collision time based on the first collision time, the second collision time, and the confidence level of the multimodal fusion target in the multimodal fusion target perception information, wherein the second collision time is the collision time of the vehicle reaching the first collision point; and output an automatic emergency braking control command based on the first target collision time.
[0034] In one possible implementation, the control module is further configured to: determine the target risk level based on the first target collision time according to a second mapping relationship between the collision time and the risk level; and output an automatic emergency braking control command according to the target risk level.
[0035] In one possible implementation, the control module is further configured to: obtain an intermediate collision time based on the first collision time and the second collision time; and correct the intermediate collision time based on the multimodal fusion target confidence, road surface adhesion coefficient, and scene correction coefficient to obtain the first target collision time.
[0036] In one possible implementation, the acoustic sensing data further includes the acoustic motion direction of the first collision target, and the wireless sensing data further includes the wireless motion direction corresponding to the second collision target. The control module is further configured to: determine whether the confidence level of the multimodal fusion target in the multimodal fusion target sensing information is greater than a preset confidence threshold, and determine whether the acoustic motion direction is consistent with the wireless motion direction; if the confidence level of the multimodal fusion target is greater than the preset confidence threshold, and the acoustic motion direction is consistent with the wireless motion direction, then the collision target is determined to be valid; if the confidence level of the multimodal fusion target is less than or equal to the preset confidence threshold, or the acoustic motion direction is inconsistent with the wireless motion direction, then the collision target is determined to be invalid.
[0037] In one possible implementation, after generating acoustic sensing data and wireless sensing data, the automatic emergency braking control device further includes a spatiotemporal alignment module (not shown). This spatiotemporal alignment module is used to: perform time alignment processing on the acoustic sensing data and wireless sensing data respectively to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data; and perform spatial alignment processing on the time-aligned acoustic sensing data and time-aligned wireless sensing data respectively based on the vehicle's on-board coordinate system to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
[0038] In one possible implementation, the acoustic signal information includes time-domain waveform information, frequency-domain frequency information, and acoustic signal quality feature information. The generation module is specifically used to: extract the occlusion-penetration acoustic signal corresponding to the first collision target based on the frequency-domain frequency information; calculate the propagation delay difference corresponding to the first collision target based on the occlusion-penetration acoustic signal; obtain the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference; estimate the acoustic lateral motion velocity corresponding to the first collision target based on the time-domain waveform information of the occlusion-penetration acoustic signal; and determine the initial acoustic confidence level corresponding to the first collision target based on the acoustic signal quality feature information.
[0039] In one possible implementation, the channel state information includes signal amplitude fluctuations, signal phase changes, and multipath interference information. The generation module is specifically used to: construct the motion index corresponding to the second collision target based on the signal amplitude fluctuations and signal phase changes; obtain the wireless lateral distance and wireless lateral motion speed corresponding to the second collision target based on the motion index; and determine the initial wireless confidence level corresponding to the second collision target based on the signal phase changes and multipath interference information.
[0040] In one possible implementation, the control module is further configured to: determine the collision time of the second target based on acoustic sensing data and wireless sensing data in response to the first collision target and the second collision target not being the same collision target; and output an automatic emergency braking control command based on the collision time of the second target.
[0041] In one possible implementation, the control module is further configured to: calculate the third collision time of the first collision target reaching the second collision point based on the acoustic lateral distance and acoustic lateral velocity corresponding to the first collision target in the acoustic sensing data, wherein the second collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the first collision target; calculate the fourth collision time of the second collision target reaching the third collision point based on the wireless lateral distance and wireless lateral velocity corresponding to the second collision target in the wireless sensing data, wherein the third collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the second collision target; and take the smaller of the third collision time and the fourth collision time to obtain the second target collision time.
[0042] In one possible implementation, during the execution of the automatic emergency braking control command, the automatic emergency braking control device further includes a monitoring module (not shown), which is used to: acquire the vehicle's attitude information, including yaw rate and sideslip angle; determine whether the yaw rate is less than or equal to a yaw rate threshold, and determine whether the sideslip angle is less than or equal to a sideslip angle threshold; if the yaw rate is less than or equal to the yaw rate threshold and the sideslip angle is less than or equal to the sideslip angle threshold, then the automatic emergency braking control command is continuously executed; if the yaw rate is greater than the yaw rate threshold or the sideslip angle is greater than the sideslip angle threshold, then a stability protection mechanism is activated.
[0043] Thirdly, the present invention provides a vehicle, comprising: a vehicle body and an automatic emergency braking control system, the automatic emergency braking control system being used to execute the automatic emergency braking control method as provided in the first aspect above.
[0044] In one possible implementation, the automatic emergency braking control system includes an acoustic sensing unit and a wireless sensing unit; when both the acoustic sensing unit and the wireless sensing unit are functioning normally, the automatic emergency braking control system activates a dual-modal fusion mode; when the acoustic sensing unit malfunctions, the automatic emergency braking control system activates a wireless sensing mode; when the wireless sensing unit malfunctions, the automatic emergency braking control system activates an acoustic sensing mode; when both the acoustic sensing unit and the wireless sensing unit malfunction, the automatic emergency braking control system disables the non-line-of-sight sensing function.
[0045] Fourthly, embodiments of the present invention provide an electronic device, including: a processor and a memory communicatively connected to the processor; the processor executes computer execution instructions stored in the memory to implement the automatic emergency braking control method as provided in the first aspect above.
[0046] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the automatic emergency braking control method provided in the first aspect above.
[0047] In a sixth aspect, embodiments of the present invention provide a computer program product, comprising: a computer program that, when executed by a processor, implements the automatic emergency braking control method provided in the first aspect above.
[0048] The beneficial effects of this invention are:
[0049] The automatic emergency braking control method, system, device, medium, and program product provided by this invention acquires multimodal environmental information, including acoustic signal information and channel state information, within the non-line-of-sight blind zone corresponding to the vehicle. Based on the acoustic signal information, acoustic sensing data corresponding to a first collision target within the non-line-of-sight blind zone is generated, and wireless sensing data corresponding to a second collision target within the non-line-of-sight blind zone is generated based on the channel state information. Further, in response to the first and second collision targets being the same collision target, the acoustic sensing data and wireless sensing data are fused to generate multimodal fused target sensing information. Then, based on the multimodal fused target sensing information, acoustic sensing data, and wireless sensing data, an automatic emergency braking control command is output and executed. This invention acquires acoustic signal information and channel state information within the non-line-of-sight (NOS) blind zone corresponding to the vehicle, and separately forms acoustic perception data and wireless perception data corresponding to the collision target within the NOS blind zone. This enables complementary characterization of the collision target within the NOS blind zone from different perception dimensions. Furthermore, when the collision target perceived acoustically and the collision target perceived wirelessly are the same, the acoustic perception data and wireless perception data are fused to improve the reliability of target detection and perception results in NOS blind zone scenarios. Then, based on the fused multimodal target perception information, acoustic perception data, and wireless perception data, an automatic emergency braking control command is output, improving the triggering accuracy and control stability of automatic emergency braking, thereby enhancing the vehicle's active safety performance. Attached Figure Description
[0050] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of the present invention;
[0051] Figure 2 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 1 ;
[0052] Figure 3 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 2 ;
[0053] Figure 4 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 3 ;
[0054] Figure 5 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 4 ;
[0055] Figure 6 This is a schematic diagram of the structure of the automatic emergency braking control system provided in an embodiment of the present invention;
[0056] Figure 7 This is a schematic diagram of the automatic emergency braking control device provided in an embodiment of the present invention;
[0057] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.
[0058] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0059] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.
[0060] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0062] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in the process, method, product, or apparatus that includes elements is not excluded. For example, the use of terms such as "first," "second," etc., to indicate names does not imply any particular order.
[0063] Currently, intelligent vehicle low-speed active safety control technology is widely used in scenarios prone to non-line-of-sight blind spots, such as underground parking garages, internal roads in residential areas, side roads in industrial parks, narrow urban roads, building corners, and temporary parking areas. In these scenarios, vehicles typically travel at low speeds, but obstacles such as surrounding walls, concrete pillars, parked vehicles, green barriers, or building corners can create non-line-of-sight blind spots, obstructing the driver's view and weakening the detection capability of traditional line-of-sight sensors for collision targets within these blind spots. Furthermore, in road traffic scenarios, "ghost peek" accidents caused by non-line-of-sight blind spots are characterized by their suddenness, short visibility distance, and insufficient driver reaction time. Related statistics show that in low-speed driving scenarios such as underground parking garages, residential areas, industrial parks, and narrow urban roads, "ghost peek" accidents account for more than 60% of all low-speed collision accidents, and once they occur, traditional AEB systems are almost unable to prevent collisions.
[0064] In related technologies, traditional AEB systems rely on line-of-sight sensors, including visible light cameras, millimeter-wave radar, lidar, and ultrasonic radar. Their common technological limitation is their inability to penetrate hard obstacles such as metal and concrete; they can only detect and locate targets when the target enters the sensor's line-of-sight range. Specifically, their shortcomings are: cameras depend on ambient light and a straight line of sight, becoming completely ineffective under obstructed conditions; millimeter-wave radar has extremely poor penetration through concrete walls, metal structures, and dense obstacles, making it difficult to identify targets behind obstructions; lidar has high detection accuracy but is expensive and strictly limited by line-of-sight, unable to achieve wall-penetrating detection; ultrasonic radar has short detection range, narrow angular coverage, poor directionality, and lacks non-line-of-sight blind zone detection capabilities. Therefore, in typical scenarios where a target suddenly appears from behind an obstacle, when using a traditional AEB system for braking control, the effective reaction time left for the AEB system is only 0.3 to 0.5 seconds. Even if the AEB system triggers braking quickly, it is difficult to bring the vehicle to a safe stop within a very short distance. Collision avoidance is difficult, and there is a defect in the low accuracy of automatic emergency braking triggering. The active safety performance of the vehicle is poor under the corresponding operating conditions.
[0065] Based on the technical problems existing in related technologies, the embodiments of the present invention acquire acoustic signal information and channel state information in the non-line-of-sight blind zone corresponding to the vehicle, and respectively form acoustic perception data and wireless perception data corresponding to the collision target in the non-line-of-sight blind zone. This enables complementary characterization of the collision target in the non-line-of-sight blind zone from different perception dimensions. Furthermore, when the collision target perceived by acoustic perception and the collision target perceived by wireless perception are the same collision target, the acoustic perception data and wireless perception data are fused to improve the reliability of target detection and perception results in non-line-of-sight blind zone scenarios. Then, based on the fused multimodal target perception information, acoustic perception data and wireless perception data, an automatic emergency braking control command is output to improve the triggering accuracy and control stability of automatic emergency braking, thereby improving the active safety performance of the vehicle.
[0066] The application scenarios of the embodiments of the present invention are described below. The automatic emergency braking control method provided by the embodiments of the present invention is applicable to active safety control of non-line-of-sight blind spots in low-speed driving scenarios of intelligent vehicles, specifically including complex environments such as underground garages, narrow roads in residential areas, building-obstructed areas, and intersection blind spots.
[0067] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of the present invention. For example... Figure 1 As shown, acoustic signal information and channel status information of the non-line-of-sight blind spot formed by the obstruction 10 are collected by the acoustic sensing unit and wireless sensing unit integrated on the vehicle, respectively. The acoustic sensing unit generates acoustic sensing data corresponding to the collision target 11 in the non-line-of-sight blind spot based on the acoustic signal information and sends the acoustic sensing data to the vehicle control unit (VCU). The wireless sensing unit generates wireless sensing data corresponding to the collision target 11 in the non-line-of-sight blind spot based on the channel status information and sends the wireless sensing data to the VCU. The VCU performs fusion calculation based on the acoustic sensing data and the wireless sensing data to generate an automatic emergency braking control command in real time and sends the control command to the integrated brake control unit (IBCU) to execute the braking action.
[0068] The specific implementation of the automatic emergency braking control method provided in this invention will be described in detail below with reference to specific embodiments.
[0069] Figure 2 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 1 .like Figure 2 As shown, a specific implementation of this automatic emergency braking control method may include the following steps:
[0070] S201, acquire multimodal environmental information in the non-line-of-sight blind spot corresponding to the vehicle, the multimodal environmental information includes acoustic signal information and channel state information.
[0071] For example, the vehicle can be an intelligent car equipped with an on-board controller, acoustic acquisition components, wireless communication components, braking actuators, and an on-board network.
[0072] Among them, the non-line-of-sight blind spot is the non-line-of-sight blind spot of the vehicle caused by obstructions. It is used to characterize the area that is difficult for the driver and conventional line-of-sight sensors to observe directly due to obstructions caused by walls, pillars, parked vehicles, building corners, green isolation facilities or temporary obstacles. This area is usually located on the diagonal side in front of the vehicle, outside the corner or near the exit of the passage, and changes in real time with the vehicle's driving posture, steering wheel angle and vehicle position.
[0073] For example, multimodal environmental information includes, but is not limited to, acoustic signal information, channel state information, and visual perception information. Specifically, acoustic signal information is used to characterize acoustic propagation characteristics from non-line-of-sight (LOS) blind spots; channel state information is used to characterize changes in the propagation state of wireless links traversing or covering LOS blind spots; and visual perception information is used to characterize the type of environmental scene the vehicle is in within its line-of-sight range.
[0074] For example, in one possible implementation, acoustic signal information can be synchronously collected by an onboard multi-microphone array deployed on the vehicle's front bumper (e.g., one microphone each on the left, middle, and right sides of the front bumper), A-pillar (e.g., two microphones each on the left and right sides of the A-pillar for through-wall acoustic perception of vehicle and building obstructions in the vehicle's side blind spots), exterior rearview mirror housing, hood edge, or roof (e.g., one microphone in the center of the roof for through-wall acoustic perception of long-distance, overhead obstructions). After each microphone collects analog acoustic signals, the data is converted from analog to digital to obtain digital time-domain waveform data, which is then timestamped and aligned with a global clock to form a multi-channel synchronous raw acoustic data stream. The onboard multi-microphone array can be a distributed multi-microphone array. In this embodiment, the number of microphones on the vehicle is no less than three, and the specific number can be determined according to actual application requirements.
[0075] Optionally, to further reduce background noise introduced by the vehicle's engine, motor, tire rolling, air conditioning fan, and in-vehicle vibrations, the controller can also perform bandpass filtering, adaptive noise reduction, and echo suppression processing on the raw acoustic data stream collected by the multi-microphone array, thereby obtaining acoustic signal information that is more suitable for collision target analysis in non-line-of-sight blind spots.
[0076] For example, acoustic signal information includes, but is not limited to, time-domain waveform information, frequency-domain frequency information, and acoustic signal quality characteristic information. Specifically, time-domain waveform information reflects the arrival time and amplitude changes of the collision target sound source; frequency-domain frequency information distinguishes different sound source characteristics such as footsteps, tire friction sounds, stroller gliding sounds, or low-speed vehicle power sounds; and acoustic signal quality characteristic information characterizes the signal-to-noise ratio, echo interference intensity, transient stability, and the degree of sound source separability.
[0077] For example, channel state information can be collected by a wireless sensing unit mounted on the vehicle. This wireless sensing unit can support one or more of the following: vehicular wireless local area network, ultra-wideband communication, Bluetooth Low Energy link, cellular vehicle-to-everything (V2X) link, or dedicated short-range communication link. In one possible implementation, the wireless sensing unit transmits a detection signal and receives the echo on the non-line-of-sight blind spot side corresponding to the vehicle, or receives link measurement results fed back from roadside nodes, carrying terminals, or cooperative sensing nodes. It then extracts the signal amplitude response, signal phase response, and multipath energy distribution under multiple subcarriers, multiple antenna ports, or multiple time slots to form channel state information.
[0078] For example, channel state information includes, but is not limited to, signal amplitude fluctuations, signal phase changes, and multipath interference information. Among them, signal amplitude fluctuations are used to reflect the propagation fading changes caused by the presence of moving targets in non-line-of-sight blind zones; signal phase changes are used to reflect the small perturbations on the propagation path length caused by the displacement of colliding targets in non-line-of-sight blind zones; and multipath interference information is used to describe the path superposition caused by reflections from building corners, metal vehicle bodies, and walls.
[0079] For example, visual perception information can be obtained through visual sensors such as cameras mounted on the vehicle.
[0080] For example, to ensure the consistency of multimodal environmental information in the time dimension, in one possible implementation, acoustic sampling frames and wireless measurement frames can be aligned with a unified time reference, and the two types of data can be cached, packaged and identified according to a preset sensing period, such as 20 milliseconds to 100 milliseconds, to generate multimodal environmental information data packets for the same non-line-of-sight occlusion blind zone.
[0081] Understandably, vehicle-mounted acoustic sensing technology utilizes the physical property that sound waves can penetrate non-metallic obstacles such as concrete, brick walls, and slabs. It uses a vehicle-mounted distributed microphone array to collect sound source characteristic signals such as pedestrian footsteps, non-motorized vehicle sounds, and wheel rolling sounds in non-line-of-sight blind spots, thereby enabling the determination of the presence of collision targets in blind spots, distance calculation, and estimation of movement direction and speed. The acoustic solution offers high ranging accuracy, precise orientation recognition, and rapid response in low-noise environments such as underground parking garages. However, in complex road environments with heavy traffic, wind noise, and vehicle vibration noise, the effective acoustic signal is easily drowned out by background noise, leading to decreased detection accuracy and increased false alarm probability. It cannot independently meet the requirements of full-scenario automatic emergency braking control. The onboard WiFi Channel State Information (WiFi CSI) sensing technology relies on changes in channel parameters caused by the reflection and scattering of wireless signals by moving targets to achieve target detection, localization, and motion state determination in non-line-of-sight blind spots. This solution reuses the vehicle's existing onboard WiFi module, ensuring wireless communication while achieving environmental perception. It boasts low hardware cost, is unaffected by acoustic noise, and exhibits excellent stability in wall-penetrating detection, effectively compensating for the performance deficiencies of acoustic sensing in high-noise environments. However, in scenarios with dense metal components and numerous vehicles, such as underground parking garages with strong multipath interference, wireless signals are prone to amplitude drift, phase distortion, and multipath superposition interference. The ranging and speed measurement accuracy is inferior to acoustic sensing, making it difficult to meet the high-precision control requirements of AEB when used alone. This invention overcomes the drawbacks of single sensing modes being constrained by the environment and lacking stability by synchronously fusing acoustic signal information and WiFi CSI channel state information, providing time-synchronized and continuous raw data support for the subsequent generation of acoustic sensing data and wireless sensing data.
[0082] When a vehicle accelerates, decelerates, turns, or deviates from its straight-line trajectory, the vehicle's own displacement directly causes systematic offset errors in acoustic time delay ranging and the lateral distance and lateral velocity calculated by the WiFi motion index. Optionally, in one possible implementation, multimodal environmental information can be further combined with the vehicle's current acceleration, yaw rate, and wheel speed information for motion compensation to reduce the disturbance caused by the vehicle's own motion to the perception results of non-line-of-sight occlusion blind spots. Specifically, vehicle acceleration, deceleration, and turning will change the reference position of the microphone and WiFi antenna in a unified vehicle coordinate system, causing a fixed offset in acoustic time delay ranging and the lateral distance and velocity calculated by the WiFi motion index. The system uses wheel speed to calculate the inter-frame longitudinal displacement of the vehicle body and updates the heading angle correction coordinate transformation matrix from the yaw rate to perform spatial compensation on the original perception data, correcting the measurement deviation caused by vehicle motion, reducing positioning and speed measurement errors, and ensuring the accuracy of subsequent multimodal fusion and collision time calculation.
[0083] S202, based on acoustic signal information, generate acoustic sensing data corresponding to the first collision target in the non-line-of-sight occlusion blind zone, and based on channel state information, generate wireless sensing data corresponding to the second collision target in the non-line-of-sight occlusion blind zone.
[0084] The first collision target is used to represent a collision target within a non-line-of-sight occlusion blind zone identified based on the acoustic link; the second collision target is used to represent a collision target within a non-line-of-sight occlusion blind zone identified based on the wireless link.
[0085] For example, acoustic sensing data is used to carry information such as the position, movement trend, and credibility of the first collision target in the non-line-of-sight occlusion blind zone; wireless sensing data is used to carry information such as the position, movement trend, and credibility of the second collision target in the non-line-of-sight occlusion blind zone.
[0086] It is understandable that, due to the different physical mechanisms of the two types of sensing links, acoustic sensing utilizes the penetration, diffraction, and reflection characteristics of sound propagation, while wireless sensing utilizes the fading, phase perturbation, and multipath variation characteristics of wireless propagation. Therefore, they form independent target sensing results, which is beneficial for subsequent determination and fusion confirmation of the same collision target.
[0087] For example, one possible implementation for generating acoustic perception data corresponding to the first collision target in the non-line-of-sight occlusion blind zone based on acoustic signal information is as follows: For the acoustic signal information collected by each channel of the multi-channel microphone array, feature extraction is first performed on the acoustic signal information to filter out the effective acoustic components corresponding to the potential first collision target in the non-line-of-sight occlusion blind zone, such as the occlusion-penetrating acoustic signal; the propagation delay difference of the occlusion-penetrating acoustic signal between different microphone channels is calculated, and the propagation delay difference of each channel is weighted and averaged to obtain the final delay difference; combined with the known geometric parameters from the vehicle to the occlusion edge, the sound wave propagation speed, and the final delay difference, the lateral distance of the potential first collision target relative to the vehicle's driving path is calculated; the lateral distance sequence of continuous sampling periods is differentially and filtered to obtain the lateral movement speed of the first collision target; based on the positive or negative sign of the speed and the trend of distance change, it is determined whether the first collision target is moving closer to or away from the vehicle's driving path, and the movement direction of the first collision target relative to the vehicle is obtained; at the same time, the initial acoustic confidence is solved by comprehensively considering the signal-to-noise ratio, the stability of the delay estimation, the continuity of the spectrum, and the smoothness of the multi-frame trajectory.
[0088] It should be noted that if multiple collision targets are detected from the acoustic signal information, the collision time of each collision target to the collision point is calculated, and the collision target with the shortest collision time is determined as the first collision target; where the collision point is the spatial intersection of the predicted driving trajectory of the vehicle and the predicted motion trajectory of the corresponding collision target.
[0089] For example, acoustic perception data includes, but is not limited to, acoustic lateral distance, acoustic lateral motion velocity, acoustic motion direction, collision target category probability, and initial acoustic confidence.
[0090] For example, one possible implementation for generating wireless sensing data corresponding to a second colliding target within a non-line-of-sight (NLS) blind zone based on channel state information is as follows: The channel state information is denoised, static background is eliminated, and motion components are separated to extract dynamic link changes caused by the motion of the second colliding target. Specifically, a sliding window filter is applied to the amplitude sequence of each subcarrier to remove slowly varying environmental interference; phase unwrapping and linear correction are performed on the phase sequence to extract phase perturbation features that can be used for motion estimation; and a channel spatial propagation feature vector is constructed based on multipath energy distribution. Subsequently, a target motion index is constructed, which is obtained by weighted fusion of amplitude change rate, phase change rate, and multipath energy offset, and is used to characterize whether a moving target exists within the NLS blind zone and the intensity of the target's motion. By combining the installation location parameters of the vehicle-mounted wireless transceiver node, the motion index is mapped and converted into wireless lateral distance and wireless lateral motion speed. For example, a pre-trained regression model, geometric inversion model or Kalman observation model can be used to solve the displacement data of the second collision target relative to the occlusion boundary and the vehicle's driving path based on the channel changes in multiple time windows, and the motion direction of the second collision target can be determined by combining the speed change trend of continuous frames.
[0091] Alternatively, in one possible implementation, to improve robustness in complex multipath environments, a multi-antenna spatial diversity or multi-link consistency verification mechanism can be introduced to perform consistency screening on target results measured by different channels in the same time period, ultimately forming the wireless sensing data corresponding to the second collision target.
[0092] For example, wireless sensing data includes, but is not limited to, wireless lateral distance, wireless lateral movement speed, wireless movement direction, link disturbance strength, and initial wireless confidence.
[0093] Understandably, by generating acoustic and wireless sensing data separately in this step, two independent sources of sensing evidence can be established before the occluded collision target is directly visible. This reduces the problems of missed detections, false detections, or estimation bias caused by relying on a single sensing modality. Especially in scenarios where acoustic information is susceptible to noise interference and wireless information is susceptible to multipath fluctuations, completing independent modeling and target extraction first is beneficial for identifying targets with genuine collision risks through subsequent cross-validation.
[0094] In one possible implementation, when multiple collision targets exist within the same timeframe, the most likely collision risk target can be selected as the first and second collision targets based on target approach trends, path intersection probabilities, and confidence levels. Specifically, acoustic and wireless sensing often simultaneously identify multiple moving targets behind walls. The system combines the target's speed approaching the lane, the probability of intersection with the vehicle's path, and the reliability of the sensing data to comprehensively determine the risk level, selecting the target with the highest risk from the two sensing paths as the first and second collision targets, respectively. Irrelevant low-risk targets are filtered out, reducing computational burden and avoiding interference with subsequent risk assessments.
[0095] S203, in response to the fact that the first collision target and the second collision target are the same collision target, the acoustic sensing data and the wireless sensing data are fused to generate multimodal fused target sensing information.
[0096] It should be noted that, in this embodiment of the invention, the determination of the same collision target is used to confirm whether the first collision target identified by the acoustic link and the second collision target identified by the wireless link correspond to the same entity in the non-line-of-sight occlusion blind zone, such as the same pedestrian, the same bicycle, or the same low-speed vehicle.
[0097] Understandably, when the first collision target and the second collision target are determined to be the same collision target, fusing the acoustic sensing data and the wireless sensing data can avoid mistakenly combining the sensing results of different collision targets into a single high-risk target, thereby reducing the risk of false braking.
[0098] For example, multimodal fusion of target perception information is used to represent the integrated collision target state after fusion, which is the core input for subsequent collision time estimation and braking decision.
[0099] For example, one possible implementation for determining that the first collision target and the second collision target are the same collision target is as follows: Based on the vehicle coordinate system, the acoustic sensing data and the wireless sensing data are spatiotemporally aligned to obtain the spatiotemporally aligned acoustic lateral distance, wireless lateral distance, acoustic lateral motion velocity, wireless lateral motion velocity, acoustic motion direction, and wireless motion direction. Furthermore, the distance difference between the acoustic lateral distance and the wireless lateral distance is calculated, the velocity difference between the acoustic lateral motion velocity and the wireless lateral motion velocity is calculated, and the acoustic motion direction and the wireless motion direction are compared. If the distance difference is less than a first threshold, the velocity difference is less than a second threshold, and the motion directions are consistent or close, then the first collision target and the second collision target are determined to meet the spatial consistency and motion consistency conditions and belong to the same collision target. If the distance difference is greater than or equal to the first threshold, or the velocity difference is greater than or equal to the second threshold, or the motion directions are inconsistent, then the first collision target and the second collision target do not meet the spatial consistency and motion consistency conditions and do not belong to the same collision target.
[0100] For example, the fusion processing method can be one or more of the following: weighted fusion, Bayesian estimation, Kalman filtering, particle filtering, or evidence theory fusion.
[0101] Understandably, this step, by first determining whether the acoustically sensed first collision target and the wirelessly sensed second collision target are the same collision target, and then performing multimodal fusion, establishes a reliable mapping relationship from independent sensed evidence to a unified risk target. This effectively solves the problems of inconsistencies between sensed objects of different modalities, unclear temporal correspondence, and the potential for misjudgment due to direct fusion. The fused multimodal target sense information has higher stability and reliability than single-modal information, providing a more sufficient basis for braking decisions.
[0102] In this embodiment of the invention, by fusing acoustic sensing data and wireless sensing data in response to the first and second collision targets being the same collision target, multimodal fused target sensing information is generated. This can eliminate the independent false detection or missed detection problems caused by environmental noise and multipath interference in acoustic and wireless sensing. For example, in an underground parking garage scenario, acoustic sensing is easily affected by air conditioning noise, and wireless sensing is easily affected by multipath reflection interference from metal columns. Through fusion processing, their advantages can be complemented, improving the target detection accuracy by more than 15%.
[0103] Alternatively, in one possible implementation, in response to the fact that the first collision target and the second collision target are not the same collision target, the fusion result may not be generated temporarily, and independent tracking may be maintained and matching may continue in subsequent perception cycles.
[0104] S204, based on multimodal fusion of target perception information, acoustic perception data and wireless perception data, outputs an automatic emergency braking control command and executes the automatic emergency braking control command.
[0105] Automatic emergency braking is used to send a braking request to the braking system when a high collision risk is detected in a non-visual blind spot. The vehicle controller then uses a brake-by-wire unit, hydraulic brake actuator, or electric brake to establish deceleration, thereby reducing or avoiding a collision.
[0106] For example, automatic emergency braking control commands include, but are not limited to, braking trigger time, target deceleration, braking force increase gradient, and whether to simultaneously output audible and visual alarm information.
[0107] For example, one possible implementation is to calculate the collision time between the target and the vehicle in a potential conflict scenario based on the multimodal fusion target perception information, including the multimodal fusion target lateral distance, multimodal fusion target lateral velocity, and the vehicle's current speed. Specifically, the time required for the target to laterally intrude into the vehicle's driving path is defined as Tt, and the time required for the vehicle to reach the target conflict intersection point is defined as Tv. The closer Tt and Tv are, the higher the consistency of the trajectory intersection between the vehicle and the target, and the higher the probability and risk level of the collision conflict.
[0108] This invention combines the motion approach trend correction of the fused target to obtain the final target collision time, and comprehensively determines the risk level by combining the multimodal fused target confidence. When the target collision time is less than a preset collision time threshold and the multimodal fused target confidence is higher than a preset confidence threshold, the system enters a braking pre-trigger state. Simultaneously, the acoustic side collision time and the wireless side collision time are independently calculated based on acoustic sensing data and wireless sensing data, respectively, to achieve dual-modal risk cross-verification. When both collision times fall within a preset danger time interval, or when a single collision time falls within a danger time interval and the initial confidence of the other corresponding time meets a preset supplementary confirmation threshold, an automatic emergency braking control command is generated and output.
[0109] For example, one possible implementation of automatic emergency braking control commands could be to adjust braking intensity in stages according to the risk level. Specifically, when a collision target just enters a high-risk zone, a smaller pre-charge pressure or light braking can be applied initially to shorten the subsequent full braking time; as the collision time further shortens or the target approach trend continues to intensify, the system switches to medium braking or braking at the maximum permissible deceleration. To avoid vehicle instability caused by changes in the road surface adhesion coefficient, the vehicle controller can work in conjunction with the anti-lock braking system, electronic stability control system, and traction control system to adjust the braking force distribution in real time according to the current wheel speed, longitudinal acceleration, and steering wheel angle.
[0110] Optionally, during the execution of the automatic emergency braking control command, the system can continue to receive new acoustic sensing data, wireless sensing data, and multimodal fusion target sensing information. Once the risk of collision is eliminated, such as when the collision target stops approaching, turns away from the conflict path, or the fusion confidence level decreases significantly, the controller can reduce the braking level or terminate the braking action to balance safety and traffic efficiency.
[0111] Optionally, in one possible implementation, the automatic emergency braking control method provided in this embodiment of the invention may further include: sending a warning message to the vehicle human-machine interaction system before outputting the automatic emergency braking control command, so as to remind the driver that there is a collision target in the current non-visual blind spot and active safety intervention is being performed.
[0112] In this embodiment of the invention, by acquiring acoustic signal information and channel state information within the non-line-of-sight (NOS) blind zone corresponding to the vehicle, and forming acoustic perception data and wireless perception data corresponding to the collision target within the NOS blind zone, complementary characterization of the collision target within the NOS blind zone can be achieved from different perception dimensions. Furthermore, when the collision target perceived by acoustic perception and the collision target perceived by wireless perception are the same collision target, the acoustic perception data and wireless perception data are fused to improve the reliability of target detection and perception results in NOS blind zone scenarios. Then, based on the fused multimodal target perception information, acoustic perception data, and wireless perception data, an automatic emergency braking control command is output to improve the triggering accuracy and control stability of automatic emergency braking, thereby enhancing the vehicle's active safety performance.
[0113] The following is combined Figure 3 A detailed explanation is provided of a specific implementation method for fusing acoustic sensing data and wireless sensing data in step S203 to generate multimodal fused target sensing information.
[0114] Figure 3 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 2 .like Figure 3 As shown, a specific implementation of the automatic emergency braking control method for fusing acoustic sensing data and wireless sensing data to generate multimodal fused target sensing information may include the following steps:
[0115] S301, based on visual perception information and acoustic signal information, generates the confidence level of the target acoustic scene and the confidence level of the target wireless scene.
[0116] In this step, visual perception information is used to characterize the road morphology, obstacle boundaries, and target visibility in the occluded environment around the vehicle; acoustic signal information is used to characterize the echo changes, reflection intensity changes, or noise disturbances caused by the presence of a collision target; target acoustic scene confidence is used to characterize the credibility of the acoustic perception results in the current driving scenario; and target wireless scene confidence is used to characterize the credibility of the wireless perception results in the current driving scenario.
[0117] For example, one possible implementation is as follows: After extracting the target region from the image frame output by the vehicle's onboard vision sensor, the image is combined with acoustic signal information to identify whether there are obstructions, corners, static obstacles, or dynamically moving targets in the non-line-of-sight blind spot. The identification result is then input into a scene confidence calculation module. This module can evaluate the consistency between visual and acoustic features based on a neural network classifier, feature matching model, or rule-based decision model, thereby generating the corresponding target acoustic scene confidence and target wireless scene confidence.
[0118] Optionally, to adapt to the complex reflection environment of underground garages, temporary parking areas, and building corner areas, confidence calculation can also incorporate parameters such as target distance, target movement direction, sound source orientation, and visual occlusion ratio to improve the stability of scene judgment.
[0119] Understandably, by generating target acoustic scene confidence and target wireless scene confidence based on visual perception information and acoustic signal information, the fusion weights of acoustic and wireless sensing can be dynamically adjusted. For example, in a noisy road scene, when the acoustic scene confidence is low, the system automatically reduces the acoustic perception weight and increases the wireless perception weight, thereby avoiding fusion misjudgments caused by noise interference.
[0120] Optionally, one possible implementation of this step is as follows: determine the target driving scene type of the vehicle based on visual perception information and acoustic signal information; determine the target acoustic scene confidence based on the first mapping relationship between the driving scene type and the acoustic scene confidence; and calculate the target wireless scene confidence based on the target acoustic scene confidence.
[0121] For example, the target driving scene type can be a quiet garage, a noisy road, a multi-metal garage, or a mixed complex scene, etc., to identify the working environment where the vehicle is located, such as the entrance and exit of an underground garage, a narrow road in the park, a building corner, a temporary parking section, etc., which are prone to non-line-of-sight occlusion.
[0122] For example, the first mapping relationship can be obtained based on historical data. Specifically, when the driving scenario is a quiet garage, the accuracy of the acoustic signal collected by the microphone is high, and the corresponding acoustic scenario confidence level can be 0.65, while the wireless scenario confidence level can be 0.35; when the driving scenario is a noisy road, the microphone is easily interfered with, and the corresponding acoustic scenario confidence level can be 0.35, while the wireless scenario confidence level can be 0.65; when the driving scenario is a multi-metal garage, the corresponding acoustic scenario confidence level can be 0.6, while the wireless scenario confidence level can be 0.4; when the driving scenario is a mixed and complex scenario, the corresponding acoustic scenario confidence level can be 0.5, while the wireless scenario confidence level can be 0.5, and so on. This embodiment of the invention does not limit the specific correspondence between the driving scenario type and the acoustic scenario confidence level in this first mapping relationship; it can be determined according to the actual application requirements.
[0123] For example, the sum of the confidence in the target acoustic scene and the confidence in the target wireless scene can be 1.
[0124] For example, in this embodiment, one possible implementation is as follows: by performing semantic segmentation, target detection, and occlusion region identification on image frames in visual perception information, noise reduction, dereverberation, and spectrum analysis on acoustic signal information, road structure features, obstacle features, and acoustic texture features are extracted, and the extracted features are input into a scene classification model to output the target driving scene type; then, according to the preset mapping table corresponding to the first mapping relationship, the acoustic scene confidence interval corresponding to the target driving scene type is found, and the target acoustic scene confidence is determined by combining the current acoustic features with historical statistical distribution; further, the confidence of wireless perception-related parameters is calculated based on the target acoustic scene confidence, such as by calculating the target wireless scene confidence through linear weighting, piecewise functions, or normalization functions, so that it is consistent with the acoustic scene confidence under the same scene semantics.
[0125] For example, another possible implementation is to use a neural network classifier or rule-based decision model to evaluate the consistency between visual features (such as road structure, obstacle boundaries, and occlusion ratio) and acoustic features (such as sound source location and signal stability) to generate a target driving scene type; based on the first mapping relationship between the driving scene type and the acoustic scene confidence, according to the target driving scene type (such as quiet garage, noisy road, and mixed complex scene), find the acoustic scene confidence interval in the preset mapping table, and combine it with the current acoustic signal quality features (such as signal-to-noise ratio and feature stability) to determine the target acoustic scene confidence.
[0126] Understandably, by determining the target driving scene type of the vehicle based on visual perception information and acoustic signal information, the stability of scene determination in complex occlusion environments can be improved. Furthermore, based on the target driving scene type, the confidence of the target acoustic scene and the confidence of the target wireless scene can be determined, reducing the impact of single-modal fluctuations on the perception results and providing a reliable scene weight basis for subsequent multimodal fusion and braking control.
[0127] S302, based on the target acoustic scene confidence and the target wireless scene confidence, performs fusion processing on acoustic sensing data and wireless sensing data to generate multimodal fusion target sensing information.
[0128] Optionally, the acoustic sensing data includes the initial acoustic confidence level corresponding to the first collision target, and the wireless sensing data includes the initial wireless confidence level corresponding to the second collision target.
[0129] For example, the initial acoustic confidence score is used to characterize the initial credibility of the first colliding target in the acoustic link, and is usually determined by acoustic signal quality characteristics (such as acoustic signal-to-noise ratio and stationarity). Specifically, features such as sound intensity variation, arrival delay difference, and spectral energy distribution can be extracted from the acoustic signal information, and the initial acoustic confidence score can be calculated based on the extracted features.
[0130] For example, the initial radio confidence level is used to characterize the original credibility of the second colliding target in the radio link, and is typically determined by phase changes and multipath interference information in the channel state information. Specifically, features such as phase perturbation, multipath peaks, and fading depth can be extracted from the channel state information, and the initial radio confidence level can be calculated based on the extracted features.
[0131] Optionally, this step may include the following steps:
[0132] S3021, the initial acoustic confidence and the initial wireless confidence are corrected respectively to obtain the corrected acoustic confidence and the corrected wireless confidence.
[0133] Among them, the corrected acoustic confidence and the corrected wireless confidence are used to correct the original confidence after it has been affected by noise, reflection and fluctuation, so as to improve the consistency of subsequent fusion results.
[0134] For example, the corrected acoustic confidence level can be expressed by the following formula:
[0135]
[0136] in, This indicates the corrected acoustic confidence level. This represents the initial acoustic confidence level. This represents the validity coefficient of the spatiotemporal alignment between acoustic sensing data and wireless sensing data (the value of this validity coefficient can be 1 when the spatiotemporal alignment is valid, and the value of this validity coefficient is less than 1 when the spatiotemporal alignment is invalid). This represents the environmental noise figure (a preset calibration coefficient; for example, under quiet and low-noise operating conditions, the environmental noise figure can be 1, and this environmental noise figure decreases as the environmental noise increases).
[0137] For example, in a quiet environment, the microphone has almost no noise, and the corresponding corrected acoustic confidence level can be 0.9. In a noisy environment, the microphone has a lot of noise, and the corresponding corrected acoustic confidence level can be 0.3, etc.
[0138] For example, the corrected wireless confidence level can be expressed by the following formula:
[0139]
[0140] in, This indicates the corrected wireless confidence level. Indicates the initial wireless confidence level. This represents the multipath interference coefficient (a preset calibration coefficient). In open, obstructed environments with few metal components, sparse obstacles, and weak multipath reflections, the degree of wireless signal distortion is low. The value can be 1; as the number of metal pillars, densely parked vehicles, and enclosed garage structures in the environment increases, multipath reflections become stronger, and channel amplitude and phase distortion intensify. The value of then gradually decreases.
[0141] In this embodiment of the invention, by correcting the initial acoustic confidence and the initial wireless confidence separately, confidence fluctuations caused by environmental noise or multipath interference can be eliminated. For example, in an underground parking garage scenario, the corrected confidence can be stabilized above 0.7, while the uncorrected confidence may drop below 0.4 due to signal distortion. The stability of the fusion result is significantly improved after correction.
[0142] S3022, based on the target acoustic scene confidence and the target wireless scene confidence, perform fusion processing on the corrected acoustic confidence and the corrected wireless confidence to generate fused acoustic confidence and fused wireless confidence.
[0143] It is understood that the embodiments of the present invention adopt a scene-adaptive fusion weight allocation mechanism. After obtaining the corrected acoustic confidence and corrected wireless confidence corresponding to each signal, the basic weights are automatically allocated based on the target acoustic scene confidence and the target wireless scene confidence, and then corrected according to the confidence, so as to improve the accuracy of the fused acoustic confidence and fused wireless confidence.
[0144] Among them, the fused acoustic confidence and fused wireless confidence are used to reflect the reliability of acoustic perception and wireless perception at the current moment.
[0145] For example, in one possible implementation, the corrected acoustic confidence and the corrected wireless confidence are multiplied by the target acoustic scene confidence and the target wireless scene confidence, respectively, by a linear weighting or normalization function to calculate the fused acoustic confidence and the fused wireless confidence.
[0146] For example, the fused acoustic confidence score can be expressed by the following formula:
[0147]
[0148] in, Indicates the confidence level of the fused acoustics. This represents the confidence level of the target acoustic scene. This indicates the confidence level of the target wireless scenario.
[0149] For example, the fused wireless confidence level can be expressed by the following formula:
[0150]
[0151] in, This indicates the fused wireless confidence level.
[0152] Understandably, a higher value for the fused acoustic confidence score indicates a higher accuracy of the acoustic link sensing results, and a higher value for the fused wireless confidence score indicates a higher accuracy of the wireless link sensing results.
[0153] S3023, based on fused acoustic confidence and fused wireless confidence, performs fusion processing on acoustic sensing data and wireless sensing data to generate multimodal fused target sensing information.
[0154] It is understood that in the embodiments of the present invention, acoustic perception and wireless perception are not directly spliced together. Instead, credibility calibration is completed first, and then scene adaptive fusion is completed. This reduces the impact of environmental noise, multipath interference, and occlusion changes on target recognition, enabling the generated multimodal fused target perception information to more accurately represent the true state of the same collision target. This provides reliable input for subsequent collision risk judgment and automatic emergency braking triggering, thereby improving the stability and consistency of collision target perception in non-line-of-sight occlusion blind spots, reducing misjudgments and omissions, and improving the accuracy and timeliness of automatic emergency braking control.
[0155] Optionally, the acoustic sensing data may also include the acoustic lateral distance between the first collision target and the vehicle and the acoustic lateral velocity corresponding to the first collision target, and the wireless sensing data may also include the wireless lateral distance between the second collision target and the vehicle and the wireless lateral velocity corresponding to the second collision target.
[0156] Among them, the acoustic lateral distance is used to characterize the lateral distance between the first collision target and the longitudinal direction of the vehicle, and the acoustic lateral motion velocity is used to characterize the motion trend of the first collision target in the lateral direction; the wireless lateral distance and the wireless lateral motion velocity are used to characterize the lateral position and lateral displacement trend of the second collision target under the channel sensing condition, respectively.
[0157] For example, acoustic perception data can be obtained by collecting acoustic signal information in the non-line-of-sight blind spot by a vehicle-mounted multi-channel distributed microphone array, followed by beamforming, direction of arrival estimation and time delay ranging. Then, combined with the vehicle coordinate system and the vehicle's attitude parameters, the radial displacement of the first collision target is converted into acoustic lateral distance, and the acoustic lateral motion speed is obtained by the rate of change of lateral distance between adjacent time moments.
[0158] For example, the wireless sensing data can be extracted by the vehicle-mounted wireless transceiver module based on channel state information, phase perturbation or multipath features to obtain the spatial change of the second collision target, and then mapped to obtain the wireless lateral distance and wireless lateral movement speed.
[0159] Optionally, one possible implementation of this step is as follows: based on the fused acoustic confidence and fused wireless confidence, the acoustic lateral distance and the wireless lateral distance are fused to generate a multimodal fused target lateral distance; based on the fused acoustic confidence and fused wireless confidence, the acoustic lateral motion velocity and the wireless lateral motion velocity are fused to generate a multimodal fused target lateral motion velocity; based on the fused acoustic confidence and fused wireless confidence, the corrected acoustic confidence and the corrected wireless confidence are fused to generate a multimodal fused target confidence.
[0160] For example, the lateral distance of a multimodal fusion target can be expressed by the following formula:
[0161]
[0162] in, Indicates the lateral distance of the target in multimodal fusion. Indicates acoustic lateral distance. Indicates the horizontal distance of the wireless circuit.
[0163] For example, the lateral motion velocity of a target through multimodal fusion can be expressed by the following formula:
[0164]
[0165] in, This represents the lateral motion velocity of the target in the multimodal fusion. Indicates the acoustic lateral velocity. This indicates the speed of wireless lateral movement.
[0166] For example, the confidence score of a multimodal fusion target can be expressed by the following formula:
[0167]
[0168] in, This represents the confidence level of the multimodal fusion target.
[0169] It is understood that, in this embodiment of the invention, by fusing acoustic lateral distance and wireless lateral distance based on fused acoustic confidence and fused wireless confidence, a multimodal fused target lateral distance is generated; by fusing acoustic lateral motion velocity and wireless lateral motion velocity based on fused acoustic confidence and fused wireless confidence, a multimodal fused target lateral motion velocity is generated; and by fusing corrected acoustic confidence and corrected wireless confidence based on fused acoustic confidence and fused wireless confidence, a multimodal fused target confidence is generated. This can reduce the error amplification problem caused by single acoustic sensing or single wireless sensing, improve the defects of inaccurate target lateral positioning, unstable motion trend estimation, and large confidence fluctuations in occluded environments, thereby improving the reliability of collision target recognition in non-line-of-sight occluded blind zone scenarios, reducing the probability of false braking and missed braking, and improving the accuracy, timeliness, and stability of automatic emergency braking control.
[0170] In this embodiment of the invention, target acoustic scene confidence and target wireless scene confidence are generated based on visual perception information and acoustic signal information. Based on the target acoustic scene confidence and target wireless scene confidence, the acoustic perception data and wireless perception data are fused to generate multimodal fused target perception information. This achieves adaptive fusion of acoustic perception data and wireless perception data using scene confidence, enabling the system to form target perception results more accurately in non-line-of-sight occlusion blind spots. This improves the stability of target recognition in non-line-of-sight occlusion blind spots, reduces the risk of false braking caused by single-modal misjudgment, and enhances the timeliness and reliability of automatic emergency braking control command output.
[0171] The following is combined Figure 4 A detailed explanation is provided on a specific implementation method of step S204, which outputs an automatic emergency braking control command based on multimodal fusion of target perception information, acoustic perception data, and wireless perception data.
[0172] Figure 4A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 3 .like Figure 4 As shown, a specific implementation of the automatic emergency braking control method, which outputs an automatic emergency braking control command based on multimodal fusion of target perception information, acoustic perception data, and wireless perception data, may include the following steps:
[0173] S401 determines whether the collision target is valid based on multimodal fusion of target perception information, acoustic perception data, and wireless perception data.
[0174] Among them, the collision target is used to integrate the first collision target corresponding to the acoustic sensing data and the second collision target corresponding to the wireless sensing data, and serves as a unified judgment object for whether to trigger braking control during the validity judgment stage.
[0175] For example, one possible implementation of this step is as follows: The acoustic sensing data and wireless sensing data are time-series aligned and target association verified. If the acoustic sensing data and wireless sensing data meet any preset consistency condition in terms of the collision target's location, movement trend, occurrence time, or confidence level, and match the collision target state described by the multimodal fusion target sensing information, the collision target is determined to be valid. If the acoustic sensing data and wireless sensing data do not meet the preset consistency condition in terms of the collision target's location, movement trend, occurrence time, or confidence level, or do not match the collision target state described by the multimodal fusion target sensing information and show a significant deviation, the collision target is determined to be invalid. The preset consistency condition can be implemented through a threshold comparison module or using a classification model trained on samples. This embodiment does not specifically limit this, nor does the present invention limit it.
[0176] In this embodiment, based on the multimodal fusion target perception information, acoustic perception data and wireless perception data, it is determined whether the collision target is valid, and if the collision target is valid, step S402 is executed; if the collision target is invalid, step S403 is executed.
[0177] S402 outputs an automatic emergency braking control command based on multimodal fusion target perception information.
[0178] Optionally, this step may include the following steps:
[0179] S4021, Calculate the first collision time when the colliding target reaches the first collision point based on the lateral distance and lateral velocity of the multimodal fusion target in the multimodal fusion target perception information.
[0180] For example, the first collision point is the spatial intersection of the predicted trajectory of the vehicle and the predicted trajectory of the collision target. The predicted trajectory of the vehicle can be obtained by short-term trajectory extrapolation based on vehicle operating parameters such as real-time vehicle speed, yaw rate, and front wheel angle, combined with the vehicle dynamics model. The predicted trajectory of the collision target can be obtained by temporal extrapolation based on the target's lateral distance, lateral velocity, and direction of motion output in multiple consecutive frames, through a uniform or uniformly variable speed motion model. The spatial intersection of the two extrapolated trajectories is the first collision point.
[0181] For example, the lateral distance of the multimodal fusion target is used to represent the lateral distance between the collision target and the vehicle's driving direction, and the lateral motion velocity of the multimodal fusion target is used to represent the motion component of the collision target along the lateral direction.
[0182] For example, the first collision time can be expressed by the following formula:
[0183]
[0184] in, Indicates the time of the first collision.
[0185] S4022, determine the first target collision time based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information.
[0186] The second collision time is the time from when the vehicle arrives at the first collision point.
[0187] For example, the second collision time can be expressed by the following formula:
[0188]
[0189] in, Indicates the time of the second collision. Indicates the speed of the vehicle. This indicates the longitudinal distance between the vehicle and the first point of impact.
[0190] For example, one possible implementation of this step is to input the first collision time, the second collision time, and the multimodal fusion target confidence into the time fusion model, and assign higher weights to targets with higher confidence and stronger time consistency between the two types, thereby determining the first target collision time.
[0191] It is understood that in this embodiment of the invention, by introducing the first collision point as a geometric reference, the lateral motion characteristics of the collision target and the predicted trajectory of the vehicle are unified into the same spatial coordinate system for calculation, so that the estimation of the collision timing no longer depends on a single instantaneous distance judgment. Furthermore, by combining the second collision time and the multimodal fusion target confidence for time correction, false braking can be suppressed when the confidence of the collision target recognition is insufficient or the trajectory of the vehicle changes, and braking control can be triggered in advance when the actual collision risk increases.
[0192] S4023 outputs an automatic emergency braking control command based on the first target collision time.
[0193] For example, the automatic emergency braking control command includes a target deceleration of the vehicle.
[0194] For example, in one possible implementation, the target deceleration of the vehicle is calculated based on the vehicle's real-time speed and the time of the first target collision, and the target deceleration is output.
[0195] For example, the target deceleration can be expressed by the following formula:
[0196]
[0197] in, This indicates the target deceleration of the vehicle. Indicates the time of the first target collision.
[0198] It is understood that, in this embodiment of the invention, by calculating the first collision time of the colliding target reaching the first collision point based on the lateral distance and lateral velocity of the multimodal fusion target in the multimodal fusion target perception information, and determining the first target collision time based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information, and further outputting an automatic emergency braking control command based on the first target collision time, it is possible to more accurately estimate the time of the colliding target reaching the intersection position based on the multimodal fusion target perception information, and form a more robust first target collision time by combining the vehicle arrival time and the multimodal fusion target confidence, thereby improving the triggering accuracy and timeliness of the automatic emergency braking control command and reducing the probability of missed braking and false braking in non-line-of-sight occlusion scenarios.
[0199] S403 does not output automatic emergency braking control commands.
[0200] For example, the vehicle controller maintains the current driving control state and does not output automatic emergency braking control commands to the braking actuator, thereby avoiding false braking to non-real collision targets.
[0201] It is understood that, in the embodiments of the present invention, the validity of the collision target is verified before the automatic emergency braking control command is output, and braking is triggered only when the collision target is determined to be valid, which can improve the reliability of braking decision in non-line-of-sight occlusion scenarios.
[0202] In this embodiment of the invention, by fusing multimodal target perception information, acoustic perception data, and wireless perception data, the validity of a collision target is determined. When the collision target is valid, an automatic emergency braking control command is output based on the multimodal target perception information. When the collision target is invalid, no automatic emergency braking control command is output. This reduces the interference caused by single-mode noise, echo, and multipath reflection, and reduces the probability of false braking caused by incorrect collision target association. Furthermore, by controlling the output of the automatic emergency braking control command only when the collision target is confirmed to be valid, the invention can balance the collision risk response speed and control stability, and improve the active safety performance in underground parking garages, residential roads, and corner blind spot scenarios.
[0203] Optionally, in this embodiment of the invention, a specific implementation of outputting an automatic emergency braking control command based on the first target collision time can be: determining the target risk level based on the second mapping relationship between the collision time and the risk level, and outputting an automatic emergency braking control command based on the target risk level.
[0204] The second mapping relationship between collision time and risk level is used to establish the correspondence between the first target collision time and the target risk level; the target risk level is used to characterize the severity of the current collision risk and serves as the basis for the graded output of automatic emergency braking commands.
[0205] For example, the second mapping relationship can be pre-stored in the calibration table of the vehicle controller, or it can be obtained by the controller offline based on historical collision samples, vehicle speed status and relative position parameters of the collision target. In practical applications, the second mapping relationship can also be implemented in other ways, and the embodiments of the present invention do not limit this.
[0206] Table 1 shows the second mapping relationship between collision time and risk level provided in the embodiments of the present invention.
[0207]
[0208] As shown in Table 1, when the collision time is greater than 4.5 seconds, the risk level is safe, no automatic emergency braking control command is output, and the vehicle maintains normal driving; when the collision time is greater than 3.5 seconds and less than or equal to 4.5 seconds, the risk level is warning, and the corresponding automatic emergency braking control command can be an audible and visual warning command to alert the occupants; when the collision time is greater than 2.5 seconds and less than or equal to 3.5 seconds, the risk level is low risk, and the corresponding automatic emergency braking control command can be the control command corresponding to the pre-pressure build-up of the brake fluid circuit; when the collision time is greater than 1.5 seconds and less than or equal to 2.5 seconds, the risk level is medium risk, and the corresponding automatic emergency braking control command can be a light braking control command or a medium braking control command; when the collision time is less than or equal to 1.5 seconds, the risk level is high risk, and the corresponding automatic emergency braking control command can be a full braking control command. It should be noted that Table 1 is only one example.
[0209] For example, in one possible implementation, when the target risk level is characterized as high risk, the automatic emergency braking control command may include a larger target braking deceleration, a higher braking pressure increase rate, and an immediate braking request; when the target risk level is characterized as low risk, the automatic emergency braking control command may include a smaller braking deceleration or a warning braking request to avoid unnecessary emergency braking. Upon receiving the automatic emergency braking control command, the braking execution unit can combine the current vehicle speed, road surface adhesion coefficient, and braking system status to complete the distribution of braking force and adjustment of braking pressure, thereby achieving automatic emergency braking control that matches the target risk level.
[0210] In this embodiment of the invention, a target risk level is determined based on a second mapping relationship between collision time and risk level, according to a first target collision time. This transforms collision risk assessment from a single time value into a tiered control basis. Furthermore, based on the target risk level, an automatic emergency braking control command is output, making braking control more consistent with the risk change characteristics in non-line-of-sight obstruction scenarios. Since the target risk level reflects the severity of the collision, the system can promptly enhance braking intensity in short-collision-time scenarios and avoid premature intervention in longer-collision-time scenarios. This improves the accuracy, stability, and traffic efficiency of automatic emergency braking and reduces the probability of false braking.
[0211] Optionally, in this embodiment of the invention, one possible way to determine the first target collision time based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information is as follows: obtain an intermediate collision time based on the first collision time and the second collision time; correct the intermediate collision time based on the multimodal fusion target confidence, the road surface adhesion coefficient, and the scene correction coefficient to obtain the first target collision time.
[0212] The intermediate collision time is used to characterize the intermediate result after combining the first and second collision times, serving as the basic quantity before the target collision time is corrected.
[0213] Among them, the road surface adhesion coefficient is used to reflect the impact of the adhesion conditions between the vehicle's tires and the road surface on the collision time; the scene correction coefficient is used to characterize the correction effect of scene factors such as underground garages, internal roads of residential areas, or corner obstruction areas on the collision risk moment.
[0214] For example, in one possible implementation, a minimization function can be used to reduce the first collision time and the second collision time to obtain the intermediate collision time.
[0215] For example, the intermediate collision time can be expressed by the following formula:
[0216]
[0217] in, This represents the intermediate collision time. min() is a function that takes the smaller value. That is, if the first collision time is less than the second collision time, the first collision time is output; if the first collision time is greater than the second collision time, the second collision time is output.
[0218] For example, the collision time of the first target can be expressed by the following formula:
[0219]
[0220] in, Indicates the time of the first target collision. This represents the road surface adhesion coefficient (which can be obtained by looking up a table using the driving force and slip ratio). This represents the scene correction factor (this factor can be obtained through experimental calibration; for example, in an underground parking garage, this factor can be 0.9, in a residential road, this factor can be 1.0, and in an open road, this factor can be 1.1, etc.).
[0221] Understandably, when the road surface adhesion coefficient is low, the vehicle braking distance increases, and the first target collision time can be advanced accordingly. When the scenario correction coefficient is characterized by strong obstruction, narrow passage, or low-speed oncoming traffic, the first target collision time can be further advanced to improve braking safety.
[0222] In this embodiment of the invention, an intermediate collision time is obtained based on the first collision time and the second collision time. The intermediate collision time is then corrected based on the multimodal fusion target confidence, road surface adhesion coefficient, and scene correction coefficient to obtain the first target collision time. This avoids the problem of premature braking or braking lag caused by relying solely on the original time estimate. The collision time estimation result takes into account perception confidence, road surface conditions, and scene environmental factors, thereby improving the accuracy, stability, and applicability of low-speed active safety control and reducing the risk of false triggering.
[0223] Optionally, the acoustic sensing data may also include the acoustic motion direction of the first collision target, and the wireless sensing data may also include the wireless motion direction corresponding to the second collision target.
[0224] Among them, the acoustic motion direction is used to characterize the motion trend of the first collision target in the non-line-of-sight blind zone, and is usually obtained by combining the time delay change, spectrum shift, angle of arrival change or multi-frame sound source localization results of the acoustic signal; the wireless motion direction is used to characterize the motion trend of the second collision target in the non-line-of-sight blind zone, and is usually obtained by combining the phase change, received power change, timing correlation or multi-frame wireless localization results in the channel state information.
[0225] Optionally, one possible implementation for determining the validity of a collision target based on multimodal fusion target perception information, acoustic perception data, and wireless perception data is as follows: determine whether the confidence level of the multimodal fusion target in the multimodal fusion target perception information is greater than a preset confidence threshold, and determine whether the acoustic motion direction is consistent with the wireless motion direction; if the confidence level of the multimodal fusion target is greater than the preset confidence threshold, and the acoustic motion direction is consistent with the wireless motion direction, then the collision target is determined to be valid; if the confidence level of the multimodal fusion target is less than or equal to the preset confidence threshold, or the acoustic motion direction is inconsistent with the wireless motion direction, then the collision target is determined to be invalid.
[0226] For example, a preset confidence threshold is used to limit the minimum level of confidence that the multimodal fusion target perception information can reach for control decisions. The value of the preset confidence threshold can be preset based on scene complexity, sensor calibration accuracy, and false detection tolerance. For example, the value of the preset confidence threshold can be 0.7. This embodiment of the invention does not limit the value of the preset confidence threshold; it can be determined according to actual application requirements.
[0227] For example, one possible implementation for determining whether the acoustic motion direction and the wireless motion direction are consistent is as follows: when both the acoustic motion direction and the wireless motion direction are approaching or moving away from the vehicle, the acoustic motion direction and the wireless motion direction are consistent; when the acoustic motion direction is approaching the vehicle and the wireless motion direction is moving away from the vehicle; or when the acoustic motion direction is moving away from the vehicle and the wireless motion direction is approaching the vehicle, the acoustic motion direction and the wireless motion direction are inconsistent.
[0228] For example, another possible implementation for determining whether the acoustic motion direction and the wireless motion direction are consistent can be: using angle difference, direction vector dot product, or azimuth interval overlap relationship to determine whether the acoustic motion direction and the wireless motion direction are consistent. Specifically, same-source verification can be performed using any one or more of the following three methods: azimuth angle difference, motion direction vector dot product, and azimuth interval overlap relationship.
[0229] 1) Azimuth angle difference judgment: Obtain the azimuth angles of the acoustic sensing target and the wireless sensing target relative to the vehicle, calculate the angle difference between the two, and determine that the motion directions are consistent when the angle difference is less than the preset angle threshold; otherwise, they are inconsistent.
[0230] 2) Direction vector dot product discrimination: The acoustic motion direction and the wireless motion direction are constructed as planar direction vectors and a dot product operation is performed. If the dot product result is greater than a preset dot product threshold, the motion directions are determined to be consistent; otherwise, they are inconsistent.
[0231] 3) Azimuth interval overlap judgment: Determine the azimuth value intervals corresponding to acoustic targets and wireless targets respectively. If the two azimuth intervals overlap, the movement direction is determined to be the same. If the intervals do not overlap at all, they are determined to be inconsistent.
[0232] It is understood that the embodiments of the present invention adopt a multimodal fusion confidence joint decision mechanism. Through a double-insurance validity decision logic, the system determines that the collision target is real and valid and enters the braking process only when the two conditions are met simultaneously: the confidence of the multimodal fusion target is greater than a preset confidence threshold (such as 0.7) and the movement direction of the collision target output by acoustic perception and WiFi perception are consistent. This filters out false judgments caused by noise interference, signal fluctuations and single-modal anomalies from the source, making the system's false trigger rate less than 0.05 times / hour, which is far superior to the existing single-modal solution.
[0233] In this embodiment of the invention, by using the confidence level of multimodal fusion target and the consistency of motion direction as the screening conditions for collision targets, false collision targets caused by noise, multipath reflection, or local echo can be filtered out in the corners of underground garages, parking space entrances and exits, and areas obscured by walls. It also avoids the false triggering of automatic emergency braking control commands based on a single modality, thereby improving the stability and effectiveness of collision target recognition. This allows vehicles to balance braking timeliness and traffic continuity in low-speed active safety control, reduces the probability of false braking, and improves the overall safety control effect in non-line-of-sight obstructed scenarios.
[0234] Optionally, after generating acoustic sensing data and wireless sensing data, the automatic emergency braking control method provided in this embodiment of the invention further includes: performing time alignment processing on the acoustic sensing data and wireless sensing data respectively to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data; and performing spatial alignment processing on the time-aligned acoustic sensing data and time-aligned wireless sensing data respectively based on the vehicle's on-board coordinate system to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
[0235] Among them, the vehicle coordinate system is used as a unified spatial reference system for the vehicle to ensure that the perception results of different modes are expressed under the same reference; the time alignment processing is used to eliminate the deviation between acoustic perception data and wireless perception data in terms of acquisition time, buffering delay and calculation delay; the spatial alignment processing is used to eliminate the differences between acoustic perception data and wireless perception data in terms of coordinate origin, orientation definition and position mapping.
[0236] For example, one possible implementation for time alignment processing of acoustic sensing data and wireless sensing data can be: marking the acoustic sensing link and wireless sensing link with timestamps based on a unified clock source, and resampling, interpolating or nearest neighbor matching the two types of data according to a preset time window to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data with consistent time reference.
[0237] For example, time alignment can be represented by the following formula:
[0238]
[0239] in, This refers to time-aligned data (such as time-aligned acoustic sensing data or time-aligned wireless sensing data). This refers to data before time alignment (such as acoustic sensing data or wireless sensing data). This indicates the rate of change of the data's state before time alignment. This indicates the sampling time deviation between acoustic sensing data and wireless sensing data.
[0240] Understandably, the above formula is used to calibrate acoustic sensing data and wireless sensing data at different sampling times to the same reference time. For example, it extrapolates and corrects wireless sensing data at 101m to 100ms to achieve timing synchronization with acoustic sensing data.
[0241] For example, the vehicle coordinate system can be constructed as follows: with the center point of the rear axle of the vehicle as the origin, the X-axis along the direction of the vehicle's forward movement; the Y-axis along the lateral direction of the vehicle body towards the non-visual-distance blind spot; and the Z-axis vertically upwards perpendicular to the horizontal plane of the vehicle body. Alternatively, the origin can be the center of mass of the vehicle, etc. This embodiment of the invention does not limit this; the specific method can be determined according to actual application requirements.
[0242] For example, the target position can be corrected during steering using a coordinate rotation matrix in the vehicle coordinate system.
[0243] For example, the coordinate rotation matrix can be represented by the following formula:
[0244]
[0245] in, Represents the coordinate rotation matrix. This indicates the heading angle of the vehicle (the angle at which the vehicle's front is facing).
[0246] Understandably, since the microphone used to collect acoustic signal information and the wireless antenna used to collect channel state information are installed in different positions within the vehicle, to ensure the accuracy of the fusion of acoustic and wireless sensing data, it is necessary to eliminate installation offset. Based on the vehicle's coordinate system, the time-aligned acoustic and wireless sensing data are unified to the global coordinate system of the vehicle's rear axle center. Since the sensing distance is the lateral distance relative to the vehicle's rear axle center, only lateral deviation needs to be eliminated.
[0247] For example, based on the vehicle's corresponding vehicle coordinate system, one possible implementation of spatial alignment processing for time-aligned acoustic sensing data and time-aligned wireless sensing data can be: mapping the time-aligned acoustic sensing data and time-aligned wireless sensing data to the vehicle coordinate system to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
[0248] For example, spatial alignment can be represented by the following formula:
[0249]
[0250] in, Indicates the acoustic lateral distance after spatiotemporal alignment. Indicates the acoustic lateral distance after time alignment. This represents the installation offset calibration value for acoustic sensing.
[0251]
[0252] in, Indicates the wireless lateral distance after spatiotemporal alignment. This indicates the wireless horizontal distance after time alignment. This indicates the installation offset calibration value for wireless sensing.
[0253] It is understood that the embodiments of the present invention establish an acoustic and WiFi-CSI dual-modal spatiotemporal synchronization mechanism suitable for vehicle dynamic environments, construct a globally unified vehicle dynamic coordinate system with the rear axle center of the vehicle as the origin, and complete real-time correction of the sensing position through a coordinate rotation matrix; at the same time, high-precision timestamps and linear interpolation are used to complete time calibration, so that the time error between the two sensing data is <1ms and the spatial coordinates are completely unified, solving the industry problems of asynchronous, mismatched and spatially offset heterogeneous sensing data, and providing a core foundation for subsequent high-precision fusion.
[0254] In this embodiment of the invention, by performing temporal and spatial alignment processing on acoustic sensing data and wireless sensing data respectively, target position offset caused by sensor installation misalignment or sampling delay can be eliminated. For example, in a vehicle turning scenario, the lateral distance error between acoustic and wireless sensing data after temporal and spatial alignment can be controlled within ±0.15m, while the error may exceed ±0.5m when not aligned, significantly improving the accuracy of target recognition.
[0255] In this embodiment of the invention, by first completing time alignment and then spatial alignment, acoustic sensing data and wireless sensing data can be expressed at the same time and under the same coordinate reference, avoiding target mismatch caused by sampling delay or spatial offset. This method can provide consistent input for subsequent multimodal fusion, improve the accuracy of collision target identification in occlusion blind spots, reduce the probability of false triggering or missed triggering of automatic emergency braking control commands, and improve the accuracy of automatic emergency braking triggering.
[0256] Optionally, the acoustic signal information includes time-domain waveform information, frequency-domain frequency information, and acoustic signal quality characteristic information.
[0257] Among them, time-domain waveform information is used to characterize the variation characteristics of acoustic signals in the time domain, frequency-domain frequency information is used to characterize the distribution characteristics of acoustic signals in the frequency domain, and acoustic signal quality characteristic information is used to characterize the reliability of acoustic signals.
[0258] For example, the acoustic signal information includes acoustic signal information collected by at least three microphones.
[0259] For example, the acoustic signal information collected by each microphone can be represented by the following formula:
[0260]
[0261] in, This represents the acoustic signal information collected by the i-th microphone at time t. The effective acoustic signal (such as footsteps, wheel sounds, etc.) that propagates from the obstructing target to the i-th microphone after passing through the obstruction at time t is represented. Let represent the ambient noise collected by the i-th microphone at time t.
[0262] Optionally, one possible implementation of generating acoustic sensing data corresponding to the first collision target within the non-line-of-sight occlusion blind zone based on acoustic signal information in step S202 can be as follows: extracting the occlusion-penetrating acoustic signal corresponding to the first collision target based on frequency domain information; calculating the propagation delay difference corresponding to the first collision target based on the occlusion-penetrating acoustic signal; obtaining the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference; estimating the acoustic lateral motion velocity corresponding to the first collision target based on the time domain waveform information of the occlusion-penetrating acoustic signal; and determining the initial acoustic confidence level corresponding to the first collision target based on the acoustic signal quality characteristic information.
[0263] Among them, the obstruction-penetrating acoustic signal is used to represent the acoustic component that propagates to the sensor through the obstruction; the obstruction-diffraction acoustic signal is used to represent the acoustic component that propagates to the sensor around the obstruction; the propagation time delay difference is used to represent the time difference between the obstruction-penetrating acoustic signal and the microphones on different paths; and the acoustic lateral distance is used to characterize the lateral positional relationship between the first collision target and the vehicle.
[0264] For example, the sampling frequency of an acoustic sensor (such as a microphone) used to acquire acoustic signal information can be set from 16 kHz to 48 kHz, and the sampling window can be from 50 ms to 200 ms, in order to balance the capture capability of non-line-of-sight target sound sources and the time delay resolution.
[0265] It is understandable that when acoustic signals pass through obstructions such as walls, the obstructions have a strong filtering effect on high-frequency sound waves. Specifically, the energy proportion of real sound waves passing through walls is extremely low in the frequency range above 4kHz; while direct sound waves and sound waves passing around walls have a uniform energy distribution across the entire frequency range.
[0266] For example, one possible implementation for extracting the occlusion penetration acoustic signal and occlusion diffraction acoustic signal corresponding to the first collision target based on frequency domain information can be as follows: For the effective acoustic signal representing the time t after penetration by the occluder and propagation from the occluder target to the i-th microphone... Regarding the effective acoustic signal Spectral energy is statistically divided into frequency bands, resulting in a low-frequency band (200Hz-4000Hz) and a high-frequency band (4000Hz-8000Hz). The high-frequency energy proportion coefficient is calculated, and when the high-frequency energy proportion coefficient is greater than or equal to a high-frequency energy proportion coefficient threshold, it indicates that the acoustic signal is effective. The high-frequency components did not show significant attenuation, thus the effective acoustic signal was determined. To block diffracted acoustic signals; when the high-frequency energy proportion coefficient is less than the high-frequency energy proportion coefficient threshold, it indicates that the effective acoustic signal... High-frequency components are significantly attenuated by obstructions (such as walls), thus determining the validity of the acoustic signal. To block penetrating acoustic signals. The high-frequency energy proportion coefficient threshold can be 10%. This embodiment of the invention does not limit the specific value of the high-frequency energy proportion coefficient threshold; it can be determined according to actual application requirements.
[0267] For example, the high-frequency energy proportion coefficient can be expressed by the following formula:
[0268]
[0269] in, Indicates the proportion of high-frequency energy. This represents the total energy corresponding to the high-frequency band. This represents the total energy corresponding to the low-frequency band.
[0270] For example, one possible implementation for calculating the propagation delay difference corresponding to the first collision target based on the occlusion-penetrating acoustic signal is to use the cross-correlation function method to calculate the propagation delay difference of the occlusion-penetrating acoustic signal reaching different microphones.
[0271] For example, cross-correlation is performed on the two frequency band signals obtained from the division, and the value corresponding to the correlation peak is taken as the time delay difference. The cross-correlation value can be expressed by the following formula:
[0272]
[0273] in: The cross-correlation value is given, where τ represents the time difference and T represents the duration of a single frame (which can be flexibly set according to the sampling rate and signal characteristics). When a significant peak occurs, the time difference τ corresponding to that peak is the time delay difference between the two microphones. .
[0274] For example, after performing cross-correlation operations on the low-frequency and high-frequency signals respectively, the two time delay difference results are weighted and fused to obtain the final propagation time delay difference between the microphones. .
[0275] For example, one possible way to obtain the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference is to solve the geometric equations of the propagation delay difference, the sound speed parameter and the calibration position of the microphone array, and then solve for the sound source coordinates corresponding to the first collision target to obtain the acoustic lateral distance of the first collision target relative to the vehicle.
[0276] For example, when the microphone array contains 3 microphones, assume that the 3 microphones are numbered as follows: The three microphones are factory-calibrated with precise coordinates, and their positions are fixed, though affected by the actual installation layout. For example, the coordinates of the three microphones can be expressed by the following formula:
[0277]
[0278] For example, suppose the coordinates of the sound source corresponding to the first collision target to be solved are... The time delay difference of sound waves reaching the two microphones The corresponding propagation distance difference satisfies c represents the speed of sound (which can be 340 m / s).
[0279] For example, microphone To the first collision target The straight-line distance can be expressed as:
[0280]
[0281] definition This indicates that the sound has reached the microphone. With microphone The time difference between them This indicates that the sound has reached the microphone. With microphone The time difference between them, corresponding to the difference in propagation distance, satisfies the following system of equations:
[0282]
[0283] Solving the two equations simultaneously, we can obtain the position of the first collision target in the vehicle's coordinate system. .
[0284] For example, the estimation of lateral motion velocity can be performed by velocity inversion based on the time-domain waveform envelope, peak shift, and pulse repetition interval of the blocked and penetrating acoustic signal, and then smoothed by combining the waveform change trend in multiple consecutive sampling windows to reduce the impact of environmental noise and transient interference.
[0285] For example, for estimating lateral movement speed, the periodic variation of the acoustic signal penetrating the obstruction can be used to invert the lateral movement speed of the target behind the non-line-of-sight blind zone and distinguish the target type, providing a basis for graded braking decisions. Specifically, the system can extract the time interval between two consecutive characteristic sound sources in time sequence and classify the speed level based on the correspondence between the sound source period and the movement speed.
[0286] Taking pedestrians as an example, the following calibration rules are established: a footstep interval > 1.2s corresponds to slow walking, with a speed range of 0–0.5 m / s; a footstep interval between 0.7s and 1.2s corresponds to normal walking, with a speed range of 0.5 m / s to 1.2 m / s; and a footstep interval < 0.7s corresponds to running, with a speed range of 1.2 m / s to 2.0 m / s. Following this calibration logic, mapping relationships between sound emission intervals and movement speeds and speed levels are established for different sound source characteristics, such as bicycles and low-speed electric vehicles.
[0287] For example, one possible way to determine the initial acoustic confidence level corresponding to the first collision target based on acoustic signal quality feature information is to construct a confidence scoring model based on the signal-to-noise ratio, spectral entropy, echo suppression degree and feature stability in the acoustic signal quality feature information, and map the scoring result to the initial acoustic confidence level, thereby characterizing the credibility of the acoustic sensing data.
[0288] This invention, in its embodiments, extracts the occlusion-penetrating acoustic signal corresponding to the first collision target based on frequency domain information, calculates the propagation delay difference based on the occlusion-penetrating acoustic signal, further obtains the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference, estimates the acoustic lateral motion velocity corresponding to the first collision target based on the time domain waveform information of the occlusion-penetrating acoustic signal, and determines the initial acoustic confidence level corresponding to the first collision target based on the acoustic signal quality characteristic information. This enables the acoustic perception data to simultaneously characterize the lateral position, lateral motion state, and data reliability of the first collision target, making the target perception results in non-line-of-sight occlusion blind spots more complete. This improves the stability of acoustic target recognition in occluded scenarios, reduces the probability of misjudgment caused by environmental noise and echoes, and provides a more reliable input basis for subsequent multimodal fusion and automatic emergency braking control, thereby improving the collision warning accuracy of low-speed active safety systems in underground parking garages, narrow roads in residential areas, and corner areas.
[0289] Optionally, the channel state information includes signal amplitude fluctuations, signal phase changes, and multipath interference information.
[0290] Among them, signal amplitude fluctuation is used to reflect the change in wireless signal amplitude as the target moves, signal phase change is used to reflect the change in wireless signal phase as the target moves, and multipath interference information is used to characterize the degree of reflection and interference in the wireless propagation environment. The three are used together to describe the channel disturbance characteristics in the obstructed environment.
[0291] Optionally, one possible implementation of generating wireless sensing data corresponding to the second collision target in the non-line-of-sight blind zone based on channel state information in step S202 is as follows: constructing a motion index corresponding to the second collision target based on signal amplitude fluctuations and signal phase changes; obtaining the wireless lateral distance and wireless lateral motion speed corresponding to the second collision target based on the motion index; and determining the initial wireless confidence level corresponding to the second collision target based on signal phase changes and multipath interference information.
[0292] Among them, the motion index is used to characterize the motion characteristics of the second collision target in wireless sensing, and is an intermediate quantity for deriving wireless lateral distance and wireless lateral motion speed; wireless lateral distance is used to represent the lateral position of the second collision target relative to the vehicle; wireless lateral motion speed is used to represent the lateral motion trend of the second collision target relative to the vehicle; and initial wireless confidence is used to characterize the initial credibility of the wireless sensing data.
[0293] Understandably, the characteristic of in-vehicle WiFi signals being disturbed by human bodies can be utilized to independently detect whether there is a moving target behind an obstruction, how far away it is, and whether it is close, without relying on sound at all, thus compensating for the disadvantage of acoustics being greatly affected by noise.
[0294] For example, in one possible implementation, signal amplitude fluctuations and signal phase changes can be extracted based on the channel estimation results between the transmitter and receiver, and multipath interference information can be statistically modeled. Further, the amplitude change and phase difference components are input into a preset motion index to construct a model. The model can be implemented using linear weighting, regression fitting, or neural network mapping, thereby outputting a motion index value related to the target's lateral movement. Subsequently, the wireless lateral distance and wireless lateral movement speed are obtained by inversion based on this motion index, and these are output as the location and speed information of the wireless sensing data. Simultaneously, an initial wireless confidence level can be assigned to the wireless sensing result based on the stability of the phase change and the strength of the multipath interference information to avoid interference from complex reflection environments on target judgment.
[0295] For example, for the channel state information collected, we can first extract dynamic signals to delete the signals corresponding to static physical objects such as walls, pillars, and the ground, and only retain the changes brought about by moving objects; then, based on the intensity of signal fluctuations caused by moving objects, we can establish a motion index.
[0296] For example, the fitness index can be expressed by the following formula:
[0297]
[0298] in, Indicates the fitness index. Indicates signal amplitude fluctuation. This indicates a change in the phase of the signal.
[0299] For example, based on the rate of change of the motion index over time, an empirical formula is established to estimate the wireless lateral distance and wireless lateral motion speed of the second collision target.
[0300] This invention, through the construction of a motion index based on signal amplitude fluctuations and phase changes, can suppress false detections in wireless sensing caused by reflections from static obstacles. For example, in a parking lot scenario, the motion index can filter out false targets caused by wall reflections, reducing the false detection rate of wireless sensing to below 5%.
[0301] For example, the wireless lateral distance can be expressed by the following formula:
[0302]
[0303] in, Indicates the horizontal distance of the wireless circuit. Indicates the distance calibration coefficient. This represents the zero-resistance constant, which can take a value of 0.01.
[0304] For example, the wireless lateral movement speed can be expressed by the following formula:
[0305]
[0306] in, Indicates the speed of wireless lateral movement. Indicates the speed calibration coefficient. This represents the rate of change of the motion index over time.
[0307] Understandably, when the vehicle is in an obstructed area such as an underground parking garage, a corner road, or a parked vehicle, the presence of a second collision target may only be apparent through channel perturbations. In this case, the motion index can transform the coupled changes in amplitude and phase into calculable motion representations, thereby obtaining the lateral distance and lateral velocity of the collision target. The initial wireless confidence level is then used to reflect the reliability of this result in the current propagation environment. Through this method, wireless sensing data can maintain its ability to represent the target's position and motion state even under non-line-of-sight conditions.
[0308] The embodiments of the present invention can incorporate channel amplitude variation, phase variation and multipath interference information into the wireless sensing modeling process, improve the stability of second collision target identification, and enhance the continuity and consistency of lateral distance and lateral motion velocity estimation, thereby providing a more reliable wireless sensing foundation for subsequent target fusion and braking control.
[0309] Figure 5 A flowchart illustrating the automatic emergency braking control method provided in this embodiment of the invention. Figure 4 .like Figure 5 As shown, a specific implementation of this automatic emergency braking control method may further include the following steps:
[0310] S501, in response to the fact that the first collision target and the second collision target are not the same collision target, determine the collision time of the second target based on acoustic sensing data and wireless sensing data.
[0311] The method for determining whether the first collision target and the second collision target are not the same collision target is similar to that described above, and will not be repeated here.
[0312] Acoustic sensing data and wireless sensing data are similar to those described above, and will not be repeated here.
[0313] Optionally, one possible implementation of this step is as follows: Calculate the third collision time of the first collision target reaching the second collision point based on the acoustic lateral distance and acoustic lateral velocity corresponding to the first collision target in the acoustic sensing data; the second collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the first collision target; calculate the fourth collision time of the second collision target reaching the third collision point based on the wireless lateral distance and wireless lateral velocity corresponding to the second collision target in the wireless sensing data; the third collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the second collision target; and take the smaller of the third and fourth collision times to obtain the second target collision time.
[0314] The third collision time is used to characterize the approach time of the first collision target calculated based on acoustic sensing data, and the fourth collision time is used to characterize the approach time of the second collision target calculated based on wireless sensing data.
[0315] The determination methods for the second and third collision points are similar to those for the first collision point, and will not be repeated here.
[0316] It is understandable that by taking the smaller of the third and fourth collision times, the second target collision time can be obtained, so that the output corresponds to the target with an earlier collision risk. This can avoid the delayed judgment caused by the single-mode estimation bias and provide a more conservative time reference for subsequent automatic emergency braking control commands.
[0317] It is understood that this embodiment can simultaneously take into account the timing information of acoustic perception and wireless perception in non-line-of-sight occlusion scenarios, improve the timeliness and robustness of second target collision time estimation, reduce the risk of false braking or delayed braking caused by single-mode error, and improve the accuracy and safety of automatic emergency braking control command output.
[0318] S502 outputs an automatic emergency braking control command based on the collision time with the second target.
[0319] The specific implementation of this step is similar to the above-mentioned output of automatic emergency braking control command based on the collision time of the first target, and will not be repeated here.
[0320] In this embodiment of the invention, in response to the fact that the first collision target and the second collision target are not the same collision target, the collision time of the second target is determined based on acoustic sensing data and wireless sensing data. Furthermore, based on the collision time of the second target, an automatic emergency braking control command is output, making the output of the automatic emergency braking control command more in line with the real collision risk, improving the collision recognition accuracy and braking trigger stability in complex non-line-of-sight blind spot scenarios, reducing the impact of environmental noise, wireless multipath fluctuations and target inconsistency, and improving the active safety performance of the vehicle in underground garages, community roads and corner obstruction areas.
[0321] It should be noted that, in order to achieve braking stability control and safety constraints, the automatic emergency braking control method provided in this embodiment of the invention also includes real-time monitoring and constraint control of the vehicle's attitude during the braking process, to ensure that the vehicle does not experience secondary dangers such as sideslip, fishtailing, or steering instability when performing graded braking, and to ensure the safety, smoothness, and vehicle stability of the braking process.
[0322] Optionally, the automatic emergency braking control method provided in this embodiment of the invention further includes, during the execution of the automatic emergency braking control command: acquiring the vehicle's attitude information, including yaw rate and sideslip angle; determining whether the yaw rate is less than or equal to a yaw rate threshold and whether the sideslip angle is less than or equal to a sideslip angle threshold; if the yaw rate is less than or equal to the yaw rate threshold and the sideslip angle is less than or equal to the sideslip angle threshold, then the automatic emergency braking control command is continuously executed; if the yaw rate is greater than the yaw rate threshold or the sideslip angle is greater than the sideslip angle threshold, then a stability protection mechanism is activated.
[0323] Among them, attitude information is used to characterize the dynamic stability of the vehicle during braking, yaw rate is used to reflect the rotation intensity of the vehicle body around the vertical axis, and sideslip angle is used to reflect the degree of deviation between the vehicle body's orientation and the actual direction of movement.
[0324] For example, the yaw rate threshold and the sideslip angle threshold are used to limit the acceptable range of attitude changes of the vehicle during braking. Exceeding the corresponding threshold indicates that the vehicle may have a risk of sideslip, fishtailing, or steering instability. The yaw rate threshold can be 15° / s, and the sideslip angle threshold can be 10°. In this embodiment of the invention, the specific values of the yaw rate threshold and the sideslip angle threshold are not limited; they can be determined according to the actual application requirements.
[0325] In this embodiment of the invention, a stability protection mechanism is activated when the yaw rate exceeds a threshold (e.g., 15° / s) or the sideslip angle exceeds a threshold (e.g., 10°), which can prevent vehicle skidding caused by sudden braking. For example, in a wet and slippery road surface scenario, the system can automatically reduce the braking intensity by 40% to prioritize vehicle stability.
[0326] Among them, the stability protection mechanism is used to limit, correct or switch the braking control when abnormal vehicle posture is detected, so as to reduce the impact of automatic emergency braking on vehicle handling stability.
[0327] For example, in one possible implementation, attitude information can be acquired in real time by an onboard inertial measurement unit, a vehicle stability control system, or a chassis domain controller. Yaw rate can be directly measured by a yaw rate sensor or estimated by wheel speed, steering angle sensors, and a vehicle kinematic model. Side slip angle can be calculated by an extended Kalman filter, a state observer, or a fusion positioning module based on the vehicle's lateral and longitudinal velocities, yaw rate, and steering input.
[0328] For example, in one possible implementation, the yaw rate threshold and the sideslip angle threshold can be pre-calibrated based on the vehicle wheelbase, tire adhesion coefficient, braking intensity and road conditions, and stored in the controller calibration parameter area. Different threshold groups can be used in low-adhesion road surfaces or high deceleration braking scenarios to adapt to the vehicle's dynamic characteristics.
[0329] For example, in one possible implementation, when the judgment result meets the conditions for continuously executing the automatic emergency braking control command, the controller maintains the original automatic emergency braking control command output unchanged so that the braking force continues to act on the braking actuator; when the judgment result meets the conditions for activating stability protection, the controller switches to the stability protection mechanism, which usually corrects the attitude by reducing the braking pressure growth rate, adjusting the front and rear wheel braking force distribution, or linking with the electronic stability program.
[0330] For example, in one possible implementation, when the stability protection mechanism is activated, the braking deceleration is automatically reduced by 40% to 60% to prioritize vehicle stability. Once the vehicle's attitude returns to a safe range, the braking intensity is gradually restored, achieving the safety logic of "stabilizing the vehicle first, then braking." This embodiment of the invention does not limit the specific control strategy corresponding to the stability protection mechanism; it can be determined based on actual application requirements.
[0331] In this embodiment of the invention, braking safety and vehicle stability are combined through threshold determination. When the vehicle attitude is within a stable range, the system can maintain braking output without interruption, thereby ensuring the collision risk suppression effect. When the vehicle attitude approaches or exceeds the stability boundary, the system can activate the protection mechanism in time to avoid instability caused by sudden braking. It can balance braking effect and driving stability in low-speed active safety control scenarios, and improve the reliability and applicability of the automatic emergency braking control process.
[0332] Based on the above embodiments, the following is combined with Figure 6 The automatic emergency braking control system provided in the embodiments of the present invention will be described in detail.
[0333] Figure 6 This is a schematic diagram of the automatic emergency braking control system provided in an embodiment of the present invention. Figure 6 As shown, the automatic emergency braking control system includes an acoustic sensing unit, a wireless sensing unit, a spatiotemporal alignment unit, a confidence assessment unit, a scene recognition unit, a scene adaptive weighted fusion unit, a fusion target calculation unit, a collision risk assessment unit, a multimodal AEB control unit, a braking stability protection unit, and a modal redundancy and fault degradation unit, forming a complete closed-loop control process of "dual perception - synchronous alignment - confidence scoring - scene recognition - scene adaptive fusion - risk assessment - graded braking - stability protection - redundancy fault tolerance".
[0334] Among them, the acoustic sensing unit is used to collect acoustic signal information in the non-line-of-sight blind spot corresponding to the vehicle based on the vehicle-mounted acoustic sensor (such as a multi-microphone array), and to process the acoustic signal information to generate and output acoustic sensing data.
[0335] The wireless sensing unit is used to collect channel status information based on the vehicle-mounted wireless module, and to process the channel status information to generate and output wireless sensing data.
[0336] The spatiotemporal alignment unit is used to establish a unified spatiotemporal reference for vehicle dynamics and to perform spatiotemporal alignment on acoustic sensing data and wireless sensing data. That is, to unify the two sensing data into the same millisecond time reference and vehicle unified coordinate system, ensuring that the time error is <1ms and the spatial position is completely coincident. This achieves millisecond-level time synchronization of dual-modal sensing data and unified global coordinate mapping, fundamentally solving the problems of asynchronous, mismatched and spatial offset heterogeneous sensing data, and laying the foundation for high-precision and effective information fusion.
[0337] The confidence assessment unit is used to quantitatively score the acoustic sensing results and WiFi sensing results respectively, providing a basis for dynamic weight allocation and reliability judgment. For example, it corrects the initial acoustic confidence in the acoustic sensing data to obtain the corrected acoustic confidence corresponding to the acoustic sensing single mode, and corrects the initial wireless confidence in the wireless sensing data to obtain the corrected wireless confidence corresponding to the wireless sensing single mode.
[0338] The scene recognition unit is used to intelligently identify the driving scene type of the vehicle based on environmental noise intensity, WiFi signal multipath intensity, vehicle speed, location information, etc., such as quiet garage, noisy road, multi-metal garage, mixed complex scene, and support scene adaptive decision-making.
[0339] The scene-adaptive weighted fusion unit is used to dynamically allocate perception weights based on the real-time scene and confidence level. That is, it combines scene weights (such as acoustic scene confidence and wireless scene confidence) with real-time confidence to dynamically correct weights (such as corrected acoustic confidence and corrected wireless confidence) to calculate the lateral distance, lateral motion velocity, and confidence level of the fused target. This achieves intelligent fusion based on "whoever is reliable, trust more", and outputs more accurate and stable lateral distance, lateral motion velocity, and confidence level of the fused target than single-modality fusion.
[0340] The fusion target calculation unit adopts a multimodal joint confidence decision mechanism to determine the validity of the collision target. That is, the collision target is determined to be valid only when the confidence of the fusion target meets the standard and the collision target motion direction of the dual-modal output is consistent, and the braking control process is started to suppress false triggering to the maximum extent and ensure system reliability.
[0341] The collision risk assessment unit constructs a time-to-collision (TTC) triple-constraint risk assessment model, which combines the basic collision time, road surface adhesion coefficient, scene correction coefficient, and fusion target confidence to achieve accurate risk level classification.
[0342] The multimodal AEB control unit adopts a graded braking strategy to output automatic emergency braking control commands such as warning, pre-pressure build-up, light braking, medium braking, and full braking according to the risk level. It achieves a smooth transition from warning, pre-pressure build-up, light braking, medium braking, to full braking according to the risk level, ensuring braking safety, comfort, and smoothness.
[0343] The braking stability protection unit is used to monitor the vehicle attitude in real time. By introducing body attitude constraints and braking stability protection mechanisms, it monitors the vehicle's yaw rate and sideslip angle in real time to prevent braking instability, skidding, and fishtailing, thus meeting automotive safety requirements.
[0344] The modal redundancy and fault degradation unit is used to monitor the status of the sensing branch in real time. It supports three-level switching: dual-mode fusion, single-mode redundancy, and original vehicle degradation, to ensure the safe operation of the system under any working condition.
[0345] Optionally, a specific implementation of the modal redundancy and fault degradation unit can be as follows: when both the acoustic sensing unit and the wireless sensing unit are normal, the automatic emergency braking control system activates the dual-modal fusion mode; when the acoustic sensing unit fails, the automatic emergency braking control system activates the wireless sensing mode; when the wireless sensing unit fails, the automatic emergency braking control system activates the acoustic sensing mode; when both the acoustic sensing unit and the wireless sensing unit fail, the automatic emergency braking control system disables the non-line-of-sight sensing function.
[0346] To monitor the real-time operating status of each module in the system, achieve fault identification, redundancy switching, and system recovery after risk mitigation, and ensure safe system operation under any conditions without functional failure risk, meeting automotive-grade functional safety requirements, this invention embodiment performs real-time cyclic self-checks on the acoustic sensing branch, wireless sensing branch, data bus, and controller status, and automatically switches operating modes based on diagnostic results. In one possible implementation, when both the acoustic and wireless sensing units are normal, the system operates in dual-modal fusion mode, using multi-modal fused target perception information for decision-making, resulting in optimal performance. When the acoustic sensing unit fails but the wireless sensing unit is normal, the system automatically switches to pure wireless sensing mode, continuing to detect collision targets in non-line-of-sight blind spots. When the wireless sensing unit fails but the acoustic sensing unit is normal, the system automatically switches to pure acoustic sensing mode, maintaining the ability to detect collision targets in non-line-of-sight blind spots. When both the acoustic and wireless sensing units fail in both modes, the system immediately shuts down the non-line-of-sight sensing function and seamlessly switches to the vehicle's basic AEB system, without affecting the vehicle's basic safety braking capability. In the event of a fault, the system will provide a warning via the instrument panel's fault indicator light and store the fault code for future maintenance.
[0347] Compared to related technologies, which lack a multimodal confidence joint decision mechanism, the AEB system is prone to false triggering or missed triggering when a single perception mode is interfered with or fails, resulting in a poor driving experience and insufficient safety. In this embodiment of the invention, a dual-modal redundancy backup mechanism is adopted. The failure of a single mode does not affect the operation of the function, and the dual failures can be safely rolled back. Combined with attitude constraint protection, the braking process is stable and does not become unstable, reducing the false triggering rate or missed triggering rate of the AEB system. At the same time, the driving experience and safety are improved.
[0348] Optionally, the automatic emergency braking control system provided in this embodiment of the invention may further include a risk release and system recovery unit (not shown), which is used to release the braking step by step and restore normal monitoring after detecting that the collision target in the non-line-of-sight blind zone has moved away and the risk has disappeared.
[0349] For example, when the collision target in the non-line-of-sight blind spot moves away from the vehicle and the time to collision (TTC) returns to a safe range (e.g., greater than 4.5 seconds), the system determines that the risk has been eliminated, gradually reduces the braking intensity until the braking is completely released, turns off the audible and visual warnings, and returns to normal monitoring status. At the same time, the system automatically clears the current collision target information, re-enters the perception process, and continues to monitor the non-line-of-sight blind spot, waiting for the next risk to occur.
[0350] Understandably, compared to the shortcomings of single-modal non-line-of-sight perception solutions, such as poor environmental adaptability, incompatibility between accuracy and robustness, lack of unified spatiotemporal reference, lack of adaptive fusion, lack of redundancy protection, and lack of automotive-grade braking strategies, the automatic emergency braking control system provided in this embodiment of the invention deeply integrates on-board distributed microphone acoustic perception with on-board WiFi CSI wireless channel perception to form a complementary and redundant non-line-of-sight perception architecture. It leverages the high accuracy of acoustic perception and the high robustness of WiFi perception to achieve optimal perception performance across all scenarios. Furthermore, by constructing a systematic, automotive-grade solution that integrates spatiotemporal unification, confidence assessment, scene adaptation, weighted fusion, joint decision-making, graded braking, attitude constraints, redundancy protection, and failure degradation, it achieves a synergistic gain of 1+1>2, solving the safety problem of ghost-protruding vehicles in non-line-of-sight blind spots.
[0351] In summary, the automatic emergency braking control method and system provided in this invention, by constructing a complete technical system that includes millisecond-level spatiotemporal synchronization, global vehicle coordinate unification, single-modal confidence assessment, intelligent driving scenario recognition, scenario adaptive weighted fusion, multimodal joint risk decision-making, graded cooperative braking, vehicle posture constraint protection, and multimodal redundancy fault tolerance and failure degradation, can solve the high-risk accident problem of "ghost peeks" in typical low-speed driving scenarios such as underground garages, narrow roads in residential areas, building obstruction, blind spots at intersections, and obstacle obstruction. It achieves high-precision, high-robustness, high-reliability, and all-weather perception of vulnerable road users such as pedestrians and non-motorized vehicles in obstructed areas and non-line-of-sight areas. Based on the fused perception results, it drives the AEB system to achieve early warning, pre-pressure build-up, graded braking, and smooth stopping, fundamentally making up for the technical defects of traditional line-of-sight sensors such as vision, millimeter-wave radar, lidar, and ultrasonic radar that cannot penetrate obstacles and fail to perceive blind spots, thus significantly improving the non-line-of-sight active safety performance of intelligent vehicles.
[0352] In summary, the automatic emergency braking control method and system provided by the embodiments of the present invention have the following beneficial effects:
[0353] 1) Dual improvement in perception accuracy and robustness: Fusion distance error ≤ ±0.25m, speed error ≤ ±0.15m / s, direction recognition accuracy ≥95%, while possessing high accuracy in quiet scenes and high robustness in noisy or strong multipath scenes. The performance in all scenes surpasses that of single-modal perception, achieving all-weather, all-scene, and all-working-condition coverage.
[0354] 2) The system's safety and reliability are greatly improved: dual-modal redundancy backup ensures that the failure of a single mode does not affect the operation of the function, and the system can safely roll back after a double failure. Combined with attitude constraint protection, the braking process is stable and does not become unstable.
[0355] 3) Comprehensive optimization of AEB braking performance: Achieves braking 150ms-300ms earlier than traditional AEB, shortens the stopping distance by 0.5m-1.2m, provides smooth and shock-free staged braking, avoids accidental braking, and improves the comfort of drivers and passengers;
[0356] 4) Extremely low hardware cost and strong adaptability: It fully reuses the original vehicle's distributed microphone and in-vehicle WiFi module, without the need for any additional sensors, controllers, or wiring harnesses, and can be quickly installed in various mass-produced models.
[0357] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.
[0358] Figure 7 This is a schematic diagram of the automatic emergency braking control device provided in an embodiment of the present invention. Figure 7As shown, the automatic emergency braking control device 70 includes an acquisition module 710, a generation module 720, a fusion processing module 730, and a control module 740.
[0359] The acquisition module 710 is used to acquire multimodal environmental information in the non-line-of-sight blind spot corresponding to the vehicle, and the multimodal environmental information includes acoustic signal information and channel state information.
[0360] The generation module 720 is used to generate acoustic sensing data corresponding to the first collision target in the non-line-of-sight occlusion blind zone based on acoustic signal information, and to generate wireless sensing data corresponding to the second collision target in the non-line-of-sight occlusion blind zone based on channel state information.
[0361] The fusion processing module 730 is used to perform fusion processing on acoustic sensing data and wireless sensing data in response to the first collision target and the second collision target being the same collision target, to generate multimodal fusion target sensing information;
[0362] The control module 740 is used to output automatic emergency braking control commands based on multimodal fusion target perception information, acoustic perception data and wireless perception data, and to execute the automatic emergency braking control commands.
[0363] In one possible implementation, the multimodal environment information also includes visual perception information. The fusion processing module 730 is specifically used to: generate target acoustic scene confidence and target wireless scene confidence based on the visual perception information and acoustic signal information; and perform fusion processing on the acoustic perception data and wireless perception data based on the target acoustic scene confidence and target wireless scene confidence to generate multimodal fused target perception information.
[0364] In one possible implementation, the acoustic sensing data includes an initial acoustic confidence level corresponding to the first collision target, and the wireless sensing data includes an initial wireless confidence level corresponding to the second collision target. Based on the target acoustic scene confidence level and the target wireless scene confidence level, the fusion processing module 730 is further configured to: correct the initial acoustic confidence level and the initial wireless confidence level respectively to obtain corrected acoustic confidence level and corrected wireless confidence level; perform fusion processing on the corrected acoustic confidence level and the corrected wireless confidence level based on the target acoustic scene confidence level and the target wireless scene confidence level to generate fused acoustic confidence level and fused wireless confidence level; and perform fusion processing on the acoustic sensing data and the wireless sensing data based on the fused acoustic confidence level and the fused wireless confidence level to generate multimodal fused target sensing information.
[0365] In one possible implementation, the acoustic sensing data further includes the acoustic lateral distance between the first collision target and the vehicle and the acoustic lateral velocity corresponding to the first collision target; the wireless sensing data further includes the wireless lateral distance between the second collision target and the vehicle and the wireless lateral velocity corresponding to the second collision target; the fusion processing module 730 is further configured to: perform fusion processing on the acoustic lateral distance and the wireless lateral distance based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral distance; perform fusion processing on the acoustic lateral velocity and the wireless lateral velocity based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target lateral velocity; and perform fusion processing on the corrected acoustic confidence and the corrected wireless confidence based on the fused acoustic confidence and the fused wireless confidence to generate a multimodal fused target confidence.
[0366] In one possible implementation, the fusion processing module 730 is further configured to: determine the target driving scene type of the vehicle based on visual perception information and acoustic signal information; determine the target acoustic scene confidence based on the target driving scene type based on a first mapping relationship between the driving scene type and the acoustic scene confidence; and calculate the target wireless scene confidence based on the target acoustic scene confidence.
[0367] In one possible implementation, the control module 740 is specifically used to: determine whether the collision target is valid based on the multimodal fusion target perception information, acoustic perception data and wireless perception data; if the collision target is valid, output an automatic emergency braking control command based on the multimodal fusion target perception information; if the collision target is invalid, do not output an automatic emergency braking control command.
[0368] In one possible implementation, the control module 740 is further configured to: calculate the first collision time of the colliding target reaching the first collision point based on the lateral distance and lateral velocity of the multimodal fusion target in the multimodal fusion target perception information, wherein the first collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the colliding target; determine the first target collision time based on the first collision time, the second collision time, and the confidence level of the multimodal fusion target in the multimodal fusion target perception information, wherein the second collision time is the collision time of the vehicle reaching the first collision point; and output an automatic emergency braking control command based on the first target collision time.
[0369] In one possible implementation, the control module 740 is further configured to: determine the target risk level based on the first target collision time according to a second mapping relationship between the collision time and the risk level; and output an automatic emergency braking control command according to the target risk level.
[0370] In one possible implementation, the control module 740 is further configured to: obtain an intermediate collision time based on the first collision time and the second collision time; and correct the intermediate collision time based on the multimodal fusion target confidence, road surface adhesion coefficient, and scene correction coefficient to obtain the target collision time.
[0371] In one possible implementation, the acoustic sensing data further includes the acoustic motion direction of the first collision target, and the wireless sensing data further includes the wireless motion direction corresponding to the second collision target. The control module 740 is further configured to: determine whether the confidence level of the multimodal fusion target is greater than a preset confidence threshold, and determine whether the acoustic motion direction is consistent with the wireless motion direction; if the confidence level of the multimodal fusion target is greater than the preset confidence threshold, and the acoustic motion direction is consistent with the wireless motion direction, then the collision target is determined to be valid; if the confidence level of the multimodal fusion target is less than or equal to the preset confidence threshold, or the acoustic motion direction is inconsistent with the wireless motion direction, then the collision target is determined to be invalid.
[0372] In one possible implementation, after generating acoustic sensing data and wireless sensing data, the automatic emergency braking control device further includes a spatiotemporal alignment module (not shown). This spatiotemporal alignment module is used to: perform time alignment processing on the acoustic sensing data and wireless sensing data respectively to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data; and perform spatial alignment processing on the time-aligned acoustic sensing data and time-aligned wireless sensing data respectively based on the vehicle's on-board coordinate system to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
[0373] In one possible implementation, the acoustic signal information includes time-domain waveform information, frequency-domain frequency information, and acoustic signal quality feature information. The generation module 720 is specifically used to: extract the occlusion-penetration acoustic signal corresponding to the first collision target based on the frequency-domain frequency information; calculate the propagation delay difference corresponding to the first collision target based on the occlusion-penetration acoustic signal; obtain the acoustic lateral distance corresponding to the first collision target based on the propagation delay difference; estimate the acoustic lateral motion velocity corresponding to the first collision target based on the time-domain waveform information of the occlusion-penetration acoustic signal; and determine the initial acoustic confidence level corresponding to the first collision target based on the acoustic signal quality feature information.
[0374] In one possible implementation, the channel state information includes signal amplitude fluctuations, signal phase changes, and multipath interference information. The generation module 720 is specifically used to: construct the motion index corresponding to the second collision target based on the signal amplitude fluctuations and signal phase changes; obtain the wireless lateral distance and wireless lateral motion speed corresponding to the second collision target based on the motion index; and determine the initial wireless confidence level corresponding to the second collision target based on the signal phase changes and multipath interference information.
[0375] In one possible implementation, the control module 740 is further configured to: in response to the first collision target and the second collision target not being the same collision target, determine the collision time of the second target based on acoustic sensing data and wireless sensing data; and output an automatic emergency braking control command based on the collision time of the second target.
[0376] In one possible implementation, the control module 740 is further configured to: calculate the third collision time of the first collision target reaching the second collision point based on the acoustic lateral distance and acoustic lateral motion velocity corresponding to the first collision target in the acoustic sensing data, wherein the second collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the first collision target; calculate the fourth collision time of the second collision target reaching the third collision point based on the wireless lateral distance and wireless lateral motion velocity corresponding to the second collision target in the wireless sensing data, wherein the third collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the second collision target; and take the smaller of the third collision time and the fourth collision time to obtain the second target collision time.
[0377] In one possible implementation, during the execution of the automatic emergency braking control command, the automatic emergency braking control device further includes a monitoring module (not shown), which is used to: acquire the vehicle's attitude information, including yaw rate and sideslip angle; determine whether the yaw rate is less than or equal to a yaw rate threshold, and determine whether the sideslip angle is less than or equal to a sideslip angle threshold; if the yaw rate is less than or equal to the yaw rate threshold and the sideslip angle is less than or equal to the sideslip angle threshold, then the automatic emergency braking control command is continuously executed; if the yaw rate is greater than the yaw rate threshold or the sideslip angle is greater than the sideslip angle threshold, then a stability protection mechanism is activated.
[0378] The automatic emergency braking control device provided in this embodiment of the invention can be used to execute the method steps of the above method embodiment. The specific implementation and technical effects are similar, and will not be repeated here.
[0379] This invention also provides a vehicle, including a vehicle body and an automatic emergency braking control system, which is used to execute the automatic emergency braking control method provided in the above embodiments.
[0380] The automatic emergency braking control system is similar to that described above, and will not be repeated here.
[0381] Understandably, this vehicle, by equipping itself with an automatic emergency braking control system, enables the system to perform automatic emergency braking in non-line-of-sight obstructed scenarios such as underground parking garages, building corners, and temporary parking areas. The automatic emergency braking control system generates target perception results based on acoustic signal information and channel state information, and then fuses these results to determine the same collision target. This improves the completeness and consistency of collision target identification within non-line-of-sight blind spots, allowing for early detection of collision risk cues before the target enters the effective field of view of traditional line-of-sight sensors. This provides the vehicle with earlier risk warnings and braking responses to sudden targets such as those appearing out of nowhere. Furthermore, by combining multimodal fusion of target perception information, acoustic perception data, and wireless perception data to output automatic emergency braking control commands, the system balances the accuracy and stability of braking triggering, reducing false braking and missed braking, thereby improving the vehicle's active safety performance and driving experience at low speeds.
[0382] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 8 As shown, the electronic device 80 includes at least one processor 801 and a memory 802.
[0383] Optionally, the electronic device 80 also includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.
[0384] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to perform the above-described method.
[0385] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0386] For example, this electronic device can be deployed in a vehicle controller, domain controller, or central computing platform. After the processor calls the computer in memory to execute instructions, it can perform an automatic emergency braking control method in non-line-of-sight blind spot scenarios. It processes acoustic signal information and channel state information, and forms acoustic perception results and wireless perception results respectively. The processor further determines the corresponding collision targets under different modalities. When it is confirmed that they belong to the same collision target, it completes the fusion processing to obtain more stable target perception information, so that the system still has a high risk identification capability under the conditions of obstruction by walls, parked vehicles, or building corners. Based on the fused target perception information and the perception results of each single modality, the automatic emergency braking control command is output, thus taking into account both the timeliness and accuracy of braking triggering. Therefore, it can reduce the probability of false braking and missed braking, and improve the reliability of low-speed active safety control.
[0387] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0388] The memory may include random access memory (RAM) and non-volatile memory (NVM), such as at least one disk storage device.
[0389] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0390] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0391] This invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0392] For example, by pre-storing the computer execution instructions corresponding to the automatic emergency braking control method in a computer-readable storage medium, the vehicle processor can directly call and execute processing flows such as acoustic signal information acquisition, channel state information acquisition, acoustic perception data generation, wireless perception data generation, same collision target determination, multi-modal fusion target perception information generation, and automatic emergency braking control command output. This allows the automatic emergency braking control method to be stably deployed in the intelligent vehicle control system as a software program. This facilitates integration and upgrades across different vehicle models or hardware platforms, and enables the vehicle to reliably perform collision target recognition and braking control in non-line-of-sight blind spots such as underground parking garages, building corners, and parked vehicles, thereby improving target detection accuracy, collision recognition reliability, and automatic emergency braking triggering stability.
[0393] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0394] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0395] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0396] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0397] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0398] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0399] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0400] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. An automatic emergency braking control method, characterized in that, include: Acquire multimodal environmental information within the non-line-of-sight blind spot corresponding to the vehicle, wherein the multimodal environmental information includes acoustic signal information and channel state information; Based on the acoustic signal information, acoustic sensing data corresponding to the first collision target in the non-line-of-sight occlusion blind zone is generated, and based on the channel state information, wireless sensing data corresponding to the second collision target in the non-line-of-sight occlusion blind zone is generated. In response to the first collision target and the second collision target being the same collision target, the acoustic sensing data and the wireless sensing data are fused to generate multimodal fused target sensing information; Based on the multimodal fusion target perception information, the acoustic perception data, and the wireless perception data, an automatic emergency braking control command is output and executed.
2. The automatic emergency braking control method according to claim 1, characterized in that, The multimodal environmental information also includes visual perception information. The process of fusing the acoustic perception data and the wireless perception data to generate multimodal fused target perception information includes: Based on the visual perception information and the acoustic signal information, generate the target acoustic scene confidence score and the target wireless scene confidence score; Based on the confidence scores of the target acoustic scene and the target wireless scene, the acoustic sensing data and the wireless sensing data are fused to generate the multimodal fused target sensing information.
3. The automatic emergency braking control method according to claim 2, characterized in that, The acoustic sensing data includes the initial acoustic confidence level corresponding to the first collision target, and the wireless sensing data includes the initial wireless confidence level corresponding to the second collision target. The step of fusing the acoustic sensing data and the wireless sensing data based on the target acoustic scene confidence level and the target wireless scene confidence level to generate the multimodal fused target sensing information includes: The initial acoustic confidence and the initial wireless confidence are corrected respectively to obtain the corrected acoustic confidence and the corrected wireless confidence. Based on the target acoustic scene confidence and the target wireless scene confidence, the corrected acoustic confidence and the corrected wireless confidence are fused to generate fused acoustic confidence and fused wireless confidence. Based on the fused acoustic confidence and the fused wireless confidence, the acoustic sensing data and the wireless sensing data are fused to generate the multimodal fused target sensing information.
4. The automatic emergency braking control method according to claim 3, characterized in that, The acoustic sensing data also includes the acoustic lateral distance between the first collision target and the vehicle and the acoustic lateral motion speed corresponding to the first collision target; the wireless sensing data also includes the wireless lateral distance between the second collision target and the vehicle and the wireless lateral motion speed corresponding to the second collision target. The process of fusing the acoustic sensing data and the wireless sensing data based on the fused acoustic confidence score and the fused wireless confidence score to generate the multimodal fused target sensing information includes: Based on the fused acoustic confidence and the fused wireless confidence, the acoustic lateral distance and the wireless lateral distance are fused to generate a multimodal fused target lateral distance; Based on the fused acoustic confidence and the fused wireless confidence, the acoustic lateral motion velocity and the wireless lateral motion velocity are fused to generate a multimodal fused target lateral motion velocity. Based on the fused acoustic confidence and the fused wireless confidence, the corrected acoustic confidence and the corrected wireless confidence are fused to generate a multimodal fused target confidence.
5. The automatic emergency braking control method according to claim 2, characterized in that, The step of generating target acoustic scene confidence and target wireless scene confidence based on the visual perception information and the acoustic signal information includes: The target driving scenario type of the vehicle is determined based on the visual perception information and the acoustic signal information. Based on the first mapping relationship between driving scenario type and acoustic scenario confidence, the target acoustic scenario confidence is determined according to the target driving scenario type; Calculate the confidence level of the target wireless scene based on the confidence level of the target acoustic scene.
6. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, The step of outputting an automatic emergency braking control command based on the multimodal fused target perception information, the acoustic perception data, and the wireless perception data includes: Based on the multimodal fused target perception information, the acoustic perception data, and the wireless perception data, determine whether the collision target is valid; If the collision target is valid, the automatic emergency braking control command is output based on the multimodal fused target perception information. If the collision target is invalid, the automatic emergency braking control command will not be output.
7. The automatic emergency braking control method according to claim 6, characterized in that, The step of outputting the automatic emergency braking control command based on the multimodal fused target perception information includes: Based on the lateral distance and lateral velocity of the multimodal fusion target in the multimodal fusion target perception information, the first collision time of the collision target reaching the first collision point is calculated. The first collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the collision target. The first target collision time is determined based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information; the second collision time is the collision time when the vehicle arrives at the first collision point. Based on the collision time with the first target, the automatic emergency braking control command is output.
8. The automatic emergency braking control method according to claim 7, characterized in that, The step of outputting the automatic emergency braking control command based on the first target collision time includes: Based on the second mapping relationship between collision time and risk level, the target risk level is determined according to the first target collision time; Based on the target risk level, the automatic emergency braking control command is output.
9. The automatic emergency braking control method according to claim 7, characterized in that, Determining the first target collision time based on the first collision time, the second collision time, and the multimodal fusion target confidence in the multimodal fusion target perception information includes: The intermediate collision time is obtained based on the first collision time and the second collision time; The intermediate collision time is corrected based on the multimodal fusion target confidence, road surface adhesion coefficient, and scene correction coefficient to obtain the first target collision time.
10. The automatic emergency braking control method according to claim 6, characterized in that, The acoustic sensing data further includes the acoustic motion direction of the first collision target, and the wireless sensing data further includes the wireless motion direction corresponding to the second collision target. Determining whether the collision target is valid based on the multimodal fused target sensing information, the acoustic sensing data, and the wireless sensing data includes: Determine whether the confidence level of the multimodal fused target in the multimodal fused target perception information is greater than a preset confidence threshold, and determine whether the acoustic motion direction is consistent with the wireless motion direction; If the confidence level of the multimodal fusion target is greater than the preset confidence threshold, and the acoustic motion direction is consistent with the wireless motion direction, then the collision target is determined to be valid. If the confidence level of the multimodal fusion target is less than or equal to the preset confidence threshold, or if the acoustic motion direction is inconsistent with the wireless motion direction, then the collision target is determined to be invalid.
11. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, After generating the acoustic sensing data and the wireless sensing data, the method further includes: The acoustic sensing data and the wireless sensing data are time-aligned respectively to obtain time-aligned acoustic sensing data and time-aligned wireless sensing data. Based on the vehicle's onboard coordinate system, spatial alignment processing is performed on the time-aligned acoustic sensing data and the time-aligned wireless sensing data to obtain spatiotemporally aligned acoustic sensing data and spatiotemporally aligned wireless sensing data.
12. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, The acoustic signal information includes time-domain waveform information, frequency-domain frequency information, and acoustic signal quality characteristic information. The step of generating acoustic perception data corresponding to the first collision target within the non-line-of-sight occlusion blind zone based on the acoustic signal information includes: Based on the frequency domain information, extract the occlusion penetration acoustic signal corresponding to the first collision target; Based on the blocked and penetrated acoustic signal, calculate the propagation delay difference corresponding to the first collision target; Based on the propagation delay difference, the acoustic lateral distance corresponding to the first collision target is obtained; Based on the time-domain waveform information of the blocked and penetrated acoustic signal, the acoustic lateral motion velocity corresponding to the first collision target is estimated. Based on the acoustic signal quality characteristic information, the initial acoustic confidence level corresponding to the first collision target is determined.
13. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, The channel state information includes signal amplitude fluctuations, signal phase changes, and multipath interference information. The step of generating wireless sensing data corresponding to the second collision target within the non-line-of-sight blind zone based on the channel state information includes: Based on the signal amplitude fluctuation and the signal phase change, a motion index corresponding to the second collision target is constructed; Based on the motion index, the wireless lateral distance and wireless lateral motion speed corresponding to the second collision target are obtained; Based on the signal phase change and the multipath interference information, the initial wireless confidence level corresponding to the second collision target is determined.
14. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, Also includes: In response to the fact that the first collision target and the second collision target are not the same collision target, the collision time of the second target is determined based on the acoustic sensing data and the wireless sensing data; The automatic emergency braking control command is output based on the collision time with the second target.
15. The automatic emergency braking control method according to claim 14, characterized in that, Determining the collision time of the second target based on the acoustic sensing data and the wireless sensing data includes: Based on the acoustic lateral distance and acoustic lateral motion velocity corresponding to the first collision target in the acoustic perception data, the third collision time of the first collision target reaching the second collision point is calculated, where the second collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted motion trajectory of the first collision target. Based on the wireless lateral distance and wireless lateral movement speed corresponding to the second collision target in the wireless sensing data, the fourth collision time of the second collision target reaching the third collision point is calculated. The third collision point is the spatial intersection point between the predicted driving trajectory of the vehicle and the predicted movement trajectory of the second collision target. The second target collision time is obtained by taking the smaller of the third collision time and the fourth collision time.
16. The automatic emergency braking control method according to any one of claims 1 to 5, characterized in that, The process of executing the automatic emergency braking control command also includes: The attitude information of the vehicle is obtained, including yaw rate and sideslip angle; Determine whether the yaw rate is less than or equal to a yaw rate threshold, and determine whether the sideslip angle is less than or equal to a sideslip angle threshold; If the yaw rate is less than or equal to the yaw rate threshold and the sideslip angle is less than or equal to the sideslip angle threshold, then the automatic emergency braking control command is continuously executed. If the yaw rate is greater than the yaw rate threshold, or the sideslip angle is greater than the sideslip angle threshold, then the stability protection mechanism is activated.
17. An automatic emergency braking control device, characterized in that, include: The acquisition module is used to acquire multimodal environmental information within the non-line-of-sight blind spot corresponding to the vehicle, wherein the multimodal environmental information includes acoustic signal information and channel state information; The generation module is used to generate acoustic sensing data corresponding to the first collision target in the non-line-of-sight occlusion blind zone based on the acoustic signal information, and to generate wireless sensing data corresponding to the second collision target in the non-line-of-sight occlusion blind zone based on the channel state information. The fusion processing module is used to perform fusion processing on the acoustic sensing data and the wireless sensing data in response to the first collision target and the second collision target being the same collision target, to generate multimodal fusion target sensing information; The control module is used to output an automatic emergency braking control command based on the multimodal fusion target perception information, the acoustic perception data, and the wireless perception data, and to execute the automatic emergency braking control command.
18. A vehicle, characterized in that, include: The vehicle body and the automatic emergency braking control system, wherein the automatic emergency braking control system is used to perform the automatic emergency braking control method as described in any one of claims 1 to 16.
19. The vehicle according to claim 18, characterized in that, The automatic emergency braking control system includes an acoustic sensing unit and a wireless sensing unit. When both the acoustic sensing unit and the wireless sensing unit are functioning normally, the automatic emergency braking control system activates the dual-modal fusion mode. When the acoustic sensing unit malfunctions, the automatic emergency braking control system activates the wireless sensing mode. When the wireless sensing unit malfunctions, the automatic emergency braking control system activates the acoustic sensing mode. When both the acoustic sensing unit and the wireless sensing unit fail, the automatic emergency braking control system disables the non-line-of-sight sensing function.
20. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the automatic emergency braking control method as described in any one of claims 1 to 16.
21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the automatic emergency braking control method as described in any one of claims 1 to 16.
22. A computer program product, characterized in that, include: A computer program that, when executed by a processor, implements the automatic emergency braking control method according to any one of claims 1 to 16.
