Vehicle door locking mechanism, multi-modal fusion vehicle door opening protection system and method

Abstract

This invention belongs to the field of automotive active safety technology. Addressing the problems of existing technologies where a single locking principle leads to difficulties in balancing response speed and locking reliability, and insufficient fault tolerance in simplified structures, this invention provides a vehicle door locking mechanism, a multimodal fusion-based door opening and passenger / vehicle safety protection system, and a door opening and passenger / vehicle safety protection method. The system includes a cross-principle heterogeneous dual locking mechanism and a collaborative timing control module. The cross-principle heterogeneous dual locking mechanism includes a magnetic locking device (5) and a door locking servo motor (12). During locking, the mechanical servo motor is first driven to its position, and then the magnetic attraction is triggered after a delay. During unlocking, the power is first cut off, and then the mechanical servo motor is reset. The system combines multimodal perception with a Raspberry Pi (1), a monocular camera (2), and an ultrasonic sensor (3), as well as a risk assessment model based on a weighted coefficient k. This design achieves a unified approach of rapid response and robust locking with high fault tolerance in a simplified structure requiring only two core components, effectively preventing "door-opening kill" accidents.

Description

Technical Field

[0001] This invention relates to the field of active safety technology for automobiles, and in particular to vehicle and occupant safety protection technology when opening vehicle doors. Background Technology

[0002] "Door-opening accidents" are a frequent safety hazard in urban road traffic. When a vehicle is parked on a non-motorized vehicle lane or sidewalk, if occupants suddenly open the door without fully observing the traffic situation behind them, a violent collision with oncoming traffic at high speed is highly likely, resulting in injuries, fatalities, and vehicle damage. Statistics show that such accidents account for over 15% of all urban traffic accidents, and occur mostly during morning and evening rush hours. They are characterized by their suddenness and short avoidance time, seriously threatening the lives and property of road users. Therefore, developing effective door-opening safety protection systems has significant industrial and social value for ensuring road traffic safety.

[0003] However, existing technical solutions have many shortcomings, mainly including the following aspects.

[0004] Existing technologies generally employ locking devices based on a single principle. For example, the closest prior art (reference document 1, CN116872886B) discloses a locking device to prevent door-opening-and-kill, in which the door locking device 200 uses a single mechanical locking method where a drive motor 25 drives a worm gear 14 and a semi-worm gear structure 23 to drive a second latch 24. The response speed of this purely mechanical structure is limited by the transmission mechanism, posing a risk of failure when faced with sudden, forceful door opening. Although some solutions use electromagnets for locking, a single electromagnet fails when power is cut off, and the holding force depends on continuous power supply, resulting in insufficient reliability. Existing technologies have long been influenced by the mindset of "redundancy of similar components," habitually employing redundant devices based on the same principle (such as dual electromagnets or dual mechanical locks in parallel) to improve reliability. Those skilled in the art generally believe that cross-principle heterogeneous combinations (such as the combination of magnetic and mechanical forces) inevitably lead to structural complexity, increased cost, and control difficulties, making them unsuitable as a conventional technological choice. This results in existing solutions failing to achieve a balance between response speed and locking stability, and making true heterogeneous fault tolerance difficult to achieve.

[0005] Existing technologies often use fixed thresholds for risk assessment without differentiating the hazard level of the target. For example, Reference 1 uses only a binary judgment based on either "the speed of the moving object reaches a set threshold" or "the driving area is within a danger zone"; Reference 2 uses a fixed speed threshold and a distance threshold for assessment. These solutions fail to consider the differences in motion characteristics (speed, braking distance, collision hazard) of different traffic participants (pedestrians, bicycles, electric vehicles, and other cars), resulting in insufficient protection for high-risk targets and a high misjudgment rate for low-risk targets. This "one-size-fits-all" judgment standard cannot meet the needs of precise protection in complex traffic environments.

[0006] Existing technologies often lack convenient voice control functionality, requiring users to manually operate the remote control or door opener, which is inconvenient in situations where the user is holding items. While some systems are equipped with remote control or voice functions, they fail to dynamically link interactive commands to real-time environmental risks, resulting in unclear infrared remote control safety interaction procedures. For example, the one-button door unlocking module in Comparative Document 2 only forcibly unlocks the door in emergencies; the regular door opening process lacks safety checks for real-time risks, posing a security hazard.

[0007] Existing technologies mostly employ single-sensor solutions, which are limited in their sensing methods and are either costly or unreliable. Some mid-to-low-end models are equipped with ultrasonic parking assist systems that are only activated in reverse gear, making it impossible to continuously monitor the area behind the door (13) while the car is in a static parking state. Furthermore, single sensors are prone to false alarms. High-end models use millimeter-wave or lidar solutions, which are expensive (over 500 yuan per set) and difficult to popularize in ordinary models. Comparative document 1 uses a single camera video analysis, which is susceptible to light interference and lacks accurate distance perception. The monitoring angle is fixed, making it difficult to adapt to the monitoring needs of different models and complex road conditions. The communication protocol design between heterogeneous processors is incomplete, resulting in insufficient data transmission reliability. The system's timed self-check content is unclear, and its fault identification and prompting capabilities are insufficient. The system's power supply method and installation details are missing, affecting the feasibility of practical applications. The types of targets that can be identified are limited, making it difficult to cover all moving targets that may pose a collision risk.

[0008] In addressing the reliability issues of vehicle door locks, the art has long adhered to the design philosophy of "redundancy of the same principle," which involves using redundant devices based on the same principle (such as dual electromagnets or dual mechanical locks) to improve reliability. This mindset stems from two main factors: firstly, the inertia of early mechanical locking technology development, with those skilled in the art generally believing that cross-principle heterogeneous combinations (such as magnetic + mechanical) lead to structural complexity, increased costs, and control difficulties, making them unsuitable as a conventional technology choice; and secondly, because "redundancy of the same principle" is easier to design and control in terms of cost compared to cross-principle heterogeneous dual locking.

[0009] Meanwhile, in terms of risk assessment, early traffic management systems mostly used a unified standard to process all target types. This simplified model was directly transplanted to the field of door protection, forming a path dependency of fixed threshold judgment.

[0010] Existing technologies focus on improving the reliability of a single locking principle or optimizing the performance of a single sensor. In contrast, this patent application establishes a cross-principle heterogeneous dual locking technology and achieves a balance between response speed and locking reliability by innovatively adapting the physical characteristics of "non-contact fast response and contact firm locking" through timing control.

[0011] Given the aforementioned shortcomings of existing technologies, there is an urgent need for a vehicle door collision avoidance warning system that integrates low-cost perception, multimodal fusion judgment, target weight classification, active physical locking, flexible angle adjustment, reliable communication, clear interaction process, and voice control. This system should be able to comprehensively identify core risk targets (including pedestrians, bicycles, electric vehicles, other cars, and other moving targets that may pose a collision risk), set differentiated safety distances and weight coefficients based on the characteristics of different targets, adopt an innovative cross-principle heterogeneous dual locking design to improve safety and fault tolerance, support voice control of rearview mirrors and door operation, fundamentally solve the "door opening kill" problem, and balance safety and convenience. Summary of the Invention

[0012] The purpose of this invention is to provide a vehicle door locking mechanism that overcomes the shortcomings of existing technologies, such as the difficulty in balancing response speed and locking reliability due to the single locking principle, and the inability to achieve high fault-tolerant locking under simplified structural conditions.

[0013] To achieve the above objectives, the vehicle door locking mechanism of the present invention includes a cross-principle heterogeneous dual locking mechanism composed of a non-contact magnetic locking device and a contact locking door rudder mechanism, and a timing control module for controlling the dual locking mechanism to operate in a preset coordinated sequence. The magnetic locking device includes an electromagnet and a metal armature that cooperates with it; the door locking servo is installed at the door pivot connection and is in transmission cooperation with the door lock body; The timing control module is configured to: first drive the locking door servo to the locked position when locking, and then control the electromagnet to be energized to attract the armature after a delay; and first cut off the power supply to the electromagnet when unlocking, and then control the locking door servo to reset after a delay.

[0014] The electromagnet is fixed to the reinforcing rib of the inner panel of the door, and the metal armature is fixed to the sheet metal of the B-pillar of the vehicle body.

[0015] The delay is 50 milliseconds for locking and 30 milliseconds for unlocking.

[0016] The door locking servo motor has a response time of 100 milliseconds.

[0017] The present invention also discloses a vehicle door opening and human safety protection system based on multimodal fusion, which adopts the vehicle door locking mechanism and includes a sensing module, a control module, an execution module and an interaction module; The execution module includes the vehicle door locking mechanism, dual-color LED warning lights, camera steering adjustment servo, rearview mirror adjustment mechanism, and voice module; The perception module includes a Raspberry Pi, a monocular camera, and an ultrasonic sensor, and is configured to collect environmental information from the side and rear of the vehicle. The Raspberry Pi runs the YOLOv8 target detection algorithm to achieve visual recognition, and the ultrasonic sensor measures the distance to obstacles in real time. The control module is configured to assess the collision risk based on the environmental information and control the locking mechanism, voice module, and dual-color LED warning light; the interaction module is configured to receive user commands and trigger safety verification.

[0018] The control module includes a risk assessment unit, which is configured to calculate a risk assessment value R = (k × ΔP) / (T × D) based on the target hazard weight coefficient, and determine the collision risk level based on the assessment value; where k is the target hazard weight coefficient, ΔP is the change in the target pixel row coordinate, T is the detection time interval, and D is the obstacle distance.

[0019] The interaction module includes an infrared remote control receiver, which is configured to forcibly trigger security checks on high-risk commands and execute them only after passing the checks. The high-risk commands include voice commands for opening and closing the car door and / or commands for pressing and holding the infrared remote control door opening button for more than 1 second via the infrared remote control receiver. The security checks include triggering a single visual inference and multiple ultrasonic ranging measurements, and completing risk verification within a set time limit.

[0020] The perception module includes a heterogeneous processor communication architecture. The Raspberry Pi is connected to a monocular camera via a MIPI CSI interface and to an STM32 microcontroller main control unit via a UART protocol (9600bps, 8N1). The target recognition results are transmitted using a 7-byte fixed-length frame format, and a communication fault judgment threshold is set.

[0021] This invention also discloses a method for protecting the safety of people and vehicles when opening car doors based on multimodal fusion. The system described herein is implemented according to the following steps: S1. Collecting environmental information of the side and rear of the vehicle; S2. Assessing collision risk; S3. Based on the risk assessment result, controlling the action of a cross-principle heterogeneous dual locking mechanism. The dual locking mechanism includes a non-contact magnetic locking device and a contact locking door servo motor. The control is executed according to a preset coordinated timing sequence: when locking, the locking door servo motor is first driven to the locking position, and after a delay, the electromagnet is energized to attract the door; when unlocking, the power supply to the electromagnet is first cut off, and after a delay, the locking door servo motor is reset.

[0022] The sensing module includes a Raspberry Pi, a monocular camera, and an ultrasonic sensor. The monocular camera is connected to the camera steering adjustment servo via a monocular camera fixing device. The control module includes an STM32 microcontroller main control unit. The interaction module includes a voice acquisition unit, a voice recognition unit, and an infrared remote control receiver. The Raspberry Pi runs the YOLOv8 target detection algorithm to achieve visual recognition; the ultrasonic sensor measures the distance to obstacles in real time; the system achieves communication between heterogeneous processors through the UART protocol (9600bps, 8N1); and the BOM cost is controlled within 300 yuan.

[0023] This invention integrates a cross-principle heterogeneous dual locking mechanism into a complete door opening safety protection system. Through the synergy of multimodal perception, risk assessment, and interactive verification, it achieves closed-loop safety control from environmental perception to physical protection.

[0024] Response time refers to the time from receiving a control signal to rotating to the locked position. A 100-millisecond response time defines the dynamic response characteristics of the contact locking mechanism, forming a tiered combination with the millisecond-level response of the magnetic locking device. Together, they constitute the dual protection timing basis of "rapid initial constraint + firm and continuous locking," meeting the requirements for rapid response. The 100-millisecond response time is an optimized parameter determined based on the performance of mainstream automotive servos and the door opening speed, ensuring that mechanical locking preparation is completed at the initial stage of door opening and before any danger occurs.

[0025] Safety checks include triggering a single YOLOv8 inference and five ultrasonic ranging measurements, completed within 200 milliseconds. The verification is based on a differentiated safety distance threshold determined by the target hazard weight coefficient. The threshold is calculated as "target original risk distance threshold × (k / 1.5)", where k is the weight coefficient of the corresponding target. If the verification fails, execution is rejected and feedback is provided through the Su-03T voice module. Low-risk commands (such as rearview mirror adjustment) are executed directly.

[0026] The infrared remote control receiver uses the NEC protocol. The 200-millisecond timeout is specifically designed for "door-opening kill" emergencies; the five ultrasonic ranging measurements are used to improve the reliability of distance measurement; the original risk distance thresholds in the threshold calculation formula are: pedestrian 1.5 meters, bicycle 1.6 meters, electric vehicle 1.7 meters, and other vehicles 2.0 meters.

[0027] The vehicle door opening safety protection method also includes a step to forcibly trigger safety checks for high-risk commands, which is applicable to various vehicle types such as sedans, SUVs, and MPVs.

[0028] The present invention has the following advantages: The cross-principle heterogeneous combination in this invention breaks through the "redundancy of the same kind" mindset that has long bound those skilled in the art. By complementing the physical characteristics of non-contact rapid response and contact-type firm locking, it achieves the response speed and locking reliability that a single locking principle cannot achieve under the condition of structural simplification (only two core components need to be fixed). Moreover, when one mechanism fails, the other mechanism can still play a protective role, which significantly improves the fault tolerance of the system.

[0029] This invention clarifies the specific arrangement of each component in the cross-principle heterogeneous dual locking mechanism, ensuring the spatial coordination between non-contact magnetic attraction and contact mechanical locking, enabling the two locking mechanisms based on different principles to effectively cooperate at specific relative positions between the vehicle door and the vehicle body. The installation method of this invention only requires fixing two core components.

[0030] Specific delay parameters are specially optimized for the door opening power characteristics (sudden opening force, magnetic attraction response delay, and mechanical servo action time), effectively avoiding mechanical jamming and electrical conflicts, and ensuring smooth and reliable locking and unlocking actions.

[0031] The risk assessment algorithm couples target type, dynamic trend, and real-time distance into three dimensions, breaking through the crude mode of fixed threshold judgment in existing technologies. By quantifying the differences in danger among different traffic participants through weights, it achieves precise risk classification, providing accurate triggering basis for cross-principle heterogeneous dual locking mechanisms and avoiding false locking and missed locking. This invention achieves differentiated protection for targets with different levels of danger through weighted quantitative assessment, improving the accuracy and adaptability of safety protection.

[0032] This invention dynamically binds user interaction commands with real-time environmental risks, and ensures the safety of high-risk operations through a mandatory verification mechanism, preventing accidental operation from breaching the protection of the main locking mechanism. This achieves a balance between convenience and security, and the mandatory coupling mechanism ensures that the convenience of interaction does not come at the expense of security.

[0033] The risk assessment algorithm conforms to the requirements of GB / T44173-2024 standard, and its k-value is set based on the motion characteristics (speed, braking distance, collision hazard) of different targets; ΔP is obtained through continuous frame detection using YOLOv8. The risk assessment unit can be expanded to adapt to other moving target types through algorithm upgrades.

[0034] This invention ensures stable and reliable data transmission between the visual recognition module and the real-time control module through a standardized heterogeneous processor communication protocol, providing a reliable data foundation for risk assessment and lockout control.

[0035] This invention defines a complete security protection process from environmental perception and risk assessment to cross-principle heterogeneous locking execution through method steps, overcoming the inherent limitations of the single locking principle in the prior art and achieving a balance between response speed and locking reliability. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall structure of the vehicle door opening safety protection system in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system hardware circuit and communication connection in an embodiment of the present invention; Figure 3This is a schematic diagram of the overall system workflow in an embodiment of the present invention; Figure 4 This is a schematic diagram of the monocular camera angle adjustment process in an embodiment of the present invention; Figure 5 This is a schematic diagram of the infrared remote control security interaction process in an embodiment of the present invention; Figure 6 This is a side view of the vehicle and a schematic diagram of the device arrangement in an embodiment of the present invention; Figure 7 This is a partial structural diagram of the door locking mechanism in an embodiment of the present invention. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0038] like Figures 1 to 7 As shown, this embodiment discloses a vehicle door locking mechanism, including a cross-principle heterogeneous dual locking mechanism composed of a non-contact magnetic locking device 5 and a contact locking door servo motor 12, and a timing control module for controlling the dual locking mechanism to operate in a preset coordinated sequence. The magnetic locking device 5 includes an electromagnet 14 and a metal armature 9 that cooperates with it; the door locking servo motor 12 is installed at the door (13) pivot connection and cooperates with the door lock body in transmission. The timing control module is configured to: first drive the locking door servo motor 12 to the locked position when locking, and then control the electromagnet 14 to be energized to attract the metal armature 9 after a delay; and first cut off the power supply to the electromagnet 14 when unlocking, and then control the locking door servo motor 12 to reset after a delay.

[0039] The non-contact magnetic locking device 5 utilizes the principle of electromagnetic adsorption to achieve rapid response, while the contact locking door servo motor 12 utilizes mechanical rigidity to achieve secure locking. The delay time can be set according to the opening dynamic characteristics of the door 13 and the response delay of the electromagnet 14. The locking delay is preferably 50 milliseconds, and the unlocking delay is preferably 30 milliseconds. The response time of the locking door servo motor 12 is preferably 100 milliseconds. The gap between the electromagnet 14 and the metal armature 9 is preferably 1-2 millimeters. The output torque of the locking door servo motor 12 is preferably ≥5 N·m.

[0040] When locking, the door locking servo motor 12 is activated first. After rotating to the locked position (response time approximately 100 milliseconds) and delaying for 50 milliseconds to ensure mechanical stability, the electromagnet 14 is then energized to attract the door, preventing mechanical impact and residual magnetic resistance from causing jamming. When unlocking, the power supply to the electromagnet 14 is first cut off. After a 30-millisecond delay to ensure the magnetic force is fully released, the door locking servo motor 12 is then activated to perform a reset action (response time approximately 100 milliseconds), preventing residual magnetic resistance from causing mechanical jamming.

[0041] This cross-principle heterogeneous combination breaks through the long-standing mindset of "redundancy of the same kind" that has bound those skilled in the art. By complementing the physical characteristics of non-contact rapid response and contact-type firm locking, it achieves the response speed and locking reliability that a single locking principle cannot achieve under the condition of structural simplification (only two core components need to be fixed). Moreover, when one mechanism fails, the other mechanism can still play a protective role, which significantly improves the fault tolerance of the system.

[0042] The timing control module can be implemented by a microcontroller such as an STM32 microcontroller; the door locking servo motor 12 can be controlled by a PWM signal; the electromagnet 14 can be controlled by a GPIO-driven power switch circuit; this locking mechanism is suitable for aftermarket modification of various vehicle models such as sedans, SUVs and MPVs.

[0043] This embodiment also discloses a vehicle door opening and human safety protection system based on multimodal fusion, including the vehicle door locking mechanism, a sensing module, a control module, an execution module and an interaction module. The interaction module includes a voice control module and an infrared remote control receiver 8.

[0044] The perception module includes a Raspberry Pi 1, a monocular camera 2, and an ultrasonic sensor 3. The Raspberry Pi 1 is connected to the monocular camera 2 via a MIPICSI interface. The perception module is used to accurately capture the distance information of targets that may pose a security threat. The Raspberry Pi 1 is responsible for YOLOv8 visual recognition and is therefore part of the perception module.

[0045] The control module includes an STM32 microcontroller main control unit 4, which is used to quickly analyze and judge security risks and the feasibility of voice command execution. The core integrated target weighting risk assessment logic; The execution module includes the vehicle door locking mechanism, dual-color LED warning light 7, camera steering adjustment servo 20, rearview mirror adjustment mechanism 21, and Su-03T voice module 6 for playing prompt sounds. The execution module is used to take protective measures and respond to control commands in a timely manner. The voice control module includes a voice acquisition unit 17 and a voice recognition unit 11, which are used to receive user voice commands and perform recognition and analysis.

[0046] The camera steering adjustment servo 20 is an execution module that receives PWM signals from the control module and adjusts the monitoring angle of the monocular camera 2 through the monocular camera fixing device 15.

[0047] This patent protects pedestrians, bicycles, electric vehicles, other cars, and other moving targets that may pose a collision risk outside the vehicle. The types of targets that can be identified can be expanded through algorithm upgrades.

[0048] like Figure 1 , Figure 4 , Figure 5As shown, the monocular camera 2 is connected to the camera steering adjustment servo 20 via the monocular camera fixing device 15. Initially, the optical axis is 15° downwards, allowing for precise capture of various targets that may pose a safety threat. The camera steering adjustment servo 20 uses a mechanical limiting structure to ensure the monocular camera 2 can achieve ±30° pitch adjustment and ±45° horizontal rotation adjustment, dynamically covering the blind spots to the side and rear of the vehicle. Users can fine-tune the monitoring angle via remote control or voice commands. The system limits the adjustment range through software to prevent visual field failure. The camera steering adjustment servo 20 receives PWM signals from the control module to drive and adjust the monitoring angle of the monocular camera 2. The ultrasonic sensor 3 and the dual-color LED warning light 7 are integrated into a single bracket and mounted on the outer side of the vehicle's headlight 19. The dual-color LED warning light 7 is red and green, and is vertically located directly below the ultrasonic sensor 3. The ultrasonic sensor 3 is positioned above it to measure the distance to obstacles in real time, and the dual-color LED warning light 7 is installed close to it below. Green indicates a safe state, and red indicates a dangerous state, which is used to visually warn of the safe state.

[0049] The door locking servo motor 12 is installed at the pivot connection of the door 13 and works in conjunction with the door lock body to control the locking and unlocking of the door 13. The door locking servo motor 12 is a standard rotary drive device including a servo motor, a reduction transmission mechanism, and a position feedback unit. It connects to the original door lock body through transmission components, requiring no modification to the original vehicle body structure. The door locking servo motor 12 is conventional technology and will not be described in detail. The rearview mirror adjustment mechanism 21 is integrated inside the rearview mirror 16 and is connected to the rearview mirror lens. It receives PWM control signals to adjust the up, down, left, and right angles of the lens.

[0050] The voice acquisition unit 17 uses a high-fidelity microphone, which is fixed to the dashboard area on the left side of the steering wheel inside the vehicle using 3M adhesive to ensure clear acquisition of user voice commands. Based on the above system structure, the first step of the vehicle door opening safety protection method of this embodiment is: S1. Acquire information about the side and rear environment of the vehicle. The Raspberry Pi 1 runs the YOLOv8 target detection algorithm in combination with the image acquisition of the monocular camera 2 to achieve visual recognition, and the ultrasonic sensor 3 measures the distance of obstacles in real time to form multimodal perception data.

[0051] like Figure 2 , Figure 6As shown, the control module includes a main control box 10, which is a metal shell structure and magnetically fixed to the corresponding position inside the car door 13. The Raspberry Pi 1 of the sensing module, the STM32 microcontroller main control unit 4 of the control module, and the voice recognition unit 11 are integrated into the main control box 10 and connected to the sensors and actuators through wires. The STM32 microcontroller main control unit 4 implements a risk assessment unit through built-in software program, which is used to calculate the risk assessment value R=(k×ΔP) / (T×D) based on the target hazard weight coefficient and determine the collision risk level, where k is the target hazard weight coefficient, ΔP is the change in the target pixel row coordinate (number of pixels), T is the detection time interval, and D is the obstacle distance.

[0052] Raspberry Pi 1 connects to monocular camera 2 via MIPI CSI interface and connects to STM32 microcontroller main control unit 4 via UART protocol (9600bps, 8N1), forming a heterogeneous processor communication architecture. This enables standardized data transmission between Raspberry Pi 1 (responsible for AI visual recognition) and STM32 microcontroller main control unit 4 (responsible for real-time control), ensuring reliable communication and stable data transmission between heterogeneous processors.

[0053] The Raspberry Pi 1 uses a 7-byte fixed-length frame format [0x5A][category code][u_L][u_H][v_L][v_H][0xA5] to transmit target identification results. Frame verification is performed through the frame header [0x5A] and frame tail [0xA5], and a communication failure judgment threshold is set (three consecutive frame verification failures or no data reception for 500ms).

[0054] Wherein: category code 0x01 represents pedestrians outside vehicles, 0x02 represents bicycles, 0x03 represents electric vehicles, and 0x04 represents other vehicles. The category codes corresponding to other potentially threatening moving targets can be extended and defined through algorithm upgrades; [u_L] is the lower 8 bits of the horizontal coordinate (u coordinate) of the target image's center pixel, and [u_H] is the higher 8 bits of the horizontal coordinate (u coordinate) of the target image's center pixel. The two are combined to form a 16-bit unsigned integer, representing the target's horizontal pixel position in the image coordinate system; [v_L] is the lower 8 bits of the vertical coordinate (v coordinate) of the target image's center pixel, and [v_H] is the higher 8 bits of the vertical coordinate (v coordinate) of the target image's center pixel. The two are combined to form a 16-bit unsigned integer, representing the target's vertical pixel position in the image coordinate system. If the STM32 microcontroller main control unit 4 fails to verify for 3 consecutive frames or receives no data for 500ms, it is determined to be a communication fault, and the dual-color LED warning light 7 is controlled to flash at a frequency of 2Hz to sound an alarm. The voice acquisition unit 17 is connected to the voice recognition unit 11 via an audio cable. The voice recognition unit 11 communicates with the STM32 microcontroller main control unit 4 via a UART serial port, transmitting the recognized command signal to the main control unit 4. The 500-millisecond threshold is set taking into account the single-frame inference latency of the YOLOv8 algorithm (typical frame interval 100 milliseconds), and a fault determination is triggered only when there is no valid data for a continuous period of time.

[0055] The STM32 microcontroller main control unit 4 controls the door locking servo 12, camera steering adjustment servo 20, magnetic locking device 5, dual-color LED warning light 7, ultrasonic sensor 3, Su-03T voice module 6, and rearview mirror adjustment mechanism 21 through GPIO pins, ensuring that each execution component responds quickly to commands.

[0056] The STM32 microcontroller main control unit 4 has a timed self-test function, automatically checking the working status of each component every 30 seconds. The checks include: ① whether the echo signal of the ultrasonic sensor 3 is normal; ② whether the feedback voltage of the door locking servo 12, the camera steering adjustment servo 20, and the rearview mirror adjustment mechanism 21 is within the preset range; ③ whether the UART heartbeat packet sent by the Raspberry Pi 1 is received normally; ④ whether the signal transmission between the voice acquisition unit 17 and the voice recognition unit 11, and between the voice recognition unit 11 and the STM32 microcontroller main control unit 4 is normal. In case of a fault, the dual-color LED warning light 7 will issue an alarm. A 2Hz flashing dual-color LED warning light 7 indicates a fault in the visual recognition module, ultrasonic sensor, or voice control module, while a 5Hz flashing indicates a fault in the locking mechanism or rearview mirror adjustment mechanism, prompting the user to have it repaired promptly.

[0057] The entire system adopts a magnetic or snap-on non-destructive installation method, which does not require large-scale modification of the original vehicle body structure 18, and is suitable for various passenger vehicle models such as sedans, SUVs, and MPVs. The system draws 12V power from the vehicle's ACC interface, which is processed by a voltage regulator chip before power supply, with a standby power consumption of <6W. The monocular camera 2 is fixed to the bottom of the rearview mirror 16 with 3M adhesive, the main control box 10 is magnetically attached to the inside of the door 13, the ultrasonic sensor 3 is snapped to the side of the headlight 19, and the voice acquisition unit 17 is fixed to the dashboard area on the left side of the steering wheel with 3M adhesive, without damaging the vehicle body structure 18. The dual locking mechanism only requires fixing two core components: the electromagnet 14 and the door locking servo 12, making installation simple.

[0058] The second step of the vehicle door opening safety protection method is: S2. Assess the collision risk. This is performed by the risk assessment unit implemented by the STM32 microcontroller main control unit 4 through the built-in software program. Based on the target hazard weight coefficient k, the risk assessment value R = (k × ΔP) / (T × D) is calculated (where k is the target hazard weight coefficient, ΔP is the change in the target pixel row coordinate, T = 0.3 seconds, and D is the distance to the obstacle). The system determines whether to trigger the locking based on the condition that R ≥ 1.0 and the distance is within the original risk distance threshold range.

[0059] like Figure 1 , Figure 3 , Figure 7 As shown, when a collision risk is determined, step S3 (third step) is executed. The electromagnet 14 of the magnetic locking device 5 is bolted to the reinforcing rib of the inner panel of the door 13, and the metal armature 9 is welded to the sheet metal of the B-pillar of the vehicle body 18. The relative positions of the electromagnet 14 and the metal armature 9 should ensure that the door closing gap is within the range of 1-2 mm to guarantee the magnetic attraction effect. When energized, the electromagnet 14 generates a force greater than 10N, attracting the metal armature 9 to achieve reliable auxiliary locking.

[0060] The timing control module executes the following coordinated timing control through the STM32 microcontroller main control unit 4: S3. Control the action of the cross-principle heterogeneous dual locking mechanism according to the risk judgment result. The dual locking mechanism includes a non-contact magnetic locking device 5 and a contact locking door servo motor 12. The control is executed according to the preset coordinated timing sequence: When locking (when the STM32 microcontroller main control unit 4 determines that there is a collision risk, the locking door servo motor 12 is first driven to the locking position (response time is about 100ms, contact locking). After a 50ms delay to ensure mechanical stability, a high level is output to control the electromagnet 14 to be energized and attract the metal armature 9, so as to realize the cross-principle heterogeneous dual physical locking of the door 13; When unlocking, the power supply of the electromagnet 14 is first cut off. After a 30ms delay to ensure that the magnetic force is completely released, the locking door servo motor 12 is controlled to perform a reset action (response time is about 100ms) to avoid mechanical jamming caused by residual magnetic resistance.

[0061] The delay time can be achieved through the software timer of the STM32 microcontroller main control unit 4, with an accuracy down to the millisecond level. The 50-millisecond delay is the additional waiting time after the door locking servo motor 12 has fully stabilized after mechanically rotating to the locked position (response time approximately 100 milliseconds), and is set at 50% of the response time as a safety margin; the 30-millisecond delay is the measured waiting time after the electromagnet 14 is de-energized and the magnetic force decays to below the safety threshold. After this waiting period, the door locking servo motor 12 performs a reset action (response time approximately 100 milliseconds).

[0062] The inner panel reinforcing rib is a structural reinforcement part of the inner panel of the door 13; the B-pillar sheet metal is the metal sheet metal structure of the B-pillar of the vehicle body 18.

[0063] This dual-locking design abandons the "redundancy" approach commonly used by those skilled in the art. Instead, it adopts a heterogeneous combination of magnetic attraction (non-contact, fast response) and mechanical locking door servo (contact, secure locking). This combination leverages the 5-millisecond response speed of the magnetic locking device to quickly and initially restrain the door's opening tendency, while the mechanical rigidity of the locking door servo ensures that the door cannot be forcibly opened. The two work together to form a dual safety guarantee.

[0064] In this invention, the original risk distance threshold refers to the safety distance benchmark parameter preset by the GB / T 44173-2024 standard based on the motion characteristics (speed and braking distance) of the external target type (pedestrian, bicycle, electric vehicle, and other vehicles), and the unit is meters (m). This threshold is a design parameter of this invention and serves as an auxiliary constraint for risk assessment—when the obstacle distance measured by the ultrasonic sensor 3 is less than this threshold, regardless of the risk assessment value R, a basic collision risk is determined to exist; at the same time, this threshold serves as the benchmark for calculating the unlocking safety distance threshold (unlocking safety distance threshold = original risk distance threshold × (k / 1.5), where k is the target hazard weight coefficient).

[0065] The risk assessment unit calculates the risk assessment value R = (k × ΔP) / (T × D) based on the target hazard weight coefficient k, where: k is preset according to the target type as k = 1.5 for pedestrians, k = 1.8 for bicycles, k = 2.1 for electric vehicles, and k = 1.65 for other cars; ΔP is the change in the row coordinate of the target pixel, corresponding to the change in the v coordinate (vertical coordinate) of the target center in the image coordinate system, representing the target approach trend. When the same target is detected in 3 consecutive frames and ΔP > 5 pixels, it is determined that there is an approach trend; T = 0.3 seconds, matching the YOLOv8 detection interval of 100 milliseconds × 3 for 3 consecutive frames; D is the actual obstacle distance measured by the ultrasonic sensor 3.

[0066] When the same target is detected in 3 consecutive frames and ΔP > 5 pixels, R ≥ 1.0, and the target distance is within the original risk distance threshold range (pedestrian < 1.5 meters, bicycle < 1.6 meters, electric vehicle < 1.7 meters, other car < 2 meters), it is judged as a high-risk state and triggers the double locking mechanism.

[0067] If the distance exceeds the original threshold, it is not considered high-risk regardless of the R value, ensuring consistency with the GB / T 44173-2024 standard. To verify the rationality of the weighted risk assessment logic, the system implementation includes the following risk assessment cases: Case 1: An electric vehicle was detected (k=2.1), with a pixel change of ΔP=6 pixels within 3 frames and a distance of D=1.6m (<1.7m threshold). Substituting into the formula, we get R=(2.1×6) / (0.3×1.6)=26.25≥1.0, which is judged as high risk, triggering double locking and the voice message "Electric vehicle is approaching rapidly from behind, do not open the door!" Case 2: Pedestrian detected (k=1.5), ΔP=5 pixels, D=1.4m (<1.5m threshold), substituting into the formula, we get R≈17.86≥1.0, which is judged as high risk, triggering double locking and the voice message "Pedestrian behind, please be careful!"; Case 3: A bicycle was detected (k=1.8), ΔP=4 pixels (<5 pixels condition), D=1.5m. Since the condition ΔP>5 pixels is not met, it is determined to be non-high risk and the safe state is maintained. Case 4: Other cars were detected (k=1.65), ΔP=7 pixels, D=1.9m (<2m threshold). Substituting into the formula, we get R≈20.26≥1.0, which is considered high risk. This triggers double locking and the voice message "Vehicles are approaching from behind. Please wait until it is safe before opening the door!"

[0068] The Su-03T voice module 6 has 10 pre-stored voice prompts: "Pedestrian behind, please be careful!" "Please wait until it is safe before opening the door!" "System locked, cannot be opened!" "Vehicle approaching from behind, do not open the door!" "Rearview mirror being adjusted!" "Door about to open, please be careful of your surroundings!" "Currently unsafe, cannot execute the door opening command!" "Bicycle approaching from behind, do not open the door!" "Electric vehicle approaching rapidly from behind, do not open the door!" "Vehicle approaching from behind, wait until it is safe before opening the door!" These prompts are triggered by GPIO pulses, with the speaker opening facing both the outside and inside of the vehicle to ensure that occupants and other road users can clearly receive the warning information. The system automatically matches voice prompts based on the target type and risk assessment value R. Pedestrians (R≥15), bicycles (R≥18), electric vehicles (R≥20), and other vehicles (R≥18) trigger specific warning voices. For other moving targets that can be further recognized, the corresponding voice prompt content can be expanded through module upgrades.

[0069] The infrared remote control receiver 8 is an interactive module that uses the NEC infrared encoding protocol. The STM32 microcontroller main control unit 4 decodes the received signal through an external interrupt. When the user issues a high-risk command through the infrared remote control receiver 8 (requiring a long press of the door open button on the remote control for more than 1 second), the system immediately forces a single YOLOv8 inference and 5 ultrasonic rangings (completed within 200ms). Based on the identified target type and the corresponding weight coefficient, the system determines whether unlocking is allowed according to the safety distance threshold (calculated as "original risk distance threshold × (k / 1.5)").

[0070] The unlocking safety distance threshold is calculated as "target original risk distance threshold × (k / 1.5)" (where k is the target hazard weighting coefficient): pedestrian 1.5 × (1.5 / 1.5) = 1.5 meters, bicycle 1.6 × (1.8 / 1.5) = 1.92 meters, electric vehicle 1.7 × (2.1 / 1.5) = 2.38 meters, other cars 2.0 × (1.65 / 1.5) = 2.2 meters. The magnetic locking device 5 and the door locking servo 12 are only allowed to be unlocked and manually opened when the target distance is ≥1.5m for pedestrians, ≥1.92m for bicycles, ≥2.38m for electric vehicles, and ≥2.2m for other cars, and there are no other high-risk moving targets. If a risk is detected, a voice prompt will say "Currently unsafe". Users can send angle adjustment commands via custom buttons on the remote control to control the adjustment of the camera or rearview mirror 16; for other moving targets identified through algorithm upgrades, a corresponding safe distance threshold can be preset for unlocking judgment.

[0071] The interaction module includes a voice control module and an infrared remote control receiver 8. The voice control module includes a voice acquisition unit 17 and a voice recognition unit 11. The voice recognition unit 11 has a pre-stored voice command library and is divided into low-risk commands and high-risk commands according to risk level: low-risk commands include rearview mirror adjustment commands ("adjust rearview mirror up", "adjust rearview mirror down", "adjust rearview mirror left", "adjust rearview mirror right" and "reset rearview mirror"); high-risk commands include door control commands ("open door" and "close door").

[0072] After the voice acquisition unit 17 acquires the user's voice, it transmits it to the voice recognition unit 11 for noise reduction, feature extraction and matching recognition. The voice recognition unit 11 has an accuracy of no less than 92% in a mixed noise environment of 60dB in the vehicle and a recognition response time of <300ms.

[0073] When a user issues a low-risk command (such as a voice command to adjust the rearview mirror), the STM32 microcontroller main control unit 4 directly controls the rearview mirror adjustment mechanism 21 to perform the corresponding action without triggering a mandatory safety check. The rearview mirror adjustment mechanism 21 achieves precise adjustment of the rearview mirror 16 up, down, left, and right through PWM pulse signals, with an adjustment step of 0.5°. At the same time, the Su-03T voice module 6 plays the prompt "Rearview mirror adjustment in progress" and automatically stops after the adjustment is completed. When a user issues a high-risk command (such as the voice command "open the car door"), the interaction module forcibly triggers a safety check and can only execute the command after passing the check. This constitutes a hard logic coupling of "command triggering → forced multimodal detection → conditional execution": The system immediately triggers a single YOLOv8 inference and 5 ultrasonic rangings (completed within 200ms), and performs a check based on the differentiated safety distance threshold determined by the target hazard weight coefficient (calculated as "original risk distance threshold × (k / 1.5)": pedestrian ≥ 1.5m, bicycle ≥ 1.92m, electric vehicle ≥ 2.38m, other cars ≥ 2.2m). If the check passes, the unlocking action is executed and the prompt "The car door is about to open, please pay attention to the surrounding safety" is played. If the check fails, the prompt "Currently unsafe, unable to execute the door opening command" is played and the command is refused to be executed.

[0074] When a user issues a voice command to "close the car door," the system directly controls the locking mechanism to perform the door closing and locking action; the voice command library can be expanded through algorithm upgrades to adapt to new functions.

[0075] In a preferred embodiment, the voice acquisition unit 17 employs a dual-microphone array positioned near the steering wheel to acquire spatial domain speech and noise differences. Upon system startup, an initial silent period can optionally be used to establish a background noise model. After acquisition, the voice signal can be denoised sequentially using any one or a combination of the following methods: (a) beamforming based on the microphone array to enhance speech in the target direction; (b) noise suppression based on spectral subtraction to subtract estimated noise components from the speech spectrum; and (c) residual noise elimination based on adaptive filtering. These denoising methods can be used individually or in combination to improve the input signal-to-noise ratio of the voice recognition unit 11. Experiments show that after employing the above denoising strategies, the system achieves a speech recognition accuracy of over 92% in a 60dB mixed noise environment inside the vehicle, with a response time of less than 300ms. Steps S1 to S3 of the above method are applicable to various passenger vehicle models such as sedans, SUVs, and MPVs, achieving closed-loop safety protection from environmental perception and risk assessment to physical locking through a standardized process.

[0076] The above embodiments are only used to illustrate and not limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention without departing from the spirit and scope of the present invention. Any modifications or partial substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A vehicle door locking mechanism, characterized in that: It includes a cross-principle heterogeneous dual locking mechanism consisting of a non-contact magnetic locking device (5) and a contact locking door servo motor (12), and a timing control module that controls the dual locking mechanism to operate in a preset coordinated sequence. The magnetic locking device (5) includes an electromagnet (14) and a metal armature (9) that cooperates with it; the door locking servo motor (12) is installed at the pivot connection of the door (13) and cooperates with the door lock body in transmission. The timing control module is configured to: drive the locking door servo motor (12) to the locking position when locking, and control the electromagnet (14) to be energized and attract the armature (9) after a delay; and cut off the power supply of the electromagnet (14) when unlocking, and control the locking door servo motor (12) to be reset after a delay.

2. The vehicle door locking mechanism according to claim 1, characterized in that: The electromagnet (14) is fixed to the inner panel reinforcing rib of the door (13), and the metal armature (9) is fixed to the B-pillar sheet metal of the vehicle body (18).

3. The vehicle door locking mechanism according to claim 1, characterized in that: The delay is 50 milliseconds for locking and 30 milliseconds for unlocking.

4. The vehicle door locking mechanism according to claim 1, characterized in that: The response time of the door locking servo (12) is 100 milliseconds.

5. A vehicle door opening and pedestrian safety protection system based on multimodal fusion, employing the vehicle door locking mechanism as described in any one of claims 1-4, characterized in that: It includes a perception module, a control module, an execution module, and an interaction module; The execution module includes the vehicle door locking mechanism as described in any one of claims 1-4, a dual-color LED warning light (7), a camera steering adjustment servo (20), a rearview mirror adjustment mechanism (21), and a voice module (6); The perception module includes a Raspberry Pi (1), a monocular camera (2) and an ultrasonic sensor (3), and is configured to collect environmental information about the side and rear of the vehicle. The Raspberry Pi (1) runs a target detection algorithm to achieve visual recognition, and the ultrasonic sensor (3) measures the distance to obstacles in real time. The control module is configured to assess the collision risk based on the environmental information and control the locking mechanism, voice module (6) and dual-color LED warning light (7) to operate; the interaction module is configured to receive user commands and trigger safety verification.

6. The system according to claim 5, characterized in that: The control module includes a risk assessment unit, which is configured to calculate a risk assessment value R = (k × ΔP) / (T × D) based on the target hazard weight coefficient, and determine the collision risk level based on the assessment value; where k is the target hazard weight coefficient, ΔP is the change in the target pixel row coordinate, T is the detection time interval, and D is the obstacle distance.

7. The system according to claim 6, characterized in that: The interaction module includes an infrared remote control receiver (8), which is configured to forcibly trigger a safety check for high-risk commands and execute them only after passing the check; the high-risk commands include voice commands for opening and closing the car door and / or commands for pressing and holding the infrared remote control door opening button for more than 1 second via the infrared remote control receiver (8); the safety check includes triggering a single visual reasoning and multiple ultrasonic ranging measurements and completing the risk verification within a set time limit.

8. The system according to any one of claims 7, characterized in that: The perception module includes a heterogeneous processor communication architecture. The Raspberry Pi (1) is connected to a monocular camera (2) through the MIPI CSI interface and to the STM32 microcontroller main control unit (4) through the UART protocol. The target recognition result is transmitted in a 7-byte fixed-length frame format, and a communication fault judgment threshold is set.

9. A method for protecting the safety of people and vehicles when opening doors based on multimodal fusion, comprising the following steps using the system described in claim 8: S1. Collecting information about the side and rear environment of the vehicle; S2. Assess collision risks; S3. Based on the risk assessment results, control the action of the cross-principle heterogeneous dual locking mechanism. The dual locking mechanism includes a non-contact magnetic locking device (5) and a contact locking door servo motor (12). The control is executed according to a preset coordinated timing sequence: when locking, first drive the locking door servo motor (12) to the locking position, and after a delay, control the electromagnet (14) to be energized and attracted. When unlocking, first cut off the power supply of the electromagnet (14), and after a delay, control the locking door servo motor (12) to be reset.

Images