Vehicle door opening anti-collision control method and system and vehicle

Abstract

This application provides a vehicle door opening collision prevention control method, system, and vehicle. The method includes: acquiring an image of an occupant; if it is determined from the image that the occupant intends to open the door, acquiring a door opening field of view; acquiring motion state information of a moving target outside the vehicle within the door opening field of view; estimating the collision time between the moving target and the vehicle door based on the motion state information; and controlling the vehicle door lock to perform a door opening restriction action based on the collision time. This effectively prevents high-risk door opening behavior and avoids the panic and user aversion to safety features caused by traditional hard locking, thus improving the intelligence level and user acceptance of the door opening collision prevention solution.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a vehicle door opening collision avoidance control method, system, and vehicle. Background Technology

[0002] Traditional vehicle door opening warning (DOW) systems mainly rely on millimeter-wave radar or fixed-field surround-view cameras deployed on the side and rear of the vehicle. Their working logic is as follows: when the vehicle is stationary and the occupant pulls the door handle or unlocks the door, if a moving target is detected approaching from behind, intervention is carried out by sounding an alarm, flashing ambient lights, or mechanically locking the door.

[0003] However, the relevant technologies have revealed the following shortcomings in practical applications: 1. Delayed Triggering Mechanism and Insufficient Reaction Time: Existing DOW (Door-Off) systems are mostly activated only the instant an occupant pulls the door handle or the door lock is unlocked. From system perception and collision risk assessment to the actuator response, there are inherent signal transmission and calculation delays, resulting in extremely limited reaction time for occupants to avoid danger. When facing rapidly approaching non-motorized vehicles or pedestrians, this delay can easily lead to the failure of collision avoidance intervention. In other words, the passive response mode triggered by the door handle cannot advance the warning and intervention time before the occupant physically touches the door, failing to allow sufficient time for subsequent decision-making.

[0004] 2. Fixed rearview field of view, unable to cover both driving and door-opening scenarios: Traditional physical rearview mirrors have a fixed field of view and a large blind spot, making it difficult to cover the non-motorized vehicle lanes and sidewalks behind the vehicle doors. With the introduction of electronic rearview mirrors (Camera Monitor Systems, or CMS), although some related technologies can provide digital images, their displays are usually cropped with a fixed, narrow field of view to meet driving regulations. In parking and door-opening scenarios, they cannot effectively present the full view to the side and rear, and lack dynamic switching between narrow driving views and wide door-opening views.

[0005] 3. Limited intervention methods, easily causing user resentment or safety hazards: Traditional intervention methods fall into two extremes: one type only relies on audible and visual alarms, depending on the occupant's judgment and easily ignored; the other type uses a direct hard lock, which easily causes panic among occupants and poses safety hazards in emergency avoidance or escape scenarios. Furthermore, existing systems lack accurate prediction of occupants' intention to open the door, and using only the handle as a trigger condition leads to frequent false alarms.

[0006] In summary, the door opening warning system in the relevant technologies has the following drawbacks: the trigger lag leads to insufficient reaction time, the rear view cannot cover both driving and parking scenarios, and the intervention methods are mechanical and simple, which are prone to false alarms and aversion. Summary of the Invention

[0007] The purpose of this application is to provide a vehicle door opening anti-collision control method, system and vehicle to achieve the technical effect of intelligent door opening anti-collision.

[0008] A first aspect of this application provides a vehicle door opening collision avoidance control method, the method comprising: Acquire images of the occupants; If it is determined from the image that the occupant intends to open the door, obtain the door opening field of view; Within the opening field of view, acquire motion state information of moving targets outside the vehicle; Based on the motion state information, the collision time between the moving target and the vehicle door is estimated; Based on the collision time, the vehicle door locks are controlled to perform an opening restriction action.

[0009] In the aforementioned implementation process, the intention to open the door is recognized and the rear view is adapted before the occupant physically touches the door handle, thus advancing the timing of warning and intervention. At the same time, the switch from a narrow driving view to a wide door opening view eliminates blind spots in the side and rear view. Based on this, the collision time is used as the decision-making basis, and humanized tactile feedback is achieved by restricting the door lock. This not only effectively prevents high-risk door opening behaviors, but also avoids the panic caused by traditional hard locking and the negative feelings of users towards safety functions, thus improving the intelligence level and user acceptance of the door opening anti-collision solution.

[0010] Furthermore, the intention of the occupant to open the door is determined through the following steps: Feature extraction is performed on the image to obtain the occupant's gaze characteristics and / or arm movement characteristics; When the gaze characteristics indicate that the occupant's gaze is directed towards the outer rear of the vehicle and the duration of this gaze exceeds a first preset threshold, it is determined that the occupant intends to open the door; and / or, When the arm movement feature indicates that the end point of the occupant's arm movement trajectory points to the door handle area, it is determined that the occupant has the intention to open the door.

[0011] In the above implementation process, by using gaze duration and arm trajectory prediction, high-precision, low-latency advance recognition of the occupant's intention to open the door is achieved, effectively avoiding missed intentions due to the failure of a single sensor or the lack of a single feature, and improving the environmental adaptability of the solution under different lighting conditions and different occupant postures.

[0012] Furthermore, the vehicle is equipped with an electronic rearview mirror module; the acquisition of the door opening field of view includes: If it is determined from the image that the occupant intends to open the door, a door opening intention pre-trigger signal is generated; Based on the pre-triggered signal of the door opening intention, the electronic rearview mirror module is controlled to switch the display screen from the driving field of view to the door opening field of view; the driving field of view is the display field of view of the electronic rearview mirror module when the vehicle is in motion; the door opening field of view is greater than the driving field of view.

[0013] In the above implementation process, the electronic rearview mirror module switches the display screen from a narrower driving field of view to a wider door opening field of view before the occupant physically touches the door handle, realizing a cascaded response of intention perception and field of view adaptation; at the same time, this switching process does not require changing the physical angle or position of the external camera, nor does it require the installation of additional sensor hardware, thus solving the pain point of the conflict between the field of view requirements of the electronic rearview mirror in driving scenarios and parking and door opening scenarios at low cost.

[0014] Furthermore, the electronic rearview mirror module includes an exterior camera; controlling the electronic rearview mirror module to switch the display screen from the driving field of view to the door opening field of view includes: Acquire full-frame raw image data captured by the vehicle's external camera; Based on the preset door opening field of view parameters, a wide-view image corresponding to the door opening field of view is cropped from the full-frame original image data; The wide-angle image is output to the electronic rearview mirror module for display.

[0015] In the above implementation process, the full-frame raw pixel data of the camera sensor is read, and the ROI cropping matrix is ​​dynamically reconstructed according to the preset door opening field of view parameters. A wide-angle image covering the non-motorized vehicle lane and sidewalk is extracted from the full frame and output to the electronic rearview mirror module. With extremely low marginal cost and instantaneous switching delay, the same external camera can output images with different field of view ranges as needed according to different driving scenarios under the same physical installation state. This not only meets the driving regulations but also meets the functional requirements of wide field of view coverage and blind spot elimination in door opening scenarios.

[0016] Furthermore, the vehicle is equipped with a surround-view imaging module; the method further includes: Acquire the lateral surround view image provided by the surround view image module; The wide-view image and the side-view image are fused to obtain a fused image; The fused image is output to the electronic rearview mirror module for display, replacing the wide-angle image.

[0017] In the above implementation process, the surround view camera is installed at a lower position and has a special angle that is more biased towards the lower side of the vehicle body. This makes up for the blind spot in the area adjacent to the door caused by the CMS camera due to its installation height and horizontal orientation, so that when the occupant intends to open the door, he / she can know the vehicle behind and the near-field environment at the same time.

[0018] Further, the image fusion of the wide-view image and the lateral panoramic image includes: Extract feature points from the wide-view image and the side-view image; The feature points of the wide-view image are matched with the feature points of the side-view image to obtain the matching result; Based on the matching results, calculate the homography transformation matrix; Based on the homography transformation matrix, the lateral panoramic image is projected onto the coordinate system of the wide-view image; Image fusion is performed in overlapping areas to fill in the lateral blind spots.

[0019] In the above implementation process, local feature points are first extracted from the two images and a matching relationship is established. Using this as a constraint, the homography transformation matrix describing the perspective projection relationship between the two camera planes is calculated. Then, the lateral panoramic image is projected onto the coordinate system of the CMS wide-view image to eliminate all geometric misalignment caused by the difference in camera installation pose. Finally, pixel information from the two sensors is synthesized in the overlapping area of ​​the images, so that the fused single image achieves a visually coherent and seamless effect in terms of color, brightness and texture, thus filling in the absolute blind spot formed by the physical installation angle limitation of the CMS camera.

[0020] Furthermore, acquiring motion state information of a moving target outside the vehicle within the door opening field of view includes: The moving target is identified within the image area corresponding to the opening field of view. The relative distance, relative speed, and relative acceleration of the moving target relative to the vehicle are obtained as the motion state information.

[0021] In the above implementation process, the moving target is identified in the area corresponding to the door opening field of view, and its relative distance, relative speed, and relative acceleration relative to the vehicle are obtained as motion state information, which provides a multi-dimensional data foundation for the accurate calculation of subsequent collision time.

[0022] Furthermore, the step of estimating the collision time between the moving target and the vehicle door based on the motion state information includes: Calculate the initial collision time based on the relative distance and the relative velocity; The motion pattern of the moving target is determined based on the relative acceleration. When the relative acceleration determines that the moving target is in a state of uniform speed or accelerated approach, the initial collision time is taken as the collision time. When the moving target is determined to be in a deceleration state based on the relative acceleration, if it is predicted that the moving target can come to a complete stop before reaching the door position, the initial collision time is positively gained to obtain the corrected collision time.

[0023] In the above implementation process, for high-risk scenarios of acceleration or constant speed approach, the collision time calculated based on the current speed is directly adopted to ensure timely intervention; for low-risk scenarios of deceleration and yielding, it is further predicted whether the moving target can stop completely before reaching the door position. If the prediction result is positive, the collision time is extended to make the system's assessment of the danger level of the moving target more lenient, thereby exempting the safe scenario where the target can stop in time even though it is close.

[0024] Furthermore, controlling the vehicle door lock to perform the door opening restriction action based on the collision time includes: When the collision time exceeds the second preset threshold, the door lock is controlled to maintain a basic damping state to allow the door to open freely; When the collision time does not exceed the second preset threshold but is greater than the third preset threshold, the door lock is controlled to apply a first-level damping; the first-level damping is greater than the basic damping, and the first-level damping is used to prompt the occupant to apply more force to open the door; the second preset threshold is greater than the third preset threshold; When the collision time does not exceed the third preset threshold, the door lock is controlled to apply a second level of damping and perform a full lock action; the second level of damping is greater than the first level of damping, and the second level of damping is used to forcibly prevent the door from opening.

[0025] In the above implementation process, a gradual intervention was adopted to replace the simple control logic of locking or not locking in the traditional DOW system. This achieved a transition from gentle reminders to strong warnings and then to mandatory protection, respecting the occupants' driving autonomy and user experience to the greatest extent while ensuring safety.

[0026] A second aspect of this application provides a vehicle door opening collision avoidance control system, the system comprising: The acquisition module is used to acquire images of the occupants; The acquisition module is further configured to acquire the door opening field of view when it is determined from the image that the occupant intends to open the door; The acquisition module is also used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The calculation module is used to estimate the collision time between the moving target and the vehicle door based on the motion state information; The control module is used to control the vehicle door locks to perform door opening restriction actions based on the collision time.

[0027] In the aforementioned implementation process, the intention to open the door is recognized and the rear view is adapted before the occupant physically touches the door handle, thus advancing the timing of warning and intervention. At the same time, the switch from a narrow driving view to a wide door opening view eliminates blind spots in the side and rear view. Based on this, the collision time is used as the decision-making basis, and humanized tactile feedback is achieved by restricting the door lock. This not only effectively prevents high-risk door opening behaviors, but also avoids the panic caused by traditional hard locking and the negative feelings of users towards safety functions, thus improving the intelligence level and user acceptance of the door opening anti-collision solution.

[0028] A third aspect of this application provides a vehicle, the vehicle comprising: Controller; The cabin sensing module is communicatively connected to the controller and is used to collect images of the occupants; An environmental perception module, which is communicatively connected to the controller, is used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The door lock execution module is communicatively connected to the controller and is used to perform door opening restriction actions on the vehicle door lock; The controller is configured to acquire the images collected by the cabin perception module; if it is determined based on the images that the occupant intends to open the door, acquire the door opening field of view; control the environmental perception module to acquire the motion state information within the door opening field of view; based on the motion state information, estimate the collision time between the moving target and the vehicle door; and based on the collision time, control the door lock execution module to perform the door opening restriction action.

[0029] In the aforementioned implementation process, the OMS actively perceives the occupant's gaze and skeletal trajectory, completing intent recognition before the occupant actually touches and opens the car door. This provides valuable computational window time for subsequent image matrix transformation, target detection, and actuator response, improving the collision avoidance success rate. It eliminates the need for costly and easily worn traditional mechanical rearview mirror physical rotation mechanisms. By dynamically reconstructing the ROI cropping matrix of the CMS high-definition image sensor in a static state and integrating AVM surround-view pixels, it achieves a software-level switch from a narrow driving angle to an ultra-wide door opening angle, resulting in a low-cost and highly reliable solution. It abandons the binary, crude intervention of related technologies that either don't lock or hard-lock. By controlling the electromagnetic damping of the E-Latch, the handle becomes heavier during the yellow warning phase, transforming external danger into a direct tactile sensation for the passenger. This gradual warning avoids causing occupant panic while providing a high degree of collision avoidance alerts, and also avoids the mechanical aversion caused by occasional false alarms from the algorithm that hard-lock the door. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A flowchart illustrating a vehicle door opening collision avoidance control method provided in an embodiment of this application; Figure 2 This is an overall flowchart provided for an embodiment of this application. Detailed Implementation

[0032] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0033] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0034] To address any of the aforementioned problems, embodiments of this application provide a vehicle door opening collision avoidance control method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating a vehicle door opening collision avoidance control method provided in an embodiment of this application.

[0035] In this embodiment, the method includes: Step S10: Acquire images of the occupants; It should be noted that the occupant image refers to two-dimensional or three-dimensional image data including information about the occupant's head, eyes, arms, and other body parts. The source of this image is not limited to a single sensor; it can be a combination of infrared cameras, visible light cameras, or ToF depth cameras. Specifically, occupant interior images are acquired through an in-cabin sensing module (such as an infrared camera or a ToF camera).

[0036] In some embodiments, the method further includes: acquiring vehicle status information; when the vehicle status information indicates that the vehicle speed is zero and the gear is in parking gear or the parking brake is activated, activating the cabin sensing module to start acquiring the occupant image.

[0037] Step S20: If it is determined from the image that the occupant intends to open the door, obtain the door opening field of view; It should be noted that this embodiment determines whether an occupant intends to open the door not by detecting the hand handle, but by predicting based on image analysis. In some embodiments, the intention recognition node is moved forward to achieve prediction before the occupant physically touches the door handle through eye tracking and / or arm trajectory prediction.

[0038] The door opening field of view is a larger field of view than the driving field of view, used to cover the blind spots of the side and rear non-motorized vehicle lanes and sidewalks.

[0039] Optionally, the door opening field of view can be obtained by: (1) re-cropping the full-frame image of the vehicle exterior camera using a software algorithm; (2) calling the preset door opening scene display parameters.

[0040] Step S30: Within the opening field of view, acquire motion state information of the moving target outside the vehicle; It should be noted that the system detects and tracks moving targets to the side and rear of the vehicle and collects their motion parameters within the wide field of view of the door opening angle, rather than detecting them within the narrow field of view in driving mode. This ensures that target detection will not miss vehicles or pedestrians in the blind spot to the side and rear due to insufficient field of view.

[0041] A moving target refers to a dynamic obstacle that may collide with the vehicle's door, such as a non-motorized vehicle (e.g., a bicycle, an electric vehicle), a motorcycle, a pedestrian, or another vehicle approaching from behind.

[0042] Motion status information includes the relative distance, relative speed, and relative acceleration of the moving target relative to the vehicle.

[0043] Step S40: Based on the motion state information, estimate the collision time between the moving target and the vehicle door; It should be noted that the collision time refers to the time required for the moving target to reach the vehicle's door position. The shorter the collision time, the higher the collision risk.

[0044] Step S50: Based on the collision time, control the vehicle door lock to perform an opening restriction action.

[0045] In practice, the door opening restriction action encompasses a variety of progressive intervention modes, ranging from free opening to opening requiring force and finally to forced locking. When the collision time is long, only basic damping is maintained; when the collision time is moderate, damping is increased; and when the collision time is short, full locking is applied, forcibly prohibiting door opening.

[0046] Controlling vehicle door lock operation refers to the variable adjustment of door lock damping and locking state through electronic control. The specific actuator includes a combination of damping adjustment unit (such as electromagnet or magnetorheological fluid device) and electronic motor.

[0047] This embodiment uses tactile feedback, i.e., changes in door handle damping, rather than simply sound and light alarms, to enable occupants to perceive danger signals when attempting to open the door, thus reducing user frustration caused by false alarms.

[0048] In this embodiment, the intention to open the door is recognized and the rear view is adapted before the occupant physically touches the door handle, thus advancing the timing of warning and intervention. At the same time, the switch from a narrow driving view to a wide door opening view eliminates blind spots in the side and rear view. Based on this, the collision time is used as the decision-making basis, and humanized tactile feedback is achieved by restricting the door lock. This not only effectively prevents high-risk door opening behavior, but also avoids the panic caused by traditional hard locking and the negative feelings of users towards safety functions, thus improving the intelligence level and user acceptance of the door opening anti-collision solution.

[0049] Based on any of the above embodiments, the intention of the occupant to open the door is determined by the following steps: Feature extraction is performed on the image to obtain the occupant's gaze characteristics and / or arm movement characteristics; When the gaze characteristics indicate that the occupant's gaze is directed towards the outer rear of the vehicle and the duration of this gaze exceeds a first preset threshold, it is determined that the occupant intends to open the door; and / or, When the arm movement feature indicates that the end point of the occupant's arm movement trajectory points to the door handle area, it is determined that the occupant has the intention to open the door.

[0050] It should be noted that deep learning algorithms, such as convolutional neural networks and pose estimation networks, are used to extract feature vectors related to the occupant's behavioral intentions from the original image.

[0051] Gaze characteristics: Feature information extracted from the image of the eye region, including but not limited to parameters such as eye orientation, head posture angles (such as yaw angle and pitch angle), gaze direction vector, gaze point coordinates, and gaze duration.

[0052] Arm movement features: Feature information extracted from images of the upper limb region of the occupant, including but not limited to the three-dimensional coordinates of the arm joints, arm motion vectors, motion velocity and acceleration, hand position coordinates and motion trajectory.

[0053] In the specific implementation, a deep learning model is used to extract visual features from the occupant images collected by the cabin perception module, and two types of biological behavioral features are extracted from them, namely gaze features and arm movement features.

[0054] Looking outwards and to the rear of the vehicle refers to an occupant's gaze being directed towards the outside of the left or right side window of the vehicle, and further behind. The underlying logic of this behavior is that, before opening the door, occupants instinctively observe oncoming traffic from the side and rear, using this eye movement to confirm the safety of their surroundings before opening the door.

[0055] The duration of gaze exceeds the first preset threshold: gaze refers to the stable focus of the gaze on a specific area rather than rapid scanning; the first preset threshold is a calibrable time parameter, which is not limited in this embodiment. Optionally, the first preset threshold is 300ms-500ms, which is used to distinguish between conscious safety observation and unconscious brief gaze deviation.

[0056] Arm movement trajectory endpoint: Using 3D skeletal joint extraction algorithms, such as OpenPose and MediaPipe, pose estimation models track the occupant's arm, especially the forearm and hand, in 3D space in real time, and extrapolate and predict its movement trend to obtain the final landing point of the hand at a future moment. The endpoint here does not mean that the arm has reached the handle, but rather the trend endpoint predicted based on the motion vector.

[0057] The end point of the arm trajectory points to the door handle area, which means that the occupant's hand is in the process of approaching the handle, but has not yet actually made contact with it.

[0058] The door handle area refers to the two-dimensional or three-dimensional spatial range of the door handle and its surroundings; it is a calibrated region of interest. The size of this area can be preset according to the actual vehicle interior layout.

[0059] Specifically, if an occupant gazes at the side window or the rear outer side of the vehicle for an extended period, it is determined that they intend to open the door. If the trajectory of an occupant's arm ends in the door handle area, it is determined that they intend to open the door.

[0060] In this embodiment, by using gaze duration and arm trajectory prediction, high-precision, low-latency advance recognition of the occupant's intention to open the door is achieved, effectively avoiding missed intentions due to the failure of a single sensor or the lack of a single feature, and improving the environmental adaptability of the solution under different lighting conditions and different occupant postures.

[0061] Based on any of the above embodiments, the vehicle is equipped with an electronic rearview mirror module; the step of obtaining the door opening field of view includes: If it is determined from the image that the occupant intends to open the door, a door opening intention pre-trigger signal is generated; Based on the pre-triggered signal of the door opening intention, the electronic rearview mirror module is controlled to switch the display screen from the driving field of view to the door opening field of view; the driving field of view is the display field of view of the electronic rearview mirror module when the vehicle is in motion; the door opening field of view is greater than the driving field of view.

[0062] It should be noted that the door opening intention pre-trigger signal is generated before the occupant actually performs the door opening action, such as 300ms-500ms before the handle is pulled. This signal itself can be a digital level transition, a CAN / LIN bus message, or an internal software interrupt.

[0063] In its implementation, the electronic rearview mirror module includes an exterior camera, an image processing unit, and an in-vehicle display screen. The central controller sends control commands to the electronic rearview mirror module to change its display parameters. This control does not involve adjusting the physical angle of the exterior camera.

[0064] The displayed image refers to the image content seen by occupants on the CMS display screen inside the vehicle. Unlike traditional physical rearview mirrors, the display image of the electronic rearview mirror module is generated by software cropping and rendering, which provides operational flexibility for switching the field of view.

[0065] The driving field of view is the display field of view used when the vehicle is in motion. It is usually cropped to a narrower horizontal field of view to meet traffic regulations, in order to ensure the image clarity and recognizability of targets on a display screen with limited resolution. For example, the driving field of view is 30°-40° horizontally, and the door opening field of view is 70°-90° horizontally.

[0066] Electronic rearview mirror modules typically use wide-angle sensors with full-frame output capability (horizontal FoV ≥ 120°) for their exterior cameras. However, due to limitations imposed by display resolution and driving regulations on image distortion, only the central area is cropped out during actual display. This embodiment achieves dynamic expansion of the field of view by redefining the ROI cropping parameters at the software level, without altering any hardware physical state.

[0067] Vehicle driving status refers to the vehicle's operating mode when the speed is greater than zero or the gear is in D / R gear. In this mode, the image cropping of the electronic rearview mirror needs to meet the relevant regulations regarding field of view, magnification, and distortion, so a narrower field of view is usually used to ensure driving safety.

[0068] In this embodiment, the electronic rearview mirror module switches the display from a narrower driving field of view to a wider door opening field of view before the occupant physically touches the door handle, realizing a cascaded response of intent perception and field of view adaptation. At the same time, this switching process does not require changing the physical angle or position of the external camera, nor does it require the installation of additional sensor hardware, thus solving the pain point of the conflict between the field of view requirements of the electronic rearview mirror in driving scenarios and parking and door opening scenarios at low cost.

[0069] Based on any of the above embodiments, the electronic rearview mirror module includes an exterior camera; controlling the electronic rearview mirror module to switch the display screen from the driving field of view to the door opening field of view includes: Acquire full-frame raw image data captured by the vehicle's external camera; Based on the preset door opening field of view parameters, a wide-view image corresponding to the door opening field of view is cropped from the full-frame original image data; The wide-angle image is output to the electronic rearview mirror module for display.

[0070] Understandably, an exterior camera is an electronic camera device installed on the side of a vehicle, such as on the front fender or at the base of the rearview mirror, to capture images of the road environment to the side and rear.

[0071] The vehicle's external camera continuously senses light and generates digital image signals at a certain frame rate, such as 30fps or 60fps.

[0072] Full-frame: The complete effective pixel area of ​​a camera image sensor, not a cropped or scaled sub-area. For example, if an exterior camera sensor is 8 megapixels with a horizontal resolution of 3840 pixels, its full-frame data refers to all 3840 columns of pixels. Since exterior cameras typically use wide-angle lenses (horizontal physical field of view ≥120°), their full-frame data includes a wide field of view from near the vehicle body to the far sides.

[0073] Raw image data refers to the lowest-level pixel data that has not undergone ROI cropping, perspective correction, distortion correction, or display adaptation processing. This data retains all the image information output by the sensor.

[0074] When driving, electronic rearview mirrors only use the central area of ​​the full-frame view, such as the horizontal 30°-40° area, while the peripheral view at both ends of the full-frame (corresponding to the horizontal 30°-120° area) is discarded during driving. This embodiment acquires full-frame data, enabling the system to access these discarded peripheral pixels and providing a data source for subsequent field-of-view expansion.

[0075] Specifically, the system reads complete pixel matrix data output by the photosensitive element of the external camera sensor. This data contains all the field-of-view information that the camera can capture at its physical field of view. The image processing unit of the central controller or electronic rearview mirror reads raw image frame data from the external camera sensor through a data transmission interface such as MIPI CSI or GMSL.

[0076] The preset door opening field of view parameters include specific numerical values ​​for the door opening field of view, such as a horizontal range of 70°-90° and its corresponding ROI cropping coordinate boundaries. The specific form of the door opening field of view parameters can be a pixel coordinate range, such as from column x1 to column x2 of the full-frame view, an angle range, such as the degree by which the left and right boundaries of the field of view expand, or a combination of both. The value of this parameter can be determined during the system calibration phase based on the vehicle model, camera installation location, and regulatory requirements.

[0077] The process of cropping a wide-view image corresponding to the door-opening field of view from the full-frame raw image data refers to extracting a region of interest (ROI) from the pixel matrix using a software algorithm. This operation is essentially a matrix slicing operation, extracting a sub-matrix within a specified row and column range from the full-frame two-dimensional pixel array. Compared to the ROI in the driving state (centered and relatively narrow), the ROI in the door-opening state is wider (extending to both sides of the image), but both share the same full-frame data source.

[0078] The wide-view image refers to the sub-image obtained after cropping, and its horizontal field of view is the preset door opening field of view. Compared with the driving image, this image contains richer information about the side and rear environment and covers traditional blind spots.

[0079] Specifically, the central controller or image processing unit transmits the cropped image data to the display driver chip of the electronic rearview mirror module through a display interface such as LVDS, eDP, or HDMI, so that it can be displayed on the in-vehicle display screen. In this way, when the occupant intends to open the door, they can see a wider side and rear view by looking at the in-vehicle display screen, effectively eliminating blind spots.

[0080] In this embodiment, the full-frame raw pixel data of the camera sensor is read, and the ROI cropping matrix is ​​dynamically reconstructed according to the preset door opening field of view parameters. A wide-angle image covering the non-motorized vehicle lane and sidewalk is extracted from the full frame and output to the electronic rearview mirror module. With extremely low marginal cost and instantaneous switching delay, the same vehicle exterior camera can output images with different field of view ranges as needed according to different driving scenarios under the same physical installation state. This not only meets the driving regulations but also meets the functional requirements of wide field of view coverage and blind spot elimination in door opening scenarios.

[0081] Based on any of the above embodiments, the vehicle is equipped with a surround-view imaging module; the method further includes: Acquire the lateral surround view image provided by the surround view image module; The wide-view image and the side-view image are fused to obtain a fused image; The fused image is output to the electronic rearview mirror module for display, replacing the wide-angle image.

[0082] It should be noted that the Around View Monitoring (AVM) system uses the surround view imaging module as a source of lateral surround view images. It refers to a panoramic surround view system composed of wide-angle fisheye cameras installed in the front, rear, left, and right directions of the vehicle, along with their image processing units. This system can synthesize 360° overhead or bird's-eye view images of the vehicle's surroundings, and can also provide raw or corrected images from the individual camera perspectives. In some embodiments, it primarily utilizes the side images captured by the left or right fisheye cameras.

[0083] Side-view surround view refers to image data covering the area to the side of the vehicle in the surround view imaging module. Unlike the side and rear view of the electronic rearview mirror module (CMS) (lens facing backward, capturing vehicles at a greater distance), the surround view camera is installed at a lower position (such as below the side rearview mirror or at the front fender) and the lens is more biased towards the lower side of the vehicle body. It mainly captures the near-field area adjacent to the door, including the ground close to the vehicle body, curbs, and the edge of the non-motorized vehicle lane adjacent to the door.

[0084] The wide-angle image covers the mid-to-long distance area extending outward from the side and rear of the vehicle, clearly showing the movement of oncoming vehicles. The side surround view image covers the near-field blind spots that the CMS camera cannot see due to its limited field of view, namely the area directly below the door, such as curbs, steps, pedestrians or obstacles adjacent to the door.

[0085] The CMS wide-angle image and the panoramic side-view image differ in the following aspects: (1) different imaging angles (one is horizontal and backward, the other is oblique and downward); (2) different coordinate systems (different extrinsic parameters of their respective cameras); (3) different lighting conditions; and (4) different image resolutions. In this embodiment, the two images are fused. Optionally, spatial registration and image stitching / fusion are performed. Spatial registration refers to projecting the panoramic side-view image onto the coordinate system of the CMS wide-angle image through homography transformation, so that the two images achieve pixel-level correspondence in the overlapping area. Image stitching / fusion refers to using a pixel-level weighted fusion algorithm in the overlapping area of ​​the images to make the two images transition naturally in the overlapping area, avoiding obvious stitching marks or boundary cracks. The fused images retain their original content in the non-overlapping areas.

[0086] The fused image combines information about oncoming vehicles at medium and long distances with information about blind spots in the near field, allowing occupants to simultaneously obtain environmental information in two dimensions—whether there are oncoming vehicles behind them and whether there are obstacles near the doors—through a single view.

[0087] In this embodiment, the surround-view camera, with its lower installation position and special angle that is more biased towards the lower side of the vehicle body, fills in the blind spot in the area adjacent to the door caused by the CMS camera's installation height and horizontal orientation, so that when the occupant intends to open the door, he / she can simultaneously know the vehicle approaching from behind and the near-field environment.

[0088] Based on any of the above embodiments, the step of image fusion of the wide-view image and the side-view image includes: Extract feature points from the wide-view image and the side-view image; The feature points of the wide-view image are matched with the feature points of the side-view image to obtain the matching result; Based on the matching results, calculate the homography transformation matrix; Based on the homography transformation matrix, the lateral panoramic image is projected onto the coordinate system of the wide-view image; Image fusion is performed in overlapping areas to fill in the lateral blind spots.

[0089] It should be noted that the purpose of this embodiment is to utilize the principle of multi-view geometry to spatially align and smoothly stitch together two camera images from different mounting positions and with different imaging characteristics into a seamless image.

[0090] Feature points refer to locations in an image that exhibit significant local variations in certain attributes, such as brightness, gradient, and texture. They are typically corner points (e.g., object corners), edge intersections, spots, or the center points of textured areas. Feature points should possess the following characteristics: (1) repeatability: the same physical point can be detected in two images under different viewing angles or lighting conditions; (2) uniqueness: the local image patches (descriptors) surrounding the point have sufficient discriminative power to facilitate subsequent matching. Optionally, feature point sources include, but are not limited to, static references such as road markings (e.g., lane line edges, arrow vertices), curb corners, edge corners of adjacent vehicles, and random patches of road texture.

[0091] For each input image, the algorithm outputs a list of feature points, each feature point containing: (1) two-dimensional pixel coordinates (x, y) in the image; (2) scale information (if the algorithm has scale invariance); (3) orientation information (if the algorithm has rotation invariance); (4) a high-dimensional feature descriptor vector, such as ORB being a 256-bit binary descriptor and SIFT being a 128-dimensional floating-point vector.

[0092] The matching process is based on the similarity measure between descriptors: (1) For floating-point descriptors, such as SIFT, Euclidean distance is used to measure similarity; (2) For binary descriptors, such as ORB and BRIEF, Hamming distance or normalized cross-correlation NCC is used.

[0093] The matching logic is as follows: for each feature point in the wide-view image, find the point with the closest descriptor distance in the feature point library of the side-view image (nearest neighbor matching), and use the ratio test of the nearest neighbor to the second nearest neighbor to filter out fuzzy matches in order to improve matching accuracy.

[0094] The matching result is a set of point pairs. The quality of the matching result is determined by the following indicators: the number of matching point pairs, the more the better for the accurate calculation of the subsequent matrix; the spatial distribution of matching points, the more uniform the distribution, the better for the accuracy of the global transformation; and the proportion of outliers, i.e. the proportion of erroneous matching point pairs, which needs to be eliminated by subsequent robust estimation methods, such as RANSAC.

[0095] Because the two images have significant viewpoint differences, a large number of incorrect matches are inevitable during the matching process. For example, a distant vehicle corner point in the CMS image may be incorrectly matched as a roadside corner point in the surround view image. Therefore, in some embodiments, preliminary screening is performed using descriptor distance thresholds and ratio tests to eliminate obviously unreasonable candidate matching pairs in order to obtain the matching result.

[0096] The homography transformation matrix is ​​a 3×3 homogeneous coordinate transformation matrix that describes the projection mapping relationship of the same plane in three-dimensional space under the viewpoints of two different cameras.

[0097] Homography transformation requires that the scene content captured in two images is approximately located on the same spatial plane. In a doorway scene, the overlapping area between the CMS image and the panoramic image is mainly the ground plane (such as the road surface and curb). These contents approximately satisfy planar constraints, therefore, homography transformation can well describe the geometric mapping relationship between the two images. Textures, markings, and other feature points on the ground are ideal observation targets for calculating the homography matrix.

[0098] Projection is the process of mapping the coordinates of each pixel position in a side-view image according to a homography matrix. Since the transformed coordinates are usually not integer pixel positions, this process often requires interpolation, such as bilinear or bicubic interpolation, to generate a smooth pixel value distribution in the target coordinate system.

[0099] The lateral panoramic image refers to the complete lateral panoramic image, not limited to the local area overlapping with the CMS. However, only the part that falls within the CMS image coordinate system after projection will be actually used in the subsequent fusion. The part that exceeds the boundary of the CMS image will be discarded during fusion.

[0100] The coordinate system of the wide-view image: The two-dimensional image coordinate system in which the CMS wide-view image is located, with the upper left corner of the image as the origin, the horizontal axis to the right as the u-axis, and the vertical axis downward as the v-axis. The projected panoramic image is aligned with this coordinate system, so that the same physical point in the two images (such as the same corner of the curb) is located at the same pixel coordinate position after transformation.

[0101] Before projection, there are differences in rotation, translation, scale, and perspective distortion between the side-view panoramic image and the CMS wide-view image; after projection, the two images are consistent in spatial geometry, and the panoramic image is distorted into the same perspective structure as the CMS image, achieving pixel-level spatial correspondence.

[0102] Homography transformation eliminates all geometric misalignment issues caused by differences in installation position, installation angle, and lens distortion between two heterogeneous cameras, enabling two images that could not be directly superimposed to have a pixel-to-pixel correspondence.

[0103] The image overlap region refers to the pixel area where the side-view panoramic image and the CMS wide-angle image coincide in spatial location after projection transformation. This region corresponds to the physical space jointly covered by both cameras, such as the near-field area adjacent to the side of the car door. Outside the overlap region, the fused image directly inherits the pixel values ​​from a single image source (the near-field area unique to the panoramic image fills in the blind spots of the CMS, while the distant area unique to the CMS image retains the original field of view).

[0104] Since two images may differ in brightness, color temperature, and contrast when depicting the same physical scene in the overlapping area (due to differences in sensor model, exposure parameters, and white balance settings), simple overlay or hard switching would result in noticeable seams or brightness jumps at the fusion boundary. Optionally, this embodiment employs a pixel-level weighted fusion algorithm, which sets gradually varying weight coefficients, such as linear or Gaussian gradients, along the direction from the CMS dominant area to the surrounding dominant area within the overlapping region. This allows the pixel values ​​of the two images to be mixed according to their weight ratios, achieving a smooth image transition.

[0105] The ultimate goal of the entire fusion operation is to fill in the blind spots on the sides and rear. Due to the physical limitations of the camera's orientation and installation height, the CMS wide-angle image cannot cover the near-field area immediately below the side of the door; the surround-view image precisely covers this area. By spatially aligning and fusing the surround-view image with the CMS image, the pixel information in the surround-view image that originally belonged only to the blind spot is embedded into the field of view of the CMS image. This results in the final displayed image containing both the original mid-to-long-range target information from the CMS and the near-field blind spot information provided by the surround view, thus achieving completeness.

[0106] In this embodiment, local feature points are first extracted from the two images and a matching relationship is established. Using this as a constraint, the homography transformation matrix describing the perspective projection relationship between the two camera planes is calculated. Then, the lateral panoramic image is projected onto the coordinate system of the CMS wide-view image to eliminate all geometric misalignment caused by the difference in camera installation pose. Finally, pixel information from the two sensors is synthesized in the overlapping area of ​​the images, so that the fused single image achieves a visually coherent and seamless integrated effect in terms of color, brightness, and texture, thus filling in the absolute blind spot formed by the physical installation angle limitation of the CMS camera.

[0107] Based on any of the above embodiments, the step of acquiring the motion state information of the moving target outside the vehicle within the door opening field of view includes: The moving target is identified within the image area corresponding to the opening field of view. The relative distance, relative speed, and relative acceleration of the moving target relative to the vehicle are obtained as the motion state information.

[0108] It should be noted that the screen area refers to the two-dimensional image space covered by the electronic rearview mirror module's display, that is, the pixel area on the CMS screen. Each pixel position on the screen corresponds to a direction in actual space, so its spatial orientation relative to the vehicle can be deduced from the target position in the image coordinate system.

[0109] The moving targets are identified by using computer vision algorithms, such as deep learning-based object detection networks like YOLO, SSD, and Faster R-CNN, or by using traditional computer vision-based background modeling and motion detection algorithms to detect and classify the bounding boxes of moving targets in the image (e.g., "pedestrian", "bicycle", "electric vehicle", "car").

[0110] For example, since the CMS camera is stationary in the door-opening scenario and the background is relatively fixed, background subtraction can be used as a moving target detection method, which can be combined with a deep learning classifier to achieve high real-time moving target recognition.

[0111] Optionally, motion state information can be obtained by: (1) through sensors, such as millimeter-wave radar; (2) through visual ranging algorithms, such as monocular ranging based on camera intrinsic parameters, or binocular parallax ranging based on stereo vision, calculated from the image; (3) through multi-sensor fusion, such as CMS visual overlay radar integration.

[0112] Relative distance: The straight-line distance in space between the moving target and a reference point on the side of the vehicle's door or body (which can be the projected distance on a horizontal plane). The shorter the relative distance, the higher the urgency of the collision.

[0113] Relative speed: The approach speed of a moving target relative to the vehicle, which is the first derivative of distance with respect to time. In some embodiments, the vehicle is stationary, so the relative speed is numerically equal to the absolute speed of the moving target, with the direction towards the vehicle being positive. The greater the relative speed, the stronger the impact force of the collision.

[0114] Relative acceleration: The acceleration of a moving target relative to the vehicle, i.e., the first derivative of velocity with respect to time. Related technologies typically only assume uniform motion based on distance and velocity. However, this embodiment introduces acceleration, enabling the system to determine whether the target's motion pattern is accelerating towards it, approaching at a constant speed, or decelerating to give way, thereby performing differentiated collision time correction calculations.

[0115] In this embodiment, a moving target is identified within the area corresponding to the door opening field of view, and its relative distance, relative speed, and relative acceleration relative to the vehicle are obtained as motion state information. This provides a multi-dimensional data foundation for the accurate calculation of subsequent collision time.

[0116] Based on any of the above embodiments, estimating the collision time between the moving target and the vehicle door based on the motion state information includes: Calculate the initial collision time based on the relative distance and the relative velocity; The motion pattern of the moving target is determined based on the relative acceleration. When the relative acceleration determines that the moving target is in a state of uniform speed or accelerated approach, the initial collision time is taken as the collision time. When the moving target is determined to be in a deceleration state based on the relative acceleration, if it is predicted that the moving target can come to a complete stop before reaching the door position, the initial collision time is positively gained to obtain the corrected collision time.

[0117] It should be noted that the initial collision time is calculated using only two parameters: relative distance and relative velocity. Relative acceleration is not considered at this time. This is because the initial collision time is only an intermediate reference value, which will be corrected or directly adopted according to the motion mode later. In addition, the collision time calculated using only relative distance and relative velocity is, in a physical sense, the collision time under the assumption of uniform motion, which provides a reference for comparison with the motion model that takes acceleration into account later.

[0118] In calculating the initial collision time based on the relative distance and the relative speed, it is assumed that the target will approach the vehicle at a constant current speed. If the relative speed is zero or negative (i.e., the target is moving away), the collision time is undefined or infinite, which can be considered a safe state in this embodiment.

[0119] The value and sign of relative acceleration (positive, negative, zero) directly reflect the trend of target velocity change. When a moving target is approaching the vehicle at a constant or accelerating speed, its velocity will not decrease on its own, and the collision risk increases over time. Therefore, directly using the collision time calculated based on the current speed (i.e., the initial collision time) is the most optimistic estimate; the actual collision time will only be shorter, not longer. When a target is approaching the vehicle at a decelerating speed, if the deceleration continues, the target may come to a complete stop before reaching the door, thus naturally dissipating the collision risk. Therefore, the initial collision time needs to be corrected to avoid false alarms caused by a moving target eventually stopping but being misjudged as continuously approaching.

[0120] The significance of motion pattern classification lies in distinguishing between two fundamentally different scenarios: a persistent threat and a threat that is dissipating on its own. This provides a basis for differentiated collision timing decisions. Uniform / accelerating approach indicates a persistent or escalating threat, requiring no correction; deceleration and yielding indicate a threat that may dissipate on its own, requiring further analysis.

[0121] Under constant speed conditions, the speed of the moving target remains unchanged, and the collision time calculated based on the current speed is an accurate estimate of the actual collision time. Under acceleration conditions, the speed of the moving target will continue to increase, and the actual collision time will be shorter than the initial collision time. Therefore, using the initial collision time as the final collision time is actually a conservative estimate and will not lead to an underestimation of risk. That is, the actual collision will only occur earlier, and intervention based on the initial collision time will not result in the intervention being too late.

[0122] When it is determined that the moving target is decelerating, it is further predicted whether the deceleration behavior is sufficient to bring the moving target to a complete stop before reaching the door position. If the prediction result is yes, the initial collision time is positively gained and corrected, that is, the collision time value is extended to obtain the final corrected collision time.

[0123] Specifically, the braking distance required for the moving target to reduce its current speed to zero is calculated, and this braking distance is compared with the current relative distance. If the braking distance is less than or equal to the current distance, it is predicted that the moving target will come to a complete stop before reaching the door position; if the braking distance is greater than the current distance, even if the deceleration continues, the moving target will still have residual speed when it reaches the door position, and a collision will still occur.

[0124] Positive gain correction refers to multiplying the initial collision time by a gain coefficient greater than 1, so that the corrected collision time value is greater than the initial collision time.

[0125] Specifically, the corrected collision time replaces the original collision time, and the vehicle door locks are then controlled to perform door opening restriction actions based on the corrected collision time.

[0126] In this embodiment, for high-risk scenarios involving acceleration or constant speed approach, the collision time calculated based on the current speed is directly used to ensure timely intervention; for low-risk scenarios involving deceleration and yielding, it is further predicted whether the moving target can stop completely before reaching the door position. If the prediction result is positive, the collision time is extended to make the system's assessment of the danger level of the moving target more lenient, thereby exempting the safe scenario where the target can stop in time even though it is close.

[0127] Based on any of the above embodiments, controlling the vehicle door lock to perform the door opening restriction action based on the collision time includes: When the collision time exceeds the second preset threshold, the door lock is controlled to maintain a basic damping state to allow the door to open freely; When the collision time does not exceed the second preset threshold but is greater than the third preset threshold, the door lock is controlled to apply a first-level damping; the first-level damping is greater than the basic damping, and the first-level damping is used to prompt the occupant to apply more force to open the door; the second preset threshold is greater than the third preset threshold; When the collision time does not exceed the third preset threshold, the door lock is controlled to apply a second level of damping and perform a full lock action; the second level of damping is greater than the first level of damping, and the second level of damping is used to forcibly prevent the door from opening.

[0128] In some embodiments, the control of the door lock to apply a first-level damping includes: driving a damping adjustment unit in the door lock execution module with a PWM signal of a first duty cycle to generate the first-level damping; the control of the door lock to apply a second-level damping and perform a full lock-up action includes: driving the damping adjustment unit with a PWM signal of a second duty cycle to generate the second-level damping, while controlling the electric motor to drive the locking mechanism to perform a lock-up action; wherein the second duty cycle is greater than the first duty cycle.

[0129] It should be noted that, optionally, the second preset threshold is 5 seconds. When a moving target is far from the vehicle, such as more than 50 meters away, or approaches at a low speed, the collision time between it and the door is usually greater than 5 seconds, giving the occupants sufficient reaction time to open the door and get out of the vehicle without any intervention from the system.

[0130] Maintaining the basic damping state of the door lock does not mean actively applying additional control force. Rather, it means that the system does not send any additional PWM drive signals, allowing the electronic door lock execution module to operate in its default state, i.e., the normal handle operation force state when the vehicle is stationary and there is no intention to open the door. In this embodiment, the basic damping is determined by the door lock's own spring resistance, frictional resistance, and lubrication state. This corresponds to the physical feel of the door being opened normally by the occupant with a normal force (approximately 15N-25N of handle force), without generating any additional operating resistance or locking force.

[0131] Optionally, the third preset threshold is 3 seconds. When the collision time is between 3 and 5 seconds, it indicates that the moving target behind has entered a relatively close range, such as within 30 meters, but has not yet posed an imminent threat.

[0132] The first level of damping is the intermediate operating resistance, which is greater than the basic damping and is used to provide tactile warning signals to the occupants. Specifically, the system achieves this level of damping by sending a PWM signal with a specific duty cycle, such as 30%-50%, to the damping adjustment unit. The electromagnet or magnetorheological fluid device in the damping adjustment unit changes its internal magnetic field strength under the drive of the PWM signal, thereby increasing the sliding friction or rotational resistance of the locking mechanism. This significantly increases the force required to operate the door handle, for example, from 15N-25N to 35N-50N. When the occupant feels a significant increase in resistance when pulling the handle, they will naturally realize the risk of opening the door and will proactively pause the opening action and observe the situation to their side and rear.

[0133] If the collision time does not exceed the third preset threshold, it indicates that the moving target behind is extremely close to the vehicle, such as within 10-15 meters, and a collision will occur in a very short time. Under this urgency, opening the door may lead to a collision, and the system must intervene.

[0134] The second-stage damping is greater than the first-stage damping. This is achieved by sending a higher duty cycle, such as a 70%-90% PWM signal, to the damping adjustment unit, further increasing the operating resistance, for example, to over 60N. In this situation, it is difficult for occupants to overcome this resistance and open the door with conventional force. Simultaneously, based on the application of the second-stage damping, the locking mechanism is driven by an electronically controlled motor to perform a complete mechanical locking action, completely locking the door lock's bolt or locking pawl. These two actions work together to form a dual safety guarantee; even if the damping adjustment unit fails, the mechanical locking can still function independently, and vice versa.

[0135] In the specific implementation, see Figure 2 , Figure 2 This application provides an overall flowchart for an embodiment of the method. The vehicle door opening anti-collision control method includes the following steps: Step S1: Vehicle Status Monitoring and System Wake-up: The central controller monitors the vehicle's driving status in real time. When the vehicle speed decelerates to V=0 and the gear is shifted to P (or the parking brake is activated), the system remains active and activates the OMS, CMS, and AVM modules to enter a collaborative standby mode.

[0136] Step S2: Pre-emptive perception of occupant multimodal behavior and door opening intention: The OMS module continuously collects image data of the corresponding occupants in the cabin. The central controller uses a deep learning model to perform real-time image analysis. Decoupling Feature A (Gaze Tracking): Determine whether the occupant's gaze continuously turns towards the side window or the rear of the vehicle and stays there for more than a preset threshold T_gaze (i.e., the first preset threshold). Decoupling Feature B (Trajectory Prediction): Using a 3D skeletal joint extraction algorithm, the motion vector of the occupant's arm on the side closest to the door is tracked.

[0137] When the system detects that the occupant's gaze is directed to the side and rear, or that the end of the arm's movement trajectory is pointing towards the door handle, but within a short period of time (preset within the range of 300ms-500ms) before the occupant's physical hand touches the door handle, the central controller determines that the occupant has the intention to force open the door and immediately sends an internal signal to pre-trigger the intention to open the door.

[0138] Step S3: Adaptive widening of the digital field of view (FoV) of the electronic rearview mirror (CMS): After receiving the pre-trigger signal indicating a door opening intention, the central controller executes a software-level dynamic field-of-view widening algorithm without altering the physical angle of the external CMS camera. 1. Read the full-frame raw pixels of the external CMS high-definition image sensor (full field of view, such as a horizontal field of view of 120°). 2. Instantly reconstruct and expand the standard ROI clipping matrix (corresponding to a horizontal field of view of 30°-40°) in driving mode into a wide-angle ROI clipping matrix specifically for door opening (corresponding to a horizontal field of view of 70°-90°). 3. Call the side-view pixel stream provided by the AVM module, and perform edge pixel-level weighted fusion of the widened CMS image and the AVM image of the vehicle side through image feature point matching and homography transformation matrix transformation to fill in the side and rear blind spots. 4. On the corresponding CMS display screen inside the vehicle, the original driving view is seamlessly switched to the widened full blind spot view, and dynamic targets to the side and rear are tracked and highlighted on the screen through picture-in-picture (PIP) local magnification or colored edge boxes.

[0139] Step S4: Side and rear dynamic target tracking and real-time collision time (TTC) estimation: While widening the digital field of view, the central controller uses computer vision algorithms (such as Kalman filtering and target detection networks) to identify moving targets (including cars, motorcycles, electric vehicles, bicycles, and running pedestrians) within the widened field of view to the side and rear, and calculates in real time the relative distance S, relative velocity V, and relative acceleration A between the target and the vehicle door, and estimates the time to collision (TTC). 1. Acceleration A is 0 (indicating the moving target is approaching at a constant speed): TTC = S / V; 2. Acceleration A is greater than 0 (indicating that the moving target is accelerating towards the target): Introduce the first-order Taylor expansion or kinematic displacement formula: ; 3. Acceleration A is less than 0 (indicating that the moving target is decelerating to give way): If the moving target is predicted to come to a complete stop before reaching the door, the central controller will perform a positive gain correction on the current TTC in real time (i.e., artificially extend the safety tolerance time) to prevent the door from being erroneously locked. First calculate the driving distance : ;in, This represents the braking time for uniformly decelerated motion, that is, the braking time required for a moving target to decelerate to a complete stop at its current speed V and current deceleration A (when A<0).

[0140] if If the predicted target comes to a complete stop before reaching the car door, the central controller will make a positive gain correction to the current TTC value in real time to achieve a safe state value.

[0141] if Then calculate the arrival time (TTC) at the car door according to the formula in step 2.

[0142] Step S5: Tiered damping and multi-dimensional interactive control of electronically controlled door locks based on TTC gradients: The central controller updates the danger level in real time based on the calculated TTC and sends commands to the E-Latch electronic door locks and in-vehicle interaction hardware via the CAN / LIN bus to implement the following graded defense interventions: Level 1, Safe Passage Status: If TTC > 5s, no collision risk is determined. The CMS display maintains a normal viewing angle or a normal wide-angle display, the E-Latch door lock resistance is at a basic normal level, and the doors can be opened freely.

[0143] Level 2, Yellow Alert: If 3s ≤ TTC ≤ 5s, a potential collision risk is identified. Moving targets on the CMS display are marked with a yellow box, and the ambient lighting on the corresponding side of the interior flashes in a pulsed manner. Simultaneously, the central controller sends a pulse width modulation (PWM) signal to the E-Latch module, activating the electromagnetic damper within the door lock, increasing the mechanical resistance of the door handle to 3 to 5 times the normal level. Occupants will feel a noticeable heaviness / damping sensation when attempting to pull the handle, providing a direct, tactile warning of danger.

[0144] Level 3, Red Danger State: If TTC < 3s, a collision is deemed imminent. The target on the CMS display turns bright red and flashes, the vehicle's audio system emits a high-frequency alarm sound, accompanied by a voice prompt. Simultaneously, the central controller drives the E-Latch module's electronic interlock motor to implement active electronic interlock (completely locking the doors). At this time, pulling the door handles will not unlock the doors. The system automatically de-interlocks and restores door unlocking functionality once the moving target has moved away, the danger has passed, and the TTC has returned to a safe range.

[0145] In this embodiment, a progressive intervention is used instead of the simple control logic of locking or unlocking the traditional DOW system. This achieves a transition from gentle reminders to strong warnings and then to mandatory protection, respecting the occupants' driving autonomy and user experience to the greatest extent possible while ensuring safety.

[0146] This application provides a vehicle door opening collision avoidance control system, the system comprising: The acquisition module is used to acquire images of the occupants; The acquisition module is further configured to acquire the door opening field of view when it is determined from the image that the occupant intends to open the door; The acquisition module is also used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The calculation module is used to estimate the collision time between the moving target and the vehicle door based on the motion state information; The control module is used to control the vehicle door locks to perform door opening restriction actions based on the collision time.

[0147] In some embodiments, the intention of the occupant to open the door is determined by the following steps: performing feature extraction on the image to obtain the occupant's gaze characteristics and / or arm movement characteristics; when the gaze characteristics indicate that the occupant's gaze is directed toward the rear of the vehicle and the duration of the gaze exceeds a first preset threshold, the intention of the occupant to open the door is determined; and / or, when the arm movement characteristics indicate that the end point of the occupant's arm movement trajectory points toward the door handle area, the intention of the occupant to open the door is determined.

[0148] In some embodiments, the vehicle is equipped with an electronic rearview mirror module; the acquisition module is further configured to generate a door opening intention pre-trigger signal when it is determined from the image that the occupant intends to open the door; based on the door opening intention pre-trigger signal, control the electronic rearview mirror module to switch the display screen from the driving field of view to the door opening field of view; the driving field of view is the display field of view of the electronic rearview mirror module when the vehicle is in motion; the door opening field of view is greater than the driving field of view.

[0149] In some embodiments, the electronic rearview mirror module includes an exterior camera; the acquisition module is further configured to acquire full-frame raw image data captured by the exterior camera; crop a wide-angle image corresponding to the door opening field of view from the full-frame raw image data according to preset door opening field of view parameters; and output the wide-angle image to the electronic rearview mirror module for display.

[0150] In some embodiments, the vehicle is equipped with a surround view imaging module; the acquisition module is further configured to acquire a side surround view image provided by the surround view imaging module; fuse the wide-angle image with the side surround view image to obtain a fused image; and output the fused image to the electronic rearview mirror module for display to replace the wide-angle image.

[0151] In some embodiments, the acquisition module is further configured to extract feature points from the wide-view image and the side-view image; perform feature point matching between the feature points of the wide-view image and the feature points of the side-view image to obtain a matching result; calculate a homography transformation matrix based on the matching result; project the side-view image onto the coordinate system of the wide-view image according to the homography transformation matrix; and perform image fusion in the overlapping area of ​​the images to fill in the side-rear blind spot.

[0152] In some embodiments, the acquisition module is further configured to identify the moving target within the screen area corresponding to the door opening field of view; and acquire the relative distance, relative speed, and relative acceleration of the moving target relative to the vehicle as the motion state information.

[0153] In some embodiments, the calculation module is further configured to calculate the initial collision time based on the relative distance and the relative velocity; determine the motion mode of the moving target based on the relative acceleration; when the moving target is determined to be in a constant speed or accelerating approach state based on the relative acceleration, the initial collision time is used as the collision time; when the moving target is determined to be in a deceleration state based on the relative acceleration, if it is predicted that the moving target can come to a complete stop before reaching the door position, the initial collision time is positively gained to obtain the corrected collision time.

[0154] In some embodiments, the control module is further configured to: control the door lock to maintain a basic damping state to allow the door to open freely when the collision time exceeds a second preset threshold; control the door lock to apply a first-level damping when the collision time does not exceed the second preset threshold but is greater than a third preset threshold; the first-level damping is greater than the basic damping, and the first-level damping is used to prompt the occupant to apply greater force to open the door; the second preset threshold is greater than the third preset threshold; control the door lock to apply a second-level damping and perform a full lock action when the collision time does not exceed the third preset threshold; the second-level damping is greater than the first-level damping, and the second-level damping is used to forcibly prevent the door from opening.

[0155] In this embodiment, the intention to open the door is recognized and the rear view is adapted before the occupant physically touches the door handle, thus advancing the timing of warning and intervention. At the same time, the switch from a narrow driving view to a wide door opening view eliminates blind spots in the side and rear view. Based on this, the collision time is used as the decision-making basis, and humanized tactile feedback is achieved by restricting the door lock. This not only effectively prevents high-risk door opening behavior, but also avoids the panic caused by traditional hard locking and the negative feelings of users towards safety functions, thus improving the intelligence level and user acceptance of the door opening anti-collision solution.

[0156] This application provides a vehicle, which includes: Controller; The cabin sensing module is communicatively connected to the controller and is used to collect images of the occupants; An environmental perception module, which is communicatively connected to the controller, is used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The door lock execution module is communicatively connected to the controller and is used to perform door opening restriction actions on the vehicle door lock; The controller is configured to acquire the images collected by the cabin perception module; if it is determined based on the images that the occupant intends to open the door, acquire the door opening field of view; control the environmental perception module to acquire the motion state information within the door opening field of view; based on the motion state information, estimate the collision time between the moving target and the vehicle door; and based on the collision time, control the door lock execution module to perform the door opening restriction action.

[0157] In its specific implementation, the vehicle includes a door opening collision avoidance system based on in-cabin multimodal intent recognition and dynamic adjustment of the electronic rearview mirror field of view. This system shares the same concept as the aforementioned vehicle door opening collision avoidance control method, which is executed on the basis of this system. The system architecture topology is shown in Table 1 below: Table 1

[0158] In this embodiment, the OMS actively senses the occupant's line of sight and skeletal trajectory, completing intent recognition before the occupant actually touches and opens the car door. This provides valuable computational window time for subsequent image matrix transformation, target detection, and actuator response, improving the collision avoidance success rate. It eliminates the need for costly and easily worn traditional mechanical rearview mirror physical rotation mechanisms. By dynamically reconstructing the ROI cropping matrix of the CMS high-definition image sensor in a static state and fusing AVM surround-view pixels, the software layer achieves a switch from a narrow driving angle to an ultra-wide door opening angle, resulting in a low-cost and highly reliable solution. It abandons the binary, crude intervention of related technologies that either don't lock or hard-lock. By controlling the electromagnetic damping of the E-Latch, the handle becomes heavier during the yellow warning phase, transforming external danger into a direct tactile sensation for the passenger. This gradual warning avoids causing occupant panic while providing a high degree of collision avoidance alerts, and also avoids the mechanical aversion caused by occasional false alarms from the algorithm that hard-lock the door.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0160] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0161] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0163] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0164] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A vehicle door opening collision prevention control method, characterized in that, The method includes: Acquire images of the occupants; If it is determined from the image that the occupant intends to open the door, obtain the door opening field of view; Within the opening field of view, acquire motion state information of moving targets outside the vehicle; Based on the motion state information, the collision time between the moving target and the vehicle door is estimated; Based on the collision time, the vehicle door locks are controlled to perform an opening restriction action.

2. The method according to claim 1, characterized in that, The following steps are used to determine if the occupant intended to open the door: Feature extraction is performed on the image to obtain the occupant's gaze characteristics and / or arm movement characteristics; When the gaze characteristics indicate that the occupant's gaze is directed towards the outer rear of the vehicle and the duration of this gaze exceeds a first preset threshold, it is determined that the occupant intends to open the door; and / or, When the arm movement feature indicates that the end point of the occupant's arm movement trajectory points to the door handle area, it is determined that the occupant has the intention to open the door.

3. The method according to claim 1, characterized in that, The vehicle is equipped with an electronic rearview mirror module; the acquisition of the door opening field of view includes: If it is determined from the image that the occupant intends to open the door, a door opening intention pre-trigger signal is generated; Based on the pre-triggered signal of the door opening intention, the electronic rearview mirror module is controlled to switch the display screen from the driving field of view to the door opening field of view; the driving field of view is the display field of view of the electronic rearview mirror module when the vehicle is in motion; the door opening field of view is greater than the driving field of view.

4. The method according to claim 3, characterized in that, The electronic rearview mirror module includes an exterior camera; controlling the electronic rearview mirror module to switch the display screen from the driving field of view to the door opening field of view includes: Acquire full-frame raw image data captured by the vehicle's external camera; Based on the preset door opening field of view parameters, a wide-view image corresponding to the door opening field of view is cropped from the full-frame original image data; The wide-angle image is output to the electronic rearview mirror module for display.

5. The method according to claim 4, characterized in that, The vehicle is equipped with a surround view imaging module; the method further includes: Acquire the lateral surround view image provided by the surround view image module; The wide-view image and the side-view image are fused to obtain a fused image; The fused image is output to the electronic rearview mirror module for display, replacing the wide-angle image.

6. The method according to claim 5, characterized in that, The step of fusing the wide-view image with the side-view image includes: Extract feature points from the wide-view image and the side-view image; The feature points of the wide-view image are matched with the feature points of the side-view image to obtain the matching result; Based on the matching results, calculate the homography transformation matrix; Based on the homography transformation matrix, the lateral panoramic image is projected onto the coordinate system of the wide-view image; Image fusion is performed in overlapping areas to fill in the lateral blind spots.

7. The method according to claim 1, characterized in that, The step of acquiring motion state information of a moving target outside the vehicle within the door opening field of view includes: The moving target is identified within the image area corresponding to the opening field of view. The relative distance, relative speed, and relative acceleration of the moving target relative to the vehicle are obtained as the motion state information.

8. The method according to claim 7, characterized in that, The step of estimating the collision time between the moving target and the vehicle door based on the motion state information includes: Calculate the initial collision time based on the relative distance and the relative velocity; The motion pattern of the moving target is determined based on the relative acceleration. When the relative acceleration determines that the moving target is in a state of uniform speed or accelerated approach, the initial collision time is taken as the collision time. When the moving target is determined to be in a deceleration state based on the relative acceleration, if it is predicted that the moving target can come to a complete stop before reaching the door position, the initial collision time is positively gained to obtain the corrected collision time.

9. The method according to claim 1, characterized in that, The step of controlling the vehicle door locks to perform door opening restriction actions based on the collision time includes: When the collision time exceeds the second preset threshold, the door lock is controlled to maintain a basic damping state to allow the door to open freely; When the collision time does not exceed the second preset threshold but is greater than the third preset threshold, the door lock is controlled to apply a first-level damping; the first-level damping is greater than the basic damping, and the first-level damping is used to prompt the occupant to apply more force to open the door; the second preset threshold is greater than the third preset threshold; When the collision time does not exceed the third preset threshold, the door lock is controlled to apply a second level of damping and perform a full lock action; the second level of damping is greater than the first level of damping, and the second level of damping is used to forcibly prevent the door from opening.

10. A vehicle door opening collision avoidance control system, characterized in that, The system includes: The acquisition module is used to acquire images of the occupants; The acquisition module is further configured to acquire the door opening field of view when it is determined from the image that the occupant intends to open the door; The acquisition module is also used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The calculation module is used to estimate the collision time between the moving target and the vehicle door based on the motion state information; The control module is used to control the vehicle door locks to perform door opening restriction actions based on the collision time.

11. A vehicle, characterized in that, The vehicles include: Controller; The cabin sensing module is communicatively connected to the controller and is used to collect images of the occupants; An environmental perception module, which is communicatively connected to the controller, is used to acquire motion state information of moving targets outside the vehicle within the door opening field of view. The door lock execution module is communicatively connected to the controller and is used to perform door opening restriction actions on the vehicle door lock; The controller is configured to acquire the images collected by the cabin perception module; if it is determined based on the images that the occupant intends to open the door, acquire the door opening field of view; control the environmental perception module to acquire the motion state information within the door opening field of view; based on the motion state information, estimate the collision time between the moving target and the vehicle door; and based on the collision time, control the door lock execution module to perform the door opening restriction action.

Images