Vehicle door anti-collision method, device, system and equipment, medium and vehicle

Abstract

The invention provides a vehicle door anti-collision method which comprises the steps that when a preset starting condition is met, data of a sensor outside a vehicle and data of a sensor inside the vehicle are collected, and the sensor inside the vehicle comprises a camera inside the vehicle and a TOF radar; detecting whether the motion trail of the moving target around the vehicle coincides with the vehicle door opening and closing area or not based on the data of the sensor outside the vehicle, and if yes, calculating the possible collision time of the moving target and the vehicle door; detecting whether an in-vehicle passenger has an intention of opening a door or not based on the in-vehicle sensor data; and performing a warning operation based on the possible collision time when it is detected that the passenger in the vehicle has the vehicle door opening intention.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a vehicle door anti-collision method, device, system, equipment, medium and vehicle. Background Art

[0002] As cars become more popular and roads become more crowded, assisted driving functions are becoming more and more popular. Using on-board millimeter-wave radar or cameras to detect and judge vehicles or pedestrians from behind and to provide anti-collision warnings when parking and opening doors is an essential application. However, the current warning is not advanced enough, and reminders and warnings are often not given in time. Sometimes, when the warning is given, the passenger or driver has already opened the door, resulting in a collision.

[0003] How to identify the passenger or driver’s intention to open the door in advance, intervene earlier and provide protection is the key to improving the system. Summary of the invention

[0004] The present invention is completed to solve the above-mentioned problems, and its purpose is to provide a vehicle door anti-collision method, device, system, equipment, medium and vehicle, which can improve the timeliness and accuracy of vehicle door anti-collision warning.

[0005] According to one aspect of the present invention, a door anti-collision method is provided, comprising the following steps: when a preset start condition is met, collecting external sensor data and internal sensor data, the internal sensor including an internal camera and a TOF radar; detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the external sensor data, and if so, calculating the possible collision time between the moving target and the door; detecting whether an occupant in the vehicle has the intention to open the door based on the internal sensor data; and performing a warning operation based on the possible collision time when it is detected that the occupant in the vehicle has the intention to open the door.

[0006] Preferably, detecting whether an occupant in the vehicle has the intention to open the door based on the in-vehicle sensor data further includes: separately collecting the image data and TOF radar data captured by the in-vehicle camera; analyzing the image data, extracting the body feature points of each occupant in the vehicle, and tracking them; constructing the radar three-dimensional data of each occupant in the vehicle based on the TOF radar data; and fusing the body feature points and the radar three-dimensional data of each occupant in the vehicle to determine whether the occupant in the vehicle has the intention to open the door.

[0007] Preferably, the human body feature points include: face, pupils and hands.

[0008] Preferably, the body feature points of each occupant in the vehicle and the radar three-dimensional data are integrated to determine whether the occupant in the vehicle has the intention to open the door. The relative coordinates of the palm of each occupant in the vehicle relative to the door switch position, the gesture movement trajectory and the facial rotation trajectory are calculated based on the radar three-dimensional data. Whether the occupant in the vehicle has the intention to open the door is determined based on the body feature points of each occupant in the vehicle, the relative coordinates of the palm of each occupant in the vehicle, the gesture movement trajectory and the facial rotation trajectory.

[0009] Preferably, detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the off-vehicle sensor data, and if so, calculating the possible collision time between the moving target and the door, further includes: predicting the motion trajectory of a moving target around the vehicle based on the off-vehicle sensor data; determining whether the motion trajectory coincides with the door opening and closing area; and calculating the possible collision time between the moving target and the door when the motion trajectory coincides with the door opening and closing area.

[0010] Preferably, the external vehicle sensor includes a millimeter wave radar and an external vehicle camera.

[0011] Preferably, each door of the cockpit of the vehicle is provided with the TOF radar.

[0012] Preferably, the vehicle door anti-collision method further comprises the following steps: after collecting the external vehicle sensor data, firstly acquiring the scene information around the vehicle based on the external vehicle sensor data, and only executing the subsequent steps when the vehicle scene is a preset scene.

[0013] Preferably, when it is detected that the occupant in the vehicle has the intention to open the door, a warning operation is performed based on the possible collision time. If the possible collision time is less than a first threshold and greater than or equal to a second threshold, a first-level warning is initiated; if the possible collision time is less than the second threshold, a second-level warning is initiated, and the corresponding door is controlled to be automatically locked.

[0014] Preferably, the preset starting condition is that the vehicle is close to stopping.

[0015] According to another aspect of the present invention, there is provided a door anti-collision device, comprising: an acquisition module, for acquiring external sensor data and internal sensor data when a preset start condition is met, the internal sensor comprising an internal camera and a TOF radar; a calculation module, for detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the external sensor data, and if so, calculating the possible collision time between the moving target and the door; a detection module, for detecting whether an occupant in the vehicle intends to open the door based on the internal sensor data; and a warning module, for performing a warning operation based on the possible collision time when it is detected that the occupant in the vehicle intends to open the door.

[0016] According to another aspect of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the method described in the above aspect.

[0017] According to another aspect of the present invention, a computer storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded by a processor and executes the method according to the above aspect of claim 1. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural block diagram of a vehicle door anti-collision system provided by an embodiment of the present invention.

[0019] Figure 2 It is a flow chart of a vehicle door anti-collision method provided by an embodiment of the present invention.

[0020] Figure 3 This is a flow chart of detecting whether a vehicle occupant intends to open a vehicle door in a vehicle door anti-collision method provided in an embodiment of the present invention.

[0021] Figure 4 It is a structural block diagram of a vehicle door anti-collision device provided in an embodiment of the present invention.

[0022] Figure 5 It is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0024] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0025] The embodiments described herein may be described with reference to plan views and / or cross-sectional views by means of idealized schematic diagrams of the present disclosure. Therefore, the example illustrations may be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the accompanying drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the accompanying drawings have schematic properties, and the shapes of the regions shown in the figures illustrate the specific shapes of the regions of the elements, but are not intended to be limiting.

[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0027] Figure 1 1 is a structural block diagram of a vehicle door anti-collision system 100 provided in an embodiment of the present invention.

[0028] The door anti-collision system 100 involved in this embodiment is composed of an external sensor 102, an internal sensor, a meter 108, a speaker 109, an electronic control unit 101, a body controller 110 and electric door locks (left front electric door lock 111, left rear electric door lock 112, right front electric door lock 113, right rear electric door lock 114), etc. Among them, the internal sensor includes an internal camera 103 and a TOF radar. In this embodiment, the TOF radar includes a left front door TOF radar 104, a right front door TOF radar 105, a left rear door TOF radar 106, and a right rear door TOF radar 107. The data of the external sensor 102 and the internal camera 103 are connected to the electronic control unit 101 through LVDS or Ethernet, and the TOF radar data is connected to the electronic control unit 101 through LIN or private CAN. The body controller 110, the meter 108, etc. communicate with the electronic control unit 101 through the body bus.

[0029] Among them, the in-vehicle camera 103 is used to provide in-vehicle visual detection, the TOF radar is used to detect the precise position of the arm, the out-of-vehicle sensor 102 is used for environmental detection and moving target detection, the electronic control unit 101 is used to receive multi-sensor signals, fuse and calculate, and give control signals, the instrument 108 is used to give a warning for opening the door and control the speaker 109 to issue a warning, and the body controller 110 is used to control the locking or unlocking of the four electric door locks. In addition, the door anti-collision system 100 also obtains the vehicle's overall motion signal via the vehicle body bus.

[0030] Figure 2 It is a flowchart of the door anti-collision method provided by an embodiment of the present invention. This specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, it may include more or fewer operation steps. The step order listed in the embodiment is only one way among the execution orders of numerous steps, and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order shown in the embodiment or the drawing, or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing).

[0031] As Figure 2 shown, the door anti-collision method provided in this embodiment includes the following steps.

[0032] S101: When the preset start condition is met, collect out-of-vehicle sensor data and in-vehicle sensor data, and the in-vehicle sensors include an in-vehicle camera and a TOF radar.

[0033] Here, the preset start condition can be that the vehicle is approaching a stop. For example, the wheel speed signal and the gear signal can be obtained from the vehicle body bus. When the wheel speed signal is close to zero or the gear information is in the parking gear, it can be determined that the vehicle is approaching a stop.

[0034] The out-of-vehicle sensor can be an out-of-vehicle camera, which is used to detect the environment around the vehicle and moving targets around the vehicle, and the out-of-vehicle sensor is configured to meet the field of view (FOV: Field of View) requirements for detecting the side and rear of the vehicle on both the left and right sides of the vehicle. In some alternative embodiments, the out-of-vehicle sensor can include an out-of-vehicle camera and a millimeter-wave radar. By using the detection method that fuses the out-of-vehicle camera and the millimeter-wave radar, the detection reliability can be further improved. In addition, the out-of-vehicle sensor can also include sensors such as lidar.

[0035] The in-vehicle sensor includes an in-vehicle camera 103 and a TOF radar. Specifically, a TOF radar is provided for each door of the vehicle cockpit. As described above, the present embodiment includes a left front door TOF radar 103, a right front door TOF radar 104, a left rear door TOF radar 106, and a right rear door TOF radar 107. Further, the TOF radar can be installed on the inner side of the door panel, with the emitting surface perpendicular to the inner wall of the inner door panel and as close as possible to the door electronic switch position.

[0036] When the preset start conditions are met, the external and internal sensor data are acquired.

[0037] S102: Detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the external sensor data, and if so, calculating the possible collision time between the moving target and the door.

[0038] This step specifically further includes:

[0039] 1) Predict the motion trajectory of mobile targets around the vehicle based on off-vehicle sensor data. Specifically, confirm whether there are mobile targets around the vehicle based on off-vehicle sensor data, and predict the motion trajectory of the mobile target. Mobile targets around the vehicle refer to, for example, objects, animals, pedestrians, and vehicles (e.g., bicycles, two-wheeled vehicles, and four-wheeled vehicles) that are within a specified distance from the vehicle. Here, the prediction of the motion trajectory of mobile targets belongs to the prior art and will not be elaborated here.

[0040] 2) Determine whether the moving trajectory of the moving target coincides with the door opening and closing area. Assume that the moving trajectory of the moving target coincides with the left rear door opening and closing area.

[0041] 3) When the moving target's trajectory coincides with the door opening and closing area, calculate the possible collision time TTC (TTC: Time to collision) between the moving target and the door (left rear door). The possible collision time TTC can be calculated based on the distance between the moving target and the left rear door opening and closing area and the speed of the moving target relative to the left rear door.

[0042] S103: Detecting whether the occupant in the vehicle intends to open the door based on the in-vehicle sensor data.

[0043] Figure 3 FIG. 1 is a flow chart of detecting whether a vehicle occupant intends to open the door in a vehicle door anti-collision method provided by an embodiment of the present invention. Figure 3 As shown, this step specifically further includes:

[0044] S1031: Collect image data and TOF radar data captured by the in-vehicle camera 103 respectively.

[0045] S1032: Analyze the image data captured by the in-vehicle camera 103, extract the body feature points of each in-vehicle occupant, and track them. Here, the in-vehicle occupants include the driver and the passengers.

[0046] Here, the human body feature points include: face, pupils and hands. Specifically, the face, pupils and hand feature points can be obtained by analyzing the image data, and the face, pupils and hands can be tracked. In some optional embodiments, the in-car camera 103 uses a binocular camera and is arranged in the middle of the inner top and the middle of the front sunroof, so that there is no obstruction between the camera FOV range and the passengers in the car.

[0047] S1033: Construct radar three-dimensional data of each vehicle occupant based on the TOF radar data.

[0048] S1034: Integrate the body feature points of each occupant in the vehicle and the radar three-dimensional data to determine whether the occupant in the vehicle intends to open the door.

[0049] Specifically, firstly, the relative coordinates of the palm of each occupant in the car relative to the switch position of the door (the left rear door in this embodiment), the gesture movement trajectory and the facial rotation trajectory can be calculated based on the radar three-dimensional data. Then, based on the body feature points of each occupant in the car, the relative coordinates of the palm of each occupant in the car, the gesture movement trajectory and the facial rotation trajectory, it is judged whether the occupant in the car has the intention to open the left rear door.

[0050] Here, the judgment can be mainly made based on the relative coordinates of the palms of each occupant in the car and the gesture movement trajectory, while being supplemented by the facial rotation trajectory and pupil direction.

[0051] S104: When it is detected that the vehicle occupant intends to open the vehicle door, a warning operation is performed based on the possible collision time.

[0052] Specifically, if the possible collision time TTC is less than the first threshold and greater than or equal to the second threshold, the first-level warning is activated; if the possible collision time is less than the second threshold, the second-level warning is activated and the corresponding door is controlled to be automatically locked.

[0053] The first threshold and the second threshold here can be set as needed, for example, the first threshold is set to 8s, and the second threshold is set to 5s.

[0054] In some optional embodiments, the door anti-collision method involved in the present application may further include the following steps: after collecting the external sensor data (i.e., after S101), firstly, the scene information around the vehicle is obtained based on the external sensor data, and the subsequent steps are performed only when the vehicle scene is a preset scene. For example, the preset scene is an urban road scene or a parking garage aisle scene, that is, only when the acquired vehicle scene is an urban road scene or a parking garage aisle scene will steps S102, S103, and S104 be continued.

[0055] In addition, the above-mentioned vehicle door anti-collision method can be continuously executed at a certain period (for example, 1 second) until it is detected based on the in-vehicle sensor that there is no person left in the vehicle.

[0056] According to the vehicle door anti-collision method involved in the present application, it is possible to analyze and track the behavior of passengers or drivers, and predict the driver's or passenger's intention to open the door in advance, thereby improving the timeliness and accuracy of the vehicle door anti-collision warning.

[0057] Figure 4 It is a structural block diagram of a vehicle door anti-collision device provided in an embodiment of the present invention.

[0058] like Figure 4 As shown, the vehicle door anti-collision device 200 includes: a collection module 201 , a calculation module 202 , a detection module 203 and a warning module 204 .

[0059] Among them, the acquisition module 201 is used to collect external sensor data and internal sensor data when the preset start conditions are met. The internal sensor includes an internal camera and a TOF radar. The calculation module 202 is used to detect whether the motion trajectory of the moving target around the vehicle coincides with the door opening and closing area based on the external sensor data, and if so, calculate the possible collision time between the moving target and the door. The detection module 203 is used to detect whether the occupant in the vehicle intends to open the door based on the internal sensor data. The warning module 204 is used to perform a warning operation based on the possible collision time when it is detected that the occupant in the vehicle has the intention to open the door.

[0060] The embodiments of the present invention provide a vehicle door anti-collision method and a vehicle door anti-collision device based on the same concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0061] Figure 5 is a structural block diagram of an electronic device provided by an embodiment of the present invention. Figure 5As shown, the present invention also provides an electronic device 300, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded by the processor and executes the method described in the above embodiment.

[0062] The present invention also provides a computer storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the method described in the above embodiment.

[0063] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0064] Those skilled in the art should be able to appreciate that the modules, units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0065] Although the present invention has been described with reference to the current specific embodiments, those skilled in the art should recognize that the scope of the invention involved in the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

Claims

1. A door anti-collision method, It is characterized in that The steps include: When the preset start conditions are met, the data of the external sensors and the internal sensors are collected, and the internal sensors include the internal camera and the TOF radar; Based on the data from the external vehicle sensor, detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area, and if so, calculating the possible collision time between the moving target and the door; Detecting whether a passenger in the vehicle intends to open the door based on in-vehicle sensor data; as well as When it is detected that the vehicle occupant intends to open the door, a warning operation is performed based on the possible collision time.

2. The vehicle door anti-collision method according to claim 1, It is characterized in that The method further comprises: detecting whether an occupant in the vehicle intends to open the door based on the in-vehicle sensor data; The image data and TOF radar data captured by the in-vehicle camera are collected respectively; Analyzing the image data, extracting the body feature points of each occupant in the vehicle, and tracking them; Constructing radar three-dimensional data of each vehicle occupant based on the TOF radar data; and The human body feature points of each vehicle occupant and the radar three-dimensional data are integrated to determine whether the vehicle occupant intends to open the door.

3. The vehicle door anti-collision method according to claim 2, It is characterized in that The human body feature points include: face, pupils and hands.

4. The vehicle door anti-collision method as claimed in claim 3, It is characterized in that By integrating the human body feature points of each occupant in the vehicle and the radar three-dimensional data, it is determined whether the occupant in the vehicle intends to open the door. Based on the radar three-dimensional data, the relative coordinates of the palm of each occupant in the vehicle relative to the door switch position, the gesture movement trajectory and the facial rotation trajectory are calculated, It is determined whether the occupants in the vehicle have the intention to open the door based on the human body feature points of each occupant in the vehicle, the relative coordinates of the palms of each occupant in the vehicle, the gesture movement trajectory and the facial rotation trajectory.

5. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that Detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the external sensor data, and if so, calculating the possible collision time between the moving target and the door, further includes: Predict the motion trajectory of moving objects around the vehicle based on external sensor data; Determining whether the motion trajectory coincides with a door opening and closing area; and When the motion trajectory coincides with the door opening and closing area, a possible collision time between the moving object and the door is calculated.

6. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that The external vehicle sensors include millimeter wave radar and external vehicle cameras.

7. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that Each door of the cockpit of the vehicle is provided with the TOF radar.

8. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that Further comprising the steps of: After collecting the off-vehicle sensor data, the scene information around the vehicle is first obtained based on the off-vehicle sensor data, and subsequent steps are performed only when the vehicle scene is a preset scene.

9. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that When it is detected that the occupant in the vehicle intends to open the door, a warning operation is performed based on the possible collision time. If the possible collision time is less than the first threshold and greater than or equal to the second threshold, a first-level warning is initiated; If the possible collision time is less than a second threshold, a secondary warning is activated, and the corresponding vehicle door is controlled to be automatically locked.

10. The vehicle door anti-collision method according to any one of claims 1 to 4, It is characterized in that The preset start condition is that the vehicle is close to a stop.

11. A vehicle door anti-collision device, It is characterized in that include: A collection module, used to collect external sensor data and internal sensor data when a preset start condition is met, wherein the internal sensor includes an internal camera and a TOF radar; A calculation module, used for detecting whether the motion trajectory of a moving target around the vehicle coincides with the door opening and closing area based on the data of the external sensor, and if so, calculating the possible collision time between the moving target and the door; A detection module, used for detecting whether a passenger in the vehicle intends to open the door based on the in-vehicle sensor data; as well as The warning module is used to perform a warning operation based on the possible collision time when it is detected that the occupant in the vehicle has the intention to open the door.

12. An electronic device, It is characterized in that The electronic device comprises a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the method according to any one of claims 1 to 10.

13. A computer storage medium, It is characterized in that The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded by a processor and executes the method according to any one of claims 1 to 10.

Images