Vehicle door obstacle avoidance control method and device, vehicle and storage medium

Abstract

The invention relates to a vehicle door obstacle avoidance control method and device, a vehicle and a storage medium. The method comprises the steps that when an automatic door opening request is received, if an ultrasonic radar of a vehicle door detects an obstacle, image data collected by a side view camera are obtained; the image data comprises obstacles; according to the image data, identifying an attribute type and an action intention of the obstacle; and when the attribute type is a character type and the action intention is a door opening intention, the vehicle door is controlled to be opened. According to the scheme, the problem that the user fails to open the door due to the fact that the user is detected as an obstacle by the ultrasonic radar of the vehicle door due to the door opening action of the user can be solved, the obstacle avoidance strategy can be optimized, and the riding experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle door obstacle avoidance control method, device and storage medium. Background Art

[0002] With the development of the vehicle industry, especially the advancement of electrification, the function of automatic door opening and closing is becoming more and more popular, which can bring great convenience, experience and sense of technology to users. In the process of automatic door opening, in order to avoid safety accidents caused by the door hitting obstacles around the vehicle, an ultrasonic radar can be installed near the outer contour of the door. The door ultrasonic radar can detect obstacles to achieve obstacle avoidance.

[0003] In the related technology, the position of the outside door handle and the position of the door ultrasonic radar are on the same straight line in the vertical direction. Therefore, when the user touches the outside handle capacitive switch or pulls the outside handle to open the door, it is easily detected as an obstacle by the door ultrasonic radar, thereby triggering the obstacle avoidance function, resulting in the user being unable to open the door outside the car, affecting the user's riding experience. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a vehicle door obstacle avoidance control method, device, vehicle and storage medium, which can solve the problem that the user's door opening action causes the user to be detected as an obstacle by the vehicle door's ultrasonic radar, resulting in the user's failure to open the door, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0005] A first aspect of the present application provides a door obstacle avoidance control method, comprising: When receiving an automatic door opening request, if the ultrasonic radar of the vehicle door detects an obstacle, image data collected by the side view camera is obtained; the image data includes the obstacle; According to the image data, identifying the attribute type and action intention of the obstacle; When the attribute type is a person type, and the action intention is a door opening intention, the vehicle door is controlled to open.

[0006] In one embodiment, when the attribute type is a person type and the action intention is a door opening intention, controlling the vehicle door to open includes: When the attribute type is a person type and the action intention is a door opening intention, obtaining a target distance between the obstacle and the ultrasonic radar; The opening angle of the door is controlled according to the target distance.

[0007] In one embodiment, controlling the opening angle of the vehicle door according to the target distance includes: When the target distance is less than or equal to the preset distance, it is determined that the obstacle is located within the movement trajectory area of ​​the vehicle door, the vehicle door is controlled to open to a first angle, and a first prompt message is sent to the obstacle; the first prompt message is used to instruct the obstacle to stay away from the vehicle door; When the target distance is greater than the preset distance, it is determined that the obstacle is outside the movement track area of ​​the door, and the door is controlled to open to a second angle; wherein the second angle is greater than the first angle.

[0008] In one embodiment, the identifying the attribute type and action intention of the obstacle according to the image data includes: Performing target detection on the image data to obtain the attribute type of the obstacle; When the attribute type is a person type, a human posture estimation is performed on the obstacle to obtain the action intention of the obstacle.

[0009] In one embodiment, the image data includes a plurality of image frames captured by the side-view camera within a reception time before and after the automatic door opening request; When the attribute type is a person type, estimating the human body posture of the obstacle to obtain the action intention of the obstacle includes: When the attribute type is a person type, detecting each human body joint point of the obstacle in each image frame respectively; In each image frame, each of the human body joints is sequentially connected according to a preset order to obtain target posture information of the obstacle in each image frame; Determining the posture change information of the obstacle based on the target posture information of the obstacle in each image frame; The movement intention of the obstacle is identified according to the posture change information.

[0010] In one embodiment, the method further comprises: When the attribute type is an object type, and / or the action intention is not an intention to open the door, the vehicle door is controlled not to open, and a second prompt message is sent to the user; wherein, the second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the vehicle door.

[0011] In one embodiment, the method further comprises: If the ultrasonic radar does not detect an obstacle, the vehicle door is controlled to open to a second angle.

[0012] A second aspect of the present application provides a door obstacle avoidance control device, comprising: An image data acquisition module, configured to acquire image data collected by a side-view camera when receiving an automatic door opening request and if the ultrasonic radar of the vehicle door detects an obstacle; the image data includes the obstacle; An auxiliary information recognition module, used to identify the attribute type and action intention of the obstacle according to the image data; The first control module is used to control the vehicle door to open when the attribute type is a person type and the action intention is a door opening intention.

[0013] A third aspect of the present application provides a vehicle, comprising: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of a vehicle, the processor is caused to execute the method as described above.

[0015] The technical solution provided by this application may have the following beneficial effects: The solution provided by the present application is that when an automatic door opening request is received, if the ultrasonic radar of the car door detects an obstacle, the image data collected by the side-view camera is obtained; the image data contains the obstacle; based on the image data, the attribute type and action intention of the obstacle are identified; when the attribute type is a person type, and the action intention is the intention to open the door, the car door is controlled to open. The present application can realize visually assisted identification of obstacle types and obstacle intentions with the help of side-view cameras. When it is determined based on the obstacle type and obstacle intention that the user is detected as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, the car door can be controlled to open, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0018] Figure 1 is a schematic diagram of the positional relationship between the exterior door handle and the door ultrasonic radar shown in an embodiment of the present application; Figure 2 is a flow chart of a vehicle door obstacle avoidance control method shown in an embodiment of the present application; Figure 3 is a schematic diagram of communication between various modules involved in the door obstacle avoidance control method shown in an embodiment of the present application; Figure 4 is another flow chart of the door obstacle avoidance control method shown in the embodiment of the present application; Figure 5 is a flow chart of door obstacle avoidance control shown in an embodiment of the present application; Figure 6 is a structural schematic diagram of a vehicle door obstacle avoidance control device shown in an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of a vehicle shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0022] In the related art, in order to ensure that the door ultrasonic radar has a sufficiently large detection area, the installation angle of the door ultrasonic radar is generally 60° vertically and 100° horizontally, and it can cover the movement trajectory area of ​​the door. For example, the front door Figure 1As shown, the ultrasonic radar needs to be installed near the outer contour of the door (near the B-pillar). Due to the shape of the door handle and the need for users to manually operate the door when the electric door does not work, the outer door handle is generally arranged near the outer contour of the door (near the B-pillar). Therefore, the position of the outer door handle and the position of the door ultrasonic radar are in the same straight line in the vertical direction. When the user touches the outer handle capacitive switch or pulls the outer handle to open the door, the vehicle inputs an automatic door opening request signal. Since the user is near the door and the user is in the FOV (Field Of View) area, the door ultrasonic radar will feedback obstacle information, thereby triggering the obstacle avoidance function, causing the user to fail to open the door, affecting the user's riding experience.

[0023] In response to the above problems, an embodiment of the present application provides a door obstacle avoidance control method. With the help of a side-view camera, visually assisted identification of obstacle types and obstacle intentions can be achieved. When it is determined based on the obstacle type and obstacle intention that the user is detected as an obstacle by the door's ultrasonic radar due to the user's door opening action, the door can be controlled to open, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0024] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0025] Figure 2 It is a flow chart of the door obstacle avoidance control method shown in the embodiment of the present application.

[0026] See also Figure 2 The door obstacle avoidance control method of the present application includes: S210, when receiving the automatic door opening request, if the ultrasonic radar of the vehicle door detects an obstacle, the image data collected by the side view camera is obtained; the image data includes the obstacle.

[0027] The execution subject of the vehicle door obstacle avoidance control may be an automatic door control module (or electric door control module) mounted on the vehicle. Figure 3 As shown, the automatic door control module is connected to the door state detection module based on CAN (Controller Area Network). The door state detection module can detect the state of each door, for example, it can detect whether the outer handle switch of each door is triggered, or detect whether the outer handle capacitor switch of each door is triggered. When it is detected that the outer handle switch or the outer handle capacitor switch of any door is triggered, the door state detection module can input an automatic door opening request, so that the automatic door control module can receive the automatic door opening request input by the door state detection module.

[0028] In addition to triggering the outer handle switch or outer handle capacitance switch of the vehicle door, the input of the automatic door opening request can also be realized by triggering the vehicle key or a mobile terminal (such as a mobile phone terminal). For example, when the user triggers the vehicle key to open the door, the automatic door control module can receive the automatic door opening request input by the vehicle key. For example, when the user triggers the mobile phone terminal to open the door, the automatic door control module can receive the automatic door opening request input by the mobile phone terminal, which is not limited in the embodiments of the present application.

[0029] like Figure 3 As shown, the automatic door control module is connected to the ultrasonic radars on the left and right sides based on LIN (Local Interconnect Network, a low-cost serial communication network). Since the detection range of the ultrasonic radar is large enough, the front doors (the main driver's door and the co-driver's door) can be equipped with ultrasonic radars respectively, and the rear doors (the rear left door and the rear right door) do not need to be equipped with ultrasonic radars. Of course, each door can also be equipped with an ultrasonic radar separately, and this embodiment of the application does not limit this.

[0030] Ultrasonic radar can emit radar waves to the outside. Since the detection range of the ultrasonic radar of the car door can cover the movement trajectory of the car door, if there is an obstacle in the movement trajectory of the car door, the ultrasonic radar can receive the echo of the radar wave reflected by the obstacle; if there is no obstacle in the movement trajectory of the car door, the ultrasonic radar cannot receive the echo of the radar wave reflected by the obstacle.

[0031] When receiving an automatic door opening request, the automatic door control module can send a first acquisition request to the ultrasonic radar of the vehicle door. The first acquisition request is used to request the acquisition of detection data collected by the ultrasonic radar of the vehicle door. Then the automatic door control module can determine whether the detection data contains obstacle information. If the detection data contains obstacle information, it means that the ultrasonic radar of the vehicle door detects an obstacle; if the detection data does not contain obstacle information, it means that the ultrasonic radar of the vehicle door does not detect an obstacle.

[0032] like Figure 3 As shown, the automatic door control module is connected to the XPU (eXtensible Processing Unit) based on the CAN network, and the XPU is connected to the AVM (Around View Monitor) based on the LVDS (Low-Voltage Differential Signaling) network. The AVM may include side view cameras on the left and right sides, wherein the side view cameras on the left and right sides may be configured at the rearview mirror position.

[0033] If the ultrasonic radar of the car door detects an obstacle, the automatic door control module can send a second acquisition request to the XPU, and the XPU forwards the second acquisition request to the AVM. The second acquisition request is used to request the acquisition of image data collected by the side-view camera. Since the side-view camera is configured at the rearview mirror position, the collection range of the side-view camera overlaps with the detection range of the ultrasonic radar. Therefore, if the ultrasonic radar of the car door detects an obstacle, the image data collected by the side-view camera includes the obstacle.

[0034] S220, identifying the attribute type and action intention of the obstacle according to the image data.

[0035] Since ultrasonic radar cannot identify the type of obstacle or the intention of the obstacle, it is easy for the user to be detected as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, thereby triggering the obstacle avoidance function and causing the user to fail to open the door.

[0036] In this regard, the embodiment of the present application can realize visual assisted identification of obstacle types and obstacle intentions by means of a side-view camera. Specifically, the XPU has a pre-trained YOLO model built in. YOLO (You Only Look Once) is a target detection algorithm whose core idea is to simplify the target detection task into a regression problem. Therefore, the automatic door control module can call the pre-trained target detection model (YOLO model) to perform recognition and processing on the image data through the target detection model to obtain a first recognition result and a second recognition result. The first recognition result can include the attribute type of the obstacle, and the second recognition result can include the action intention of the obstacle.

[0037] S230, when the attribute type is a person type and the action intention is a door opening intention, control the vehicle door to open.

[0038] The attribute type may include any of the person type and the object type, and the action intention may include any of the door opening intention and the non-door opening intention, wherein the non-door opening intention may refer to other intentions except the door opening intention, for example, the non-door opening intention may include the passing intention, the standing intention, the parking intention, etc.

[0039] The automatic door control module can first determine whether the attribute type of the obstacle is a person type, and then determine whether the action intention of the obstacle is an intention to open the door.

[0040] In one example, when the attribute type of the obstacle is a person type, it means that the obstacle detected by the ultrasonic radar may be a user (car owner) or a pedestrian, so the automatic door control module can further determine whether the obstacle's action intention is to open the door. When the obstacle's action intention is to open the door, it means that the obstacle detected by the ultrasonic radar is the user, and the user's door opening action causes the user to be detected as an obstacle by the ultrasonic radar of the car door, so the automatic door control module can control the car door to open.

[0041] In another example, when the attribute type of the obstacle is a person type, it means that the obstacle detected by the ultrasonic radar may be a user (car owner) or a pedestrian, so the automatic door control module can further determine whether the obstacle's action intention is to open the door. When the obstacle's action intention is not to open the door, for example, the obstacle's action intention is to pass by, it means that the obstacle detected by the ultrasonic radar is a pedestrian. Because the pedestrian passes by the vehicle, the pedestrian is detected as an obstacle by the ultrasonic radar of the car door. In order to avoid the car door hitting the pedestrian, the automatic door control module can control the car door not to open.

[0042] In another example, when the attribute type of the obstacle is an object type, it means that the obstacles detected by the ultrasonic radar are other vehicles, trees, traffic cones, and other objects around the vehicle. Therefore, the automatic door control module does not need to further determine whether the obstacle's action intention is to open the door. In order to avoid the door from hitting the object, the automatic door control module can control the door not to open.

[0043] As can be seen from this example, the solution provided by this application, when receiving an automatic door opening request, if the ultrasonic radar of the car door detects an obstacle, then the image data collected by the side-view camera is obtained; the image data contains the obstacle; according to the image data, the attribute type and action intention of the obstacle are identified; when the attribute type is a person type, and the action intention is the intention to open the door, the car door is controlled to open. This application can realize visual assistance to identify the obstacle type and obstacle intention with the help of the side-view camera. When it is determined based on the obstacle type and obstacle intention that the user is detected as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, the car door can be controlled to open, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0044] Figure 4 It is another flow chart of the vehicle door obstacle avoidance control method shown in the present application.

[0045] See also Figure 4 The door obstacle avoidance control method of the present application includes: S410, when receiving the automatic door opening request, if the ultrasonic radar of the vehicle door detects an obstacle, the image data collected by the side view camera is obtained; the image data includes the obstacle.

[0046] In actual applications, when a user opens a door by triggering an outer handle switch or an outer handle capacitance switch, the automatic door control module can receive an automatic door opening request input by the door state detection module. Alternatively, when a user opens a door by triggering a vehicle key, the automatic door control module can receive an automatic door opening request input by the vehicle key. Alternatively, when a user opens a door by triggering a mobile terminal (such as a mobile phone terminal), the automatic door control module can receive an automatic door opening request input by the mobile terminal.

[0047] When receiving an automatic door opening request, the automatic door control module can send a first acquisition request to the ultrasonic radar of the vehicle door. The first acquisition request is used to request the acquisition of detection data collected by the ultrasonic radar of the vehicle door. Then the automatic door control module can determine whether the detection data contains obstacle information. If the detection data contains obstacle information, it means that the ultrasonic radar of the vehicle door detects an obstacle; if the detection data does not contain obstacle information, it means that the ultrasonic radar of the vehicle door does not detect an obstacle.

[0048] If the ultrasonic radar of the car door detects an obstacle, the automatic door control module can send a second acquisition request to the XPU, and the XPU forwards the second acquisition request to the AVM. The second acquisition request is used to request the acquisition of image data collected by the side-view camera. Since the side-view camera is configured at the rearview mirror position, the collection range of the side-view camera overlaps with the detection range of the ultrasonic radar. Therefore, if the ultrasonic radar of the car door detects an obstacle, the image data collected by the side-view camera includes the obstacle.

[0049] S420: Identify the attribute type and action intention of the obstacle according to the image data.

[0050] The automatic door control module can call a pre-trained target detection model (YOLO model) to perform recognition and processing on the image data through the target detection model to obtain a first recognition result and a second recognition result. The first recognition result can include the attribute type of the obstacle, and the second recognition result can include the action intention of the obstacle.

[0051] In one embodiment, identifying the attribute type and action intention of the obstacle according to the image data may include: Perform target detection on the image data to obtain the attribute type of the obstacle; when the attribute type is a person type, perform human posture estimation on the obstacle to obtain the action intention of the obstacle.

[0052] The automatic door control module can input the image data containing obstacles into the target detection model, and the target detection model can perform target detection on the image data to output the attribute type of the obstacle. Among them, the target detection process can include steps such as image segmentation, feature extraction, bounding box prediction, category prediction and non-maximum suppression. In the image segmentation step, the target detection model can divide the input image data into multiple grids, each grid is responsible for predicting n bounding boxes and the confidence of each bounding box, n is a positive integer, where the confidence represents the probability of the existence of the target object in the bounding box and the accuracy of the bounding box. In the feature extraction step, the target detection model can use a convolutional neural network to extract image features, and the image features may include information such as color, texture, and shape. In the bounding box prediction step, each grid predicts the coordinates (x, y, w, h) and confidence of n bounding boxes based on the extracted features, where (x, y) represents the center coordinates of the bounding box, and (w, h) represents the width and height of the bounding box. In the category prediction step, each grid can predict the category of the target object in each bounding box. In the non-maximum suppression step, after obtaining the prediction results of all grids, the target detection model can remove redundant bounding boxes through the non-maximum suppression algorithm to retain the best prediction results. The target detection model can determine the bounding box belonging to the obstacle from the remaining bounding boxes, and then output the prediction results within the bounding box belonging to the obstacle. The prediction result is the first recognition result, and the first recognition result can include the attribute type of the obstacle.

[0053] When the attribute type is a person type, the automatic door control module can further identify and process the obstacle through the target detection model. For example, the target detection model estimates the human posture of the obstacle belonging to the person type to output a second recognition result, which can include the action intention of the obstacle.

[0054] In one embodiment, the image data includes multiple image frames captured by the side view camera within the receiving time before and after the automatic door opening request; when the attribute type is a person type, the human body posture estimation of the obstacle is performed to obtain the action intention of the obstacle, which may include: When the attribute type is a person type, each human joint point of the obstacle in each image frame is detected respectively; in each image frame, each human joint point is connected in sequence according to a preset order to obtain the target posture information of the obstacle in each image frame; based on the target posture information of the obstacle in each image frame, the posture change information of the obstacle is determined; according to the posture change information, the action intention of the obstacle is identified.

[0055] The image data acquired by the automatic door control module may include multiple image frames captured by the side-view camera within the receiving time before and after the automatic door opening request. For example, if the receiving time of the automatic door opening request is T, then the receiving time before and after the automatic door opening request may be (Tt, T+t), where T is the moment and t is the duration. The automatic door control module may acquire the image data captured by the side-view camera within (Tt, T+t), and the image data includes multiple image frames.

[0056] It should be noted that the data acquisition format of the side-view camera can be image data or video data. If the automatic door control module obtains video data, for example, the automatic door control module obtains video data collected by the side-view camera within (Tt, T+t), the automatic door control module can capture multiple image frames from the video data.

[0057] When the attribute type is a person type, the automatic door control module can detect each human joint point of the obstacle in each image frame respectively through the target detection model. Wherein, the human joint point refers to the point connecting two or more bones, which allows the movement and flexibility of the human body, and the human joint point can include at least one of the head joint point, the neck joint point, the shoulder joint point, the elbow joint point, the wrist joint point, the waist joint point, the hip joint point, the knee joint point, the ankle joint point and the plantar joint point. In each image frame, the automatic door control module can connect each human joint point in sequence according to the preset order through the target detection model, so as to obtain the target posture information of the obstacle in each image frame, wherein the preset order can be set according to the description system of the human body structure and movement, for example, the preset order can be: head joint point → neck joint point → shoulder joint point → elbow joint point → wrist joint point → waist joint point → hip joint point → knee joint point → ankle joint point → plantar joint point. The spatial position of the obstacle in each part of the human body in the current image frame can be analyzed through the target posture information, so based on the target posture information of the obstacle in each image frame, the posture change information of the obstacle can be determined. The movement trajectory, posture and amplitude of the obstacle in the reception time before and after the automatic door opening request can be analyzed through the posture change information. Therefore, the movement intention of the obstacle can be identified based on the posture change information.

[0058] S430, when the attribute type is a person type and the action intention is a door opening intention, obtain the target distance between the obstacle and the ultrasonic radar.

[0059] The attribute type may include any of the person type and the object type, and the action intention may include any of the door opening intention and the non-door opening intention, wherein the non-door opening intention may refer to other intentions except the door opening intention, for example, the non-door opening intention may include the passing intention, the standing intention, the parking intention, etc.

[0060] The automatic door control module can first determine whether the attribute type of the obstacle is a person type, and then determine whether the action intention of the obstacle is an intention to open the door.

[0061] When the attribute type of the obstacle is a person type, it means that the obstacle detected by the ultrasonic radar may be a user (car owner) or a pedestrian, so the automatic door control module can further determine whether the obstacle's action intention is to open the door. When the obstacle's action intention is to open the door, it means that the obstacle detected by the ultrasonic radar is the user. Due to the user's door opening action, the user is detected as an obstacle by the ultrasonic radar of the car door. In order to avoid the car door hitting the user, the automatic door control module can obtain the target distance between the obstacle (user) and the ultrasonic radar.

[0062] Ultrasonic radar emits radar waves and starts timing at the moment of emission. When the radar waves propagate in the air and encounter an obstacle, they will be reflected back immediately. The reflected radar waves can be called echoes. The ultrasonic radar stops timing when it receives the echo. By calculating the time difference between the emission time of the radar wave and the reception time of the echo, the target distance between the obstacle and the ultrasonic radar can be calculated.

[0063] S440, controlling the opening angle of the door according to the target distance.

[0064] In order to prevent the car door from hitting the user, the automatic door control module can accurately control the opening angle of the car door based on the target distance between the obstacle (user) and the ultrasonic radar.

[0065] In one embodiment, controlling the opening angle of the door according to the target distance may include: When the target distance is less than or equal to the preset distance, it is determined that the obstacle is within the movement trajectory area of ​​the door, the door is controlled to open to a first angle, and a first prompt message is sent to the obstacle; the first prompt message is used to instruct the obstacle to stay away from the door; until the target distance is greater than the preset distance, it is determined that the obstacle is outside the movement trajectory area of ​​the door, the door is controlled to open to a second angle; wherein the second angle is greater than the first angle.

[0066] Since the detection range of the ultrasonic radar of the car door can cover the movement trajectory area of ​​the car door, it is possible to judge whether the obstacle is within the movement trajectory area of ​​the car door based on the target distance between the obstacle and the ultrasonic radar. The movement trajectory area of ​​the car door refers to the door opening area, such as the arc area when the car door opens from the minimum angle to the maximum angle.

[0067] When the distance between the obstacle and the ultrasonic radar target is less than or equal to the preset distance, it means that the obstacle is currently located in the movement trajectory of the car door. In order to avoid the car door hitting the user, the automatic door control module can open the door at a small angle and remind the user to stay away so that the door opening action can be completed smoothly. Figure 3 As shown, the automatic door control module is connected to the electric struts and ECU (Electronic Control Unit) on the left and right sides based on the CAN network. When it is determined that the obstacle is located in the movement trajectory area of ​​the car door, the automatic door control module can send a first control request to the electric struts and ECU. The first control request is used to drive the electric struts and ECU to control the car door to open to a first angle. The first angle can be set to a small angle of 5%, so that the car door can be opened without hitting the user. At the same time, the automatic door control module can send a first prompt message to the central control HMI (Human Machine Interface), so that the HMI can send a first prompt message to the obstacle (user) by voice. The first prompt message is used to instruct the obstacle (user) to stay away from the car door.

[0068] After the door is opened to the first angle, the automatic door control module can monitor in real time whether the target distance between the obstacle and the ultrasonic radar is greater than the preset distance. If the target distance between the obstacle and the ultrasonic radar is still less than or equal to the preset distance, the automatic door control module controls the door to stop so that the door remains at the first angle. If the target distance between the obstacle and the ultrasonic radar is greater than the preset distance, it means that the obstacle is currently outside the movement trajectory area of ​​the door, so that the door will not hit the user even if it is opened to the maximum angle. Therefore, the automatic door control module can send a second control request to the electric support rod and ECU. The second control request is used to drive the electric support rod and ECU to control the door to open to the second angle. The second angle can be set to the maximum angle, so that the user can get on the car conveniently, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0069] In one embodiment, the method may further include: When the attribute type is an object type, and / or the action intention is not an intention to open the door, the vehicle door is controlled not to open, and a second prompt message is sent to the user; wherein the second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the vehicle door.

[0070] In one example, when the attribute type of the obstacle is a person type, and the action intention of the obstacle is not an intention to open the door, for example, the action intention of the obstacle is an intention to pass by, it means that the user (car owner) triggers the vehicle key or mobile terminal to open the door outside the detection area of ​​the ultrasonic radar, but a pedestrian passes by the vehicle, causing the pedestrian to be detected as an obstacle by the ultrasonic radar of the car door. In order to avoid the car door hitting the pedestrian, the automatic door control module can control the car door not to open, and can send a second prompt message to the mobile terminal (such as a mobile phone terminal), so that the mobile terminal (such as a mobile phone terminal) can send the second prompt message to the user by voice, and the second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the car door, so that the user can promptly understand that it is not suitable to open the door at present, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0071] In another example, when the attribute type of the obstacle is an object type, it means that the user (car owner) triggers the vehicle key or mobile terminal to open the door outside the detection area of ​​the ultrasonic radar, but there are other vehicles, trees, traffic cones and other objects around the vehicle, causing the ultrasonic radar to detect the obstacle. In order to avoid the car door hitting the object, the automatic door control module can control the car door not to open, and can send a second prompt message to the mobile terminal (such as a mobile phone terminal), so that the mobile terminal (such as a mobile phone terminal) can send the second prompt message to the user by voice. The second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the car door, so that the user can promptly understand that it is not suitable to open the door at present, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0072] In one embodiment, the method may further include: If the ultrasonic radar does not detect an obstacle, the door is controlled to open to the second angle.

[0073] In one example, when receiving an automatic door opening request, if the ultrasonic radar of the door does not detect an obstacle, it means that the user (owner) has opened the door by triggering the vehicle key or mobile terminal outside the detection area of ​​the ultrasonic radar, and there are no pedestrians around the vehicle, and there are no other vehicles, trees, or traffic cones. Therefore, the ultrasonic radar does not detect an obstacle, and the automatic door control module can control the door to open. Specifically, the automatic door control module can send a third control request to the electric support rod and the ECU, and the third control request is used to drive the electric support rod and the ECU to control the door to open to a second angle. The second angle can be set to the maximum angle, so that the user can get on the car conveniently, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0074] As can be seen from this example, the present application can realize visually assisted identification of obstacle types and obstacle intentions with the help of side-view cameras. When it is determined based on the obstacle type and obstacle intention that the user is detected as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, the car door can be controlled to open, thereby optimizing the obstacle avoidance strategy and improving the user's riding experience.

[0075] Furthermore, the present application utilizes a target detection model (YOLO model) to perform a secondary judgment on the user's intention to open the door. When it is determined that the user is detected as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, the door can be opened at a small angle and the user can be reminded to stay away from the car door, thereby successfully completing the door opening action.

[0076] Furthermore, this application analyzes the current user pain points and designs a door opening obstacle avoidance logic strategy. With the help of the current vehicle's original sensors (ultrasonic radar + side-view camera), it can effectively solve the problem of the user's failure to open the door due to the user's door opening action being detected as an obstacle by the car door's ultrasonic radar. The implementation cost is low and the effect is good.

[0077] Figure 5 It is a door obstacle avoidance control flow chart. In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are explained below through the following examples.

[0078] S501, receiving an automatic door opening request; wherein, the input of the automatic door opening request may include triggering an external handle switch or an external handle capacitance switch of the vehicle door, triggering a vehicle key, triggering a mobile terminal, etc.

[0079] S502, determine whether the ultrasonic radar of the vehicle door detects an obstacle; if so, proceed to step S503; if not, proceed to step S512.

[0080] S503, if the ultrasonic radar of the vehicle door detects an obstacle, image data collected by the side view camera is obtained; wherein the image data includes the obstacle.

[0081] S504: Identify the attribute type and action intention of the obstacle according to the image data.

[0082] S505, determining whether the attribute type of the obstacle is a person type; if so, proceeding to step S506; if not, proceeding to step S511.

[0083] S506, if the attribute type of the obstacle is a person type, determine whether the obstacle's action intention is to open the door; if so, proceed to step S507; if not, proceed to step S511.

[0084] S507, if the obstacle's action intention is to open the door, the door is controlled to open to a first angle (the first angle can be a small angle of 5%), and a voice reminder is given to the user to stay away from the door.

[0085] S508, determining whether the user is outside the movement track area of ​​the vehicle door; if so, proceeding to step S509; if not, proceeding to step S510.

[0086] S509, if the user is outside the movement track area of ​​the door, the door is controlled to open to a second angle (the second angle may be a maximum angle), and the process ends.

[0087] S510: If the user is within the movement track area of ​​the door, the door is controlled to stop so that the door is maintained at the first angle, and the process returns to step S508.

[0088] S511, if the attribute type of the obstacle is an object type, and / or the action intention of the obstacle is not to open the door, the door is controlled not to be opened, and a voice reminder is given to the user that there is obstacle information in the movement trajectory area of ​​the door, and the process ends.

[0089] S512: If the ultrasonic radar of the door does not detect an obstacle, the door is controlled to open to a second angle (the second angle may be a maximum angle), and the process ends.

[0090] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a door obstacle avoidance control device, a vehicle, a computer-readable storage medium and corresponding embodiments.

[0091] Figure 6 It is a schematic diagram of the structure of a vehicle door obstacle avoidance control device shown in an embodiment of the present application.

[0092] See also Figure 6 The present application provides a door obstacle avoidance control device, which may include: The image data acquisition module 610 is used to acquire image data collected by the side view camera when receiving the automatic door opening request if the ultrasonic radar of the vehicle door detects an obstacle; the image data includes the obstacle; An auxiliary information recognition module 620 is used to recognize the attribute type and action intention of the obstacle according to the image data; The first control module 630 is used to control the vehicle door to open when the attribute type is a person type and the action intention is a door opening intention.

[0093] In one implementation, the first control module 630 may include: The target distance acquisition submodule is used to obtain the target distance between the obstacle and the ultrasonic radar when the attribute type is the person type and the action intention is the door opening intention; The opening angle control submodule is used to control the opening angle of the door according to the target distance.

[0094] In one embodiment, the opening angle control submodule may include: A first angle opening unit is used to control the door to open to a first angle and send a first prompt message to the obstacle when the target distance is less than or equal to the preset distance and the obstacle is determined to be within the movement trajectory of the door; the first prompt message is used to instruct the obstacle to stay away from the door; The second angle opening unit is used to control the door to open to a second angle when it is determined that the obstacle is outside the movement trajectory area of ​​the door when the target distance is greater than the preset distance; wherein the second angle is greater than the first angle.

[0095] In one implementation, the auxiliary information identification module 620 may include: The target detection submodule is used to perform target detection on image data and obtain the attribute type of obstacles; The human posture estimation submodule is used to estimate the human posture of the obstacle and obtain the action intention of the obstacle when the attribute type is the person type.

[0096] In one embodiment, the image data includes a plurality of image frames captured by the side-view camera within a reception time before and after the automatic door opening request; The human posture estimation submodule may include: A human joint point detection unit, for detecting each human joint point of an obstacle in each image frame when the attribute type is a person type; A human joint point connection unit is used to connect the human joint points in sequence in each image frame according to a preset order to obtain target posture information of obstacles in each image frame; a posture change information determining unit, configured to determine the posture change information of the obstacle based on the target posture information of the obstacle in each image frame; The action intention recognition unit is used to recognize the action intention of the obstacle according to the posture change information.

[0097] In one embodiment, the device may further include: The second control module is used to control the vehicle door not to open and send a second prompt message to the user when the attribute type is an object type and / or the action intention is not an intention to open the door; wherein the second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the vehicle door.

[0098] In one embodiment, the device may further include: The third control module is used to control the vehicle door to open to a second angle if the ultrasonic radar does not detect an obstacle.

[0099] As can be seen from this example, the solution provided by this application, when receiving an automatic door opening request, if the ultrasonic radar of the car door detects an obstacle, then the image data collected by the side-view camera is obtained; the image data contains the obstacle; according to the image data, the attribute type and action intention of the obstacle are identified; when the attribute type is a person type, and the action intention is the intention to open the door, the door is controlled to open. This application uses the side-view camera to assist in judging the attribute type and action intention of the obstacle. When it is determined based on the attribute type and action intention that the user is misjudged as an obstacle by the ultrasonic radar of the car door due to the user's door opening action, the door can be controlled to open, thereby solving the problem of being unable to open the door due to the misjudgment of the ultrasonic radar, optimizing the obstacle avoidance strategy, and improving the user's riding experience. Regarding the device in the above embodiment, the specific way in which each module performs the operation has been described in detail in the embodiment of the method, and will not be explained in detail here.

[0100] Figure 7 It is a schematic diagram of the structure of a vehicle shown in an embodiment of the present application.

[0101] See also Figure 7 , the vehicle 700 includes a memory 710 and a processor 720 .

[0102] The processor 720 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0103] The memory 710 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM may store static data or instructions required by the processor 720 or other modules of the computer. The permanent storage may be a readable and writable storage device. The permanent storage may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 710 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 710 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0104] The memory 710 stores executable codes, and when the executable codes are processed by the processor 720 , the processor 720 can execute part or all of the above-mentioned methods.

[0105] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0106] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by the vehicle's processor, the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0107] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A door obstacle avoidance control method, characterized in that: include: When receiving an automatic door opening request, if the ultrasonic radar of the vehicle door detects an obstacle, image data collected by the side view camera is obtained; the image data includes the obstacle; According to the image data, identifying the attribute type and action intention of the obstacle; When the attribute type is a person type, and the action intention is a door opening intention, the vehicle door is controlled to open.

2. The method according to claim 1, characterized in that When the attribute type is a person type and the action intention is a door opening intention, controlling the vehicle door to open includes: When the attribute type is a person type and the action intention is a door opening intention, obtaining a target distance between the obstacle and the ultrasonic radar; The opening angle of the door is controlled according to the target distance.

3. The method according to claim 2, characterized in that The controlling the opening angle of the door according to the target distance includes: When the target distance is less than or equal to the preset distance, it is determined that the obstacle is located within the movement trajectory area of ​​the vehicle door, the vehicle door is controlled to open to a first angle, and a first prompt message is sent to the obstacle; the first prompt message is used to instruct the obstacle to stay away from the vehicle door; When the target distance is greater than the preset distance, it is determined that the obstacle is outside the movement track area of ​​the door, and the door is controlled to open to a second angle; wherein the second angle is greater than the first angle.

4. The method according to claim 1, characterized in that: The step of identifying the attribute type and action intention of the obstacle according to the image data includes: Performing target detection on the image data to obtain the attribute type of the obstacle; When the attribute type is a person type, a human posture estimation is performed on the obstacle to obtain the action intention of the obstacle.

5. The method according to claim 4, characterized in that The image data includes a plurality of image frames captured by the side-view camera within a reception time before and after the automatic door opening request; When the attribute type is a person type, estimating the human body posture of the obstacle to obtain the action intention of the obstacle includes: When the attribute type is a person type, detecting each human body joint point of the obstacle in each image frame respectively; In each image frame, each of the human body joints is sequentially connected according to a preset order to obtain target posture information of the obstacle in each image frame; Determining the posture change information of the obstacle based on the target posture information of the obstacle in each image frame; The movement intention of the obstacle is identified according to the posture change information.

6. The method according to claim 1, characterized in that The method further comprises: When the attribute type is an object type, and / or the action intention is not an intention to open the door, the vehicle door is controlled not to open, and a second prompt message is sent to the user; wherein, the second prompt message is used to remind the user that there is obstacle information in the movement trajectory area of ​​the vehicle door.

7. The method according to claim 1, characterized in that The method further comprises: If the ultrasonic radar does not detect an obstacle, the vehicle door is controlled to open to a second angle.

8. A door obstacle avoidance control device, characterized in that: include: An image data acquisition module, configured to acquire image data collected by a side-view camera when receiving an automatic door opening request and if the ultrasonic radar of the vehicle door detects an obstacle; the image data includes the obstacle; An auxiliary information recognition module, used to identify the attribute type and action intention of the obstacle according to the image data; The first control module is used to control the vehicle door to open when the attribute type is a person type and the action intention is a door opening intention.

9. A vehicle, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of a vehicle, causes the processor to execute the method according to any one of claims 1 to 7.

Images