Door control method, door control device, vehicle terminal, vehicle and medium

Abstract

The present invention discloses a door control system and a door control method. The door control system includes a vehicle-mounted camera, a vehicle-mounted radar and a controller. The vehicle-mounted camera is used to collect image information of obstacles around the vehicle. The vehicle-mounted radar is used to collect ultrasonic information of obstacles around the vehicle. The controller is connected to the vehicle-mounted camera and the vehicle-mounted radar respectively. The controller is used to fuse the image information of obstacles around the vehicle and the ultrasonic information of obstacles around the vehicle, obtain the spatial state change relationship between the obstacles around the vehicle and the door, and control the door according to the spatial state change relationship. The above-mentioned door control method, door control device, vehicle-mounted terminal, vehicle and medium can improve the intelligence of the door and enhance the safety and reliability of the door.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle doors, and in particular to a door control method, a door control device, a vehicle-mounted terminal, a vehicle and a medium. Background Art

[0002] With the rapid development of science and technology, vehicle intelligence has become a trend. In related technologies, solutions for intelligent doors only focus on the implementation of door lock systems, while ignoring the impact of environmental factors on the door opening and closing process. For example, in the intelligent solution for swing doors, only the focus is on how to achieve automatic door opening and closing, without considering the situation where environmental factors hinder the door opening and closing process, resulting in a low degree of intelligentization of the door, affecting the user experience. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one object of the present invention is to provide a vehicle door control system, which can improve the intelligence level of the vehicle door and enhance the safety and reliability of the vehicle door.

[0004] A second object of the present invention is to provide a vehicle door control method.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention proposes a door control system, which includes a vehicle-mounted camera, a vehicle-mounted radar and a controller, wherein the vehicle-mounted camera is used to collect image information of obstacles around the vehicle, and the vehicle-mounted radar is used to collect ultrasonic information of obstacles around the vehicle. The controller is connected to the vehicle-mounted camera and the vehicle-mounted radar, respectively, and the controller is used to fuse the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, obtain the spatial state change relationship between the obstacles around the vehicle and the door, and control the door according to the spatial state change relationship.

[0006] According to the door control system of the embodiment of the present invention, the door is controlled according to the spatial state change relationship of the obstacles around the vehicle relative to the door, thereby improving the intelligence level of the door and enhancing the safety and reliability of the door.

[0007] In some embodiments of the present invention, the controller is also used to establish a spatial rectangular coordinate system based on the positional relationship between the vehicle-mounted camera and the vehicle-mounted radar, and to determine the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system based on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, and to determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door based on the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system.

[0008] In some embodiments of the present invention, the controller is also used to determine the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system, calculate the distance between the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system and the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system, and determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door based on the distance.

[0009] In some embodiments of the present invention, the controller is also used to determine that the obstacles around the vehicle are in the dangerous space of the door when the distance is less than a first threshold; to determine that the obstacles around the vehicle are in the safe space of the door when the distance is greater than a second threshold; and to determine that the obstacles around the vehicle are in the critical space of the door when the distance is greater than or equal to the first threshold and less than or equal to the second threshold, and the first threshold is less than the second threshold.

[0010] In some embodiments of the present invention, the controller is also used to control the door to prohibit executing the received door opening and closing instructions when the obstacle around the vehicle is in the dangerous space of the door and the door is in a stopped state; control the door to stop moving and perform anti-collision protection when the obstacle around the vehicle is in the dangerous space of the door and the door is in a moving state; control the door to execute the received door opening and closing instructions when the obstacle around the vehicle is in the safe space of the door and the door is in the stopped state; control the door to execute the received door opening and closing instructions when the obstacle around the vehicle is in the safe space of the door and the door is in the moving state; control the door to postpone the execution of the received door opening and closing instructions and perform anti-collision protection when the obstacle around the vehicle is in the critical space of the door and the door is in a stopped state; and control the door according to the movement time required for the obstacle around the vehicle to move to the dangerous space of the door when the obstacle around the vehicle is in the critical space of the door and the door is in a moving state.

[0011] In some embodiments of the present invention, the controller is also used to control the vehicle door to stop moving and perform anti-collision protection when the vehicle door is executing a door closing command and the movement duration is less than or equal to the remaining closing time of the vehicle door; and to control the vehicle door to continue executing the door closing command when the vehicle door is executing the door closing command and the movement duration is greater than the remaining closing time of the vehicle door.

[0012] In some embodiments of the present invention, the vehicle-mounted radar includes a first radar and a second radar, and the vehicle-mounted camera, the first radar and the second radar are located on the same straight line. The controller is also used to establish the spatial rectangular coordinate system with the vehicle-mounted camera as the origin and the straight line as the x-axis.

[0013] In some embodiments of the present invention, the three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component and a z-axis component, wherein the controller is also used to determine the projection length of the line connecting the first radar and the obstacle around the vehicle on the x-axis based on the ultrasonic information collected by the first radar and the ultrasonic information collected by the second radar, and determine the first x-axis component based on the distance between the first radar and the vehicle-mounted camera and the projection length, and determine the two-dimensional coordinates of the obstacle around the vehicle in the two-dimensional imaging image based on the image information, and determine the first y-axis component and the z-axis component based on the first x-axis component and the two-dimensional coordinates, and use the first x-axis component, the first y-axis component and the z-axis component as the three-dimensional coordinates of the obstacle around the vehicle.

[0014] In some embodiments of the present invention, the two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller is also used to determine the proportional relationship between the shooting size of the vehicle-mounted camera and the size of the two-dimensional imaging image based on the first x-axis component and the second x-axis component, and to determine the first y-axis component based on the proportional relationship and the second y-axis component, and to determine the z-axis component based on the proportional relationship.

[0015] In some embodiments of the present invention, the vehicle door control system further includes a display, and the display is connected to the controller, wherein the controller is further used to control the display to dynamically display the relationship of the spatial state changes.

[0016] To achieve the above-mentioned purpose, a second embodiment of the present invention proposes a door control method, which includes: obtaining image information of obstacles around the vehicle, and obtaining ultrasonic information of the obstacles around the vehicle; fusing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle to obtain the spatial state change relationship between the obstacles around the vehicle and the door; and controlling the door according to the spatial state change relationship.

[0017] According to the vehicle door control method of the embodiment of the present invention, the vehicle door is controlled according to the spatial state change relationship of the obstacles around the vehicle relative to the vehicle door, thereby improving the intelligence level of the vehicle door and enhancing the safety and reliability of the vehicle door.

[0018] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a structural block diagram of a door control system according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of intelligent obstacle recognition of a door control system according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a scene of a door control system according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a scene of a door control system according to an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of a scene of a door control system according to an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of a scene of a door control system according to an embodiment of the present invention;

[0025] Figure 7 is a flow chart of a door control method according to an embodiment of the present invention;

[0026] Figure 8 is a flow chart of a door control method according to an embodiment of the present invention;

[0027] Fig. 9 is a flow chart of a door control method according to an embodiment of the present invention;

[0028] Fig.10 is a flow chart of a door control method according to an embodiment of the present invention;

[0029] Fig.11 is a flow chart of a door control method according to an embodiment of the present invention;

[0030] Fig.12 is a flow chart of a door control method according to an embodiment of the present invention;

[0031] Fig.13 is a flow chart of a door control method according to an embodiment of the present invention;

[0032] Fig.14 4 is a flow chart of a door control method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0034] See also Figure 1 The present invention proposes a door control system 100, which includes a vehicle-mounted camera 10, a vehicle-mounted radar 20 and a controller 30. The vehicle-mounted camera 10 is used to collect image information of obstacles around the vehicle, and the vehicle-mounted radar 20 is used to collect ultrasonic information of obstacles around the vehicle. The controller 30 is connected to the vehicle-mounted camera 10 and the vehicle-mounted radar 20 respectively. The controller 30 is used to fuse the image information of obstacles around the vehicle and the ultrasonic information of obstacles around the vehicle, obtain the spatial state change relationship between the obstacles around the vehicle and the door 40, and control the door 40 according to the spatial state change relationship.

[0035] According to the vehicle door control system 100 of the embodiment of the present invention, the vehicle door is controlled according to the spatial state change relationship of the obstacles around the vehicle relative to the vehicle door 40, thereby improving the intelligence level of the vehicle door 40 and enhancing the safety and reliability of the vehicle door 40.

[0036] It can be understood that, in the related technologies, obstacles around the vehicle are usually identified only based on ultrasonic information of obstacles around the vehicle collected by the vehicle-mounted radar, or only based on image information of obstacles around the vehicle collected by the vehicle-mounted camera. The data collected by the vehicle-mounted radar and the vehicle-mounted camera are not integrated to identify the obstacles around the vehicle based on the integrated data, and the identification of obstacles around the vehicle is not applied to the door control solution.

[0037] Furthermore, in the related technology of identifying obstacles around the vehicle based only on the ultrasonic information of obstacles around the vehicle collected by the vehicle-mounted radar, according to the ultrasonic principle, the vehicle-mounted radar sends ultrasonic waves, and after the ultrasonic waves hit the obstacles around the vehicle, a part of them is reflected back and detected by the vehicle-mounted radar, and the actual distance between the vehicle and the obstacles around the vehicle is calculated based on the detected reflected waves. However, due to the installation height angle, as well as the ultrasonic divergence angle and other issues, the vehicle-mounted radar has a working blind spot, and the vehicle-mounted radar cannot accurately identify the size and contour of the obstacles around the vehicle. In the related technology of identifying obstacles around the vehicle based only on the image information of obstacles around the vehicle collected by the vehicle-mounted camera, the vehicle-mounted camera is used to collect image information for intuitive analysis. However, due to the installation height and angle of the vehicle-mounted camera, after the image information is analyzed, calculated and restored, there is a certain error between the actual obstacle information around the vehicle and the displayed information. In addition, the vehicle-mounted camera is too dependent on environmental factors. For example, at night or in a dimly lit environment, or in rainy and snowy weather, the image information collected by the vehicle-mounted camera is of low quality, which is not conducive to analysis, calculation and restoration. At the same time, if the vehicle-mounted camera is accidentally contaminated or blocked, it will be even more difficult to collect real and reliable image information.

[0038] Moreover, the door control system in the related art only focuses on the realization of automatic door opening and closing, and does not recognize obstacles around the vehicle and automatically open and close the door according to the spatial state change relationship of the obstacles around the vehicle. In other words, the door control system in the related art has problems such as low intelligence and low reliability.

[0039] The vehicle door control system 100 of this embodiment takes into account the advantage that the vehicle-mounted radar 20 is less affected by the environment during operation, for example, it can detect obstacles around the vehicle at night or in dimly lit places, and takes into account the advantage that the vehicle-mounted camera 10 can achieve blind spot detection based on the installation position, and fuses and analyzes the image information collected by the vehicle-mounted camera 10 and the ultrasonic information collected by the vehicle-mounted radar 20, combining the advantages of both parties to compensate for the disadvantages of both parties, thereby ensuring the accuracy of identifying obstacles around the vehicle, so that the vehicle door control system 100 can be implemented, and the vehicle door control system 100 adds a related scheme for judging the spatial state change relationship between the obstacles around the vehicle and the door 40, so that the vehicle door control system 100 of this embodiment is more intelligent, reliable and safe during operation. At the same time, the vehicle door control system 100 implemented by the present invention based on the vehicle-mounted camera 10 and the vehicle-mounted radar 20 already installed on the vehicle can also effectively control costs.

[0040] Specifically, the vehicle door 40 may include a flat door, a double door, a side sliding door, etc. The actuator of the vehicle door 40 may include a door, a lock, etc.

[0041] The vehicle-mounted camera 10 may include multiple cameras, and each vehicle-mounted camera 10 may be oriented in a different direction, so as to monitor the environment around the vehicle from multiple angles and collect image information of obstacles around the vehicle. The multiple vehicle-mounted cameras 10 may be arranged to be distributed around the vehicle body, or may be arranged to be gathered on the top of the vehicle. When the multiple vehicle-mounted cameras 10 are gathered on the top of the vehicle, the multiple cameras may be arranged to be fixedly mounted on the top of the vehicle, or may be arranged to be rotatably mounted on the top of the vehicle, and the multiple vehicle-mounted cameras 10 may be arranged to rotate synchronously, or may be arranged to rotate individually.

[0042] The vehicle-mounted camera 10 may also include one, and one vehicle-mounted camera 10 may be arranged to be rotatably mounted on the top of the vehicle, so as to achieve the effect of monitoring the environment around the vehicle. The vehicle-mounted camera 10 may shoot obstacles around the vehicle within its own direction range at a preset sampling frequency to obtain image information of obstacles around the vehicle, or may shoot obstacles around the vehicle within its own direction range in real time by recording to obtain image information of obstacles around the vehicle.

[0043] There may be multiple vehicle-mounted radars 20 , and each vehicle-mounted radar 20 may be oriented in a different direction, so as to detect the environment around the vehicle from multiple directions and collect ultrasonic information of obstacles around the vehicle.

[0044] The obstacles around the vehicle may be movable persons or objects around the vehicle, such as pedestrians, vehicles, animals, etc. The spatial state change relationship of the obstacles around the vehicle can be understood as the change relationship of the spatial position of the obstacles around the vehicle relative to the vehicle door 40 .

[0045] In some embodiments, the controller 30 is also used to receive feedback signals from the actuator of the door 40, such as lock status, door speed, door position, current, voltage, anti-pinch signal, etc. It can be understood that feedback control of the door 40 based on the feedback signal of the actuator of the door 40 can accurately realize the hovering of the smart door, realize the anti-pinch function during opening and closing, and quickly realize functions such as emergency avoidance.

[0046] In some embodiments, the controller 30 is also used to receive a door control signal, which may include one or more of the vehicle speed information, the vehicle gear information and the received voice information. The controller 30 is used to control the door 40 according to the received door control signal.

[0047] In one example, the controller 30 is used to detect the vehicle speed information and gear information of the vehicle; when the vehicle speed information and gear information of the vehicle meet the preset conditions, the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle are fused and processed to obtain the spatial state change relationship between the obstacles around the vehicle and the door 40, and the door 40 is controlled according to the spatial state change relationship. In this way, the door 40 is controlled in time, which is more intelligent and improves the user experience. The preset conditions may include that the vehicle speed is less than or equal to the preset speed (for example, 3km / h), and the gear is the preset gear (for example, neutral gear, parking gear). It can be understood that when the vehicle speed is greater than the preset speed or the gear is the forward gear or the reverse gear, the user may not have the intention to open or close the door. At this time, there is no need to determine the spatial state change relationship, nor is there any need to control the door 40 according to the spatial state change relationship.

[0048] In another example, the controller 30 is used to analyze the received user voice to determine whether the user voice includes door opening and closing instructions, and when receiving the door opening and closing instructions, fuse the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle to obtain the spatial state change relationship between the obstacles around the vehicle and the door 40, and control the door 40 according to the spatial state change relationship.

[0049] It can be understood that the controller 30 can be used for intelligent obstacle recognition, door anti-pinch detection, door speed detection, door position detection, door opening and closing intention detection, emergency avoidance triggering, alarm protection control, central control LCD screen smart door system display control, etc.

[0050] In one example, the process of fusing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle is as follows: Figure 2 As shown, by processing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle acquired at multiple moments, the spatial state change relationship between the obstacles around the vehicle and the door 40 can be determined.

[0051] In some embodiments of the present invention, the controller 30 is also used to establish a spatial rectangular coordinate system based on the positional relationship between the vehicle-mounted camera 10 and the vehicle-mounted radar 20, and to determine the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system based on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, and to determine the spatial state change relationship between the obstacles around the vehicle and the door 40 based on the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system.

[0052] In this way, the spatial position of the obstacles around the vehicle relative to the door 40 can be determined more accurately.

[0053] Specifically, the spatial state change relationship can be divided based on different spatial regions, so that after determining the three-dimensional coordinates of the obstacles around the vehicle, the spatial region where the obstacles around the vehicle are located can be determined, and then the spatial state change relationship of the obstacles around the vehicle can be quickly determined.

[0054] In some embodiments of the present invention, the vehicle-mounted radar 20 includes a first radar and a second radar, and the vehicle-mounted camera 10, the first radar and the second radar are located on the same straight line. The controller 30 is also used to establish a spatial rectangular coordinate system with the vehicle-mounted camera 10 as the origin and the straight line as the x-axis.

[0055] In this way, it is easy to determine the three-dimensional coordinates of obstacles around the vehicle.

[0056] Specifically, please combine Figure 3 , take the vehicle camera O as the origin, the straight line where the vehicle camera O, the first radar L1 and the second radar L2 are located as the x-axis, the x-axis is basically parallel to the horizontal plane, the straight line passing through the vehicle camera O and perpendicular to the horizontal plane is the y-axis, and the straight line passing through the vehicle camera O and parallel to the horizontal plane and perpendicular to the x-axis is the z-axis, to establish a spatial rectangular coordinate system.

[0057] In some embodiments of the present invention, the three-dimensional coordinates of the obstacles around the vehicle include a first x-axis component, a first y-axis component and a z-axis component, wherein the controller 30 is also used to determine the projection length of the line connecting the first radar and the obstacles around the vehicle on the x-axis based on the ultrasonic information collected by the first radar and the ultrasonic information collected by the second radar, and determine the first x-axis component based on the distance between the first radar and the vehicle-mounted camera 10 and the projection length, and determine the two-dimensional coordinates of the obstacles around the vehicle in the two-dimensional imaging image based on the image information, and determine the first y-axis component and the z-axis component based on the first x-axis component and the two-dimensional coordinates, and use the first x-axis component, the first y-axis component and the z-axis component as the three-dimensional coordinates of the obstacles around the vehicle.

[0058] Please combine again Figure 3 , since the positions of the vehicle-mounted camera O, the first radar L1, and the second radar L2 are fixed in advance, the distance a1 between the vehicle-mounted camera O and the first radar L1 is a known quantity, and the distance a2 between the first radar L1 and the second radar L2 is also a known quantity. The distance b1 between the first radar L1 and the obstacle W around the vehicle can be determined based on the ultrasonic information collected by the first radar L1. The distance b2 between the second radar L2 and the obstacle W around the vehicle can be determined based on the ultrasonic information collected by the second radar L2.

[0059] In the triangle formed by the first radar L1, the second radar L2 and the obstacle W around the vehicle, since the lengths of the three sides of the triangle a2, b1 and b2 are all known quantities, the projection length a3 of the line L1W connecting the first radar L1 and the obstacle W around the vehicle on the x-axis, that is, the length of the line segment L1H, can be calculated based on the Pythagorean theorem.

[0060] The sum of the distance a1 between the first radar L1 and the vehicle-mounted camera O and the projection length a3 is taken as the first x-axis component x1.

[0061] The two-dimensional imaging image can be understood as a two-dimensional image generated according to the external light signal, i.e., image information, received by the vehicle-mounted camera 10 within the actual shooting size range. When the obstacles around the vehicle in the actual space are within the shooting size range of the vehicle-mounted camera 10, the obstacles around the vehicle can be presented in the two-dimensional imaging image, and the two-dimensional coordinates corresponding to the obstacles around the vehicle in the two-dimensional imaging image are known. The size of the two-dimensional imaging image includes a first long side and a first wide side, wherein the first long side corresponds to the horizontal plane, and the first wide side is perpendicular to the first long side. An x-axis is established according to the first long side of the two-dimensional imaging image, and a y-axis is established according to the first wide side of the two-dimensional imaging image, so that the second x-axis component x0 and the second y-axis component y0 of the obstacles around the vehicle can be determined according to the position of the obstacles around the vehicle in the two-dimensional imaging image, thereby determining the two-dimensional coordinates of the obstacles around the vehicle as (x0, y0). In some embodiments, the size of the two-dimensional imaging image is the size of the display screen.

[0062] In some embodiments of the present invention, the two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller 30 is also used to determine the proportional relationship between the shooting size of the vehicle-mounted camera 10 and the size of the two-dimensional imaging image based on the first x-axis component and the second x-axis component, and to determine the first y-axis component based on the proportional relationship and the second y-axis component, and to determine the z-axis component based on the proportional relationship.

[0063] It can be understood that the shooting size of the vehicle-mounted camera 10 includes the second long side and the second wide side, wherein the second long side corresponds to the horizontal plane, and the second wide side is perpendicular to the second long side. When the parameters and installation position of the vehicle-mounted camera 10 are determined, there is a proportional relationship between the shooting size of the vehicle-mounted camera 10 and the distance from the obstacles around the vehicle to the x-axis and the size of the two-dimensional imaging image. For example, when the parameters and installation position of the vehicle-mounted camera 10 are determined, when the size of the display screen, i.e., the size of the two-dimensional imaging image, is 8 inches, the external environment at a distance of 5 meters from the x-axis is shot, and the shooting size is 13*9.8 meters. The range is displayed on the full screen on the display screen; the external environment at a distance of 10 meters from the x-axis is shot, and the shooting size is 26*19.5 meters. The range is displayed on the full screen on the display screen. Therefore, the proportional relationship between the shooting size of the vehicle-mounted camera 10 and the size of the two-dimensional imaging image can be determined according to the ratio of the first x-axis component to the second x-axis component.

[0064] Furthermore, after determining the proportional relationship between the shooting size of the vehicle-mounted camera 10 and the size of the two-dimensional imaging image, the first y-axis component y1 and the z-axis component z1 can be determined according to the proportional relationship, thereby determining the three-dimensional coordinates of the obstacles around the vehicle as (x1, y1, z1).

[0065] In some embodiments of the present invention, the controller 30 is also used to determine the three-dimensional coordinates of the vehicle door 40 in the spatial rectangular coordinate system, calculate the distance between the three-dimensional coordinates of obstacles around the vehicle in the spatial rectangular coordinate system and the three-dimensional coordinates of the vehicle door 40 in the spatial rectangular coordinate system, and determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door 40 based on the distance.

[0066] In this way, the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door 40 are determined based on the same spatial rectangular coordinate system, which makes it convenient to determine the distance between the obstacles around the vehicle and the door 40 according to the three-dimensional coordinates, and then determine the spatial state change relationship between the obstacles around the vehicle and the door 40.

[0067] Specifically, the three-dimensional coordinates of the door 40 can be determined according to the installation position of the door 40. In some embodiments, the coordinates of the center point of the door frame can be used as the three-dimensional coordinates of the door 40. In one example, the three-dimensional coordinates of the obstacle around the vehicle are (x1, y1, z1), and the three-dimensional coordinates of the door 40 are (x2, y2, z2). The distance D between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 can be calculated by solving the following formula: D 2 =(x1-x2) 2 +(y1-y2) 2 +(z1-z2) 2 .

[0068] In some embodiments of the present invention, the controller 30 is also used to determine that obstacles around the vehicle are in a dangerous space of the door 40 when the distance is less than a first threshold; determine that obstacles around the vehicle are in a safe space of the door 40 when the distance is greater than a second threshold; and determine that obstacles around the vehicle are in a critical space of the door 40 when the distance is greater than or equal to the first threshold and less than or equal to the second threshold, and the first threshold is less than the second threshold.

[0069] In this way, after determining the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door 40, by calculating the distance between the obstacles around the vehicle and the door 40, the spatial state change relationship of the obstacles around the vehicle relative to the door 40 can be quickly determined.

[0070] Specifically, the spatial state change relationship may include a dangerous space at the vehicle door 40 , a safe space at the vehicle door 40 , and a critical space at the vehicle door 40 .

[0071] The dangerous space of the door 40 can be understood as the entire travel space area of ​​the door 40 during the process of opening and closing the door. The vehicle surrounding obstacles within this space area are effective vehicle surrounding obstacles. No matter whether the vehicle surrounding obstacles are in a moving state or a stationary state, the door cannot be opened and closed.

[0072] The safety space of the door 40 can be understood as a space area that does not affect the opening and closing of the door. The obstacles around the vehicle in this space area are invalid obstacles around the vehicle, and there is no need to worry about whether the obstacles around the vehicle are in a moving state or a stationary state.

[0073] The critical space of the vehicle door 40, that is, the spatial area between the safe space and the dangerous space, has uncertainty in the obstacles surrounding the vehicle. If the obstacles surrounding the vehicle in this spatial area have a tendency to move toward the dangerous space, they can be determined as valid obstacles surrounding the vehicle. If the obstacles surrounding the vehicle in this spatial area have a tendency to move toward the safe space, they can be determined as invalid obstacles surrounding the vehicle.

[0074] In an example, the first threshold is 1m and the second threshold is 2m, that is, when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door 40 is less than 1m, it can be determined that the obstacles around the vehicle are in the dangerous space of the door 40; when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door 40 is greater than 2m, it can be determined that the obstacles around the vehicle are in the safe space of the door 40; when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door 40 is greater than or equal to 1m and less than or equal to 2m, it can be determined that the obstacles around the vehicle are in the critical space of the door 40.

[0075] In some embodiments, the first threshold and the second threshold may include a tolerance range (e.g., ±20 cm, ±30 cm). In this way, the safety of the door when it is opened and closed can be further improved. For example, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is less than 1m±20cm, it can be determined that the obstacle around the vehicle is in the dangerous space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is greater than 2m±20cm, it can be determined that the obstacle around the vehicle is in the safe space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is greater than or equal to 1m±20cm and less than or equal to 2m±20cm, it can be determined that the obstacle around the vehicle is in the critical space of the door 40.

[0076] In some embodiments of the present invention, the controller 30 is further used to control the door 40 to prohibit the execution of the received door opening and closing instructions when the obstacle around the vehicle is in the dangerous space of the door 40 and the door 40 is in a stopped state; control the door 40 to stop moving and perform anti-collision protection when the obstacle around the vehicle is in the dangerous space of the door 40 and the door 40 is in a moving state; control the door 40 to execute the received door opening and closing instructions when the obstacle around the vehicle is in the safe space of the door 40 and the door 40 is in a stopped state; control the door 40 to execute the received door opening and closing instructions when the obstacle around the vehicle is in the safe space of the door 40 and the door 40 is in a moving state; control the door 40 to postpone the execution of the received door opening and closing instructions and perform anti-collision protection when the obstacle around the vehicle is in the critical space of the door 40 and the door 40 is in a stopped state; and control the door 40 according to the movement time required for the obstacle around the vehicle to move to the dangerous space of the door 40.

[0077] In this way, the relationship between the spatial state changes and the state of the door 40 is comprehensively considered, and the door control is performed according to the relationship between the spatial state changes and the state of the door 40, which can make the opening and closing of the door 40 more intelligent and improve safety. It can be understood that in the related art, the door opening or closing action is immediately executed after receiving the door opening and closing command. If the door opening and closing command is issued by the user without paying attention to the vehicles and pedestrians in the front and rear, the direct door opening at this time may collide with pedestrians or vehicles, causing serious traffic accidents. In the door control system of the embodiment of the present invention, after receiving the door opening and closing command, the spatial state change relationship of the obstacles around the vehicle is first actively identified, and then it is determined whether to execute the received door opening and closing command based on the identification result, which can effectively warn the user in advance to prevent collisions caused by directly opening and closing the door 40 and ensure user safety.

[0078] Specifically, the stopped state may be a state where the door 40 remains stationary after being fully opened, or a state where the door 40 remains stationary after being fully closed, or a state where the door 40 remains stationary at any travel position between fully opened and fully closed. The moving state may be a state where the door 40 gradually opens, or a state where the door 40 gradually closes. The moving state of the door 40 may be generated by the door 40 automatically moving according to the door opening and closing instructions, or by the user manually operating the door 40.

[0079] Furthermore, when obstacles around the vehicle are in the dangerous space of the door 40 and the door 40 is in a stopped state, if a door opening and closing command is detected, it is not executed.

[0080] When obstacles around the vehicle are in the dangerous space of the door 40, if the door is being opened or closed, the door opening or closing action is stopped immediately and anti-collision protection is performed to prevent damage to the user's personal safety and property due to continued opening or closing of the door 40.

[0081] When obstacles around the vehicle are in the safe space of the door 40, no matter the door 40 is in a stopped state or in a moving state, the door 40 can be opened and closed normally according to the received door opening and closing instructions.

[0082] When obstacles around the vehicle are in the critical space of the door 40 and the door 40 is in a stopped state, if a door opening and closing command is received, that is, the user intends to open and close the door, it will not be executed. At the same time, anti-collision protection will be performed, and the door opening and closing command will be executed after the obstacles around the vehicle enter the safe space.

[0083] When obstacles around the vehicle are in the critical space of the door 40, if the door is being opened or closed, how to control the door 40 can be further determined based on the movement time required for the obstacles around the vehicle to move to the dangerous space of the door 40.

[0084] In some embodiments of the present invention, the controller 30 is also used to control the vehicle door 40 to stop moving and perform anti-collision protection when the vehicle door 40 is executing a door closing command and the movement duration is less than or equal to the remaining closing time of the vehicle door 40; and to control the vehicle door 40 to continue executing the door closing command when the vehicle door 40 is executing a door closing command and the movement duration is greater than the remaining closing time of the vehicle door 40.

[0085] In this way, during the process of closing the car door, while ensuring safety, the door closing command is completed as much as possible, thereby avoiding the user waiting due to the inability to close the door and preventing the user's schedule from being delayed.

[0086] Specifically, please combine Figure 4According to the ultrasonic information of the obstacles around the vehicle collected by the vehicle-mounted radar 20 at different times, it can be inferred whether the obstacles around the vehicle in the image information are the same obstacles around the vehicle, and the movement trend of the obstacles around the vehicle relative to the door 40. For example, if the distance between the obstacles around the vehicle detected by the radar at different times and the vehicle remains unchanged, it can be considered that the obstacles around the vehicle are in a stationary state; if the distance between the obstacles around the vehicle detected by the radar and the vehicle gradually decreases, it can be considered that the movement trend of the obstacles around the vehicle is approaching the door 40; if the distance between the obstacles around the vehicle detected by the radar and the vehicle gradually increases, it can be considered that the movement trend of the obstacles around the vehicle is moving away from the door 40.

[0087] Please combine Figure 5 According to the image information of the obstacles around the vehicle collected by the vehicle-mounted camera 10 at different times, it is possible to infer whether the obstacles around the vehicle in the image information are the same obstacles around the vehicle, and the movement trend of the obstacles around the vehicle relative to the vehicle door 40. For example, multiple groups of two-dimensional imaging images can be obtained by using multiple vehicle-mounted cameras 10 installed at different positions, and image analysis can be performed on the multiple groups of two-dimensional imaging images. Each group of two-dimensional imaging images is divided into multiple grids, and the outline of the obstacles around the vehicle in each group of two-dimensional imaging images is plotted. If the area of ​​the outline of the obstacles around the vehicle does not change, it can be considered that the obstacles around the vehicle are in a stationary state; if the area of ​​the outline of the obstacles around the vehicle gradually increases, it can be considered that the movement trend of the obstacles around the vehicle is close to the vehicle door 40; if the area of ​​the outline of the obstacles around the vehicle gradually decreases, it can be considered that the movement trend of the obstacles around the vehicle is away from the vehicle door 40.

[0088] Furthermore, the movement time required for the obstacles around the vehicle to move to the dangerous space of the door 40 can be calculated by combining the movement trend of the obstacles around the vehicle and the estimated movement speed of the obstacles around the vehicle. The current position of the door 40 and the closing speed of the door 40 can be determined through relevant sensor data and current data, and then the remaining closing time of the door 40 can be determined according to the current position of the door 40 and the closing speed of the door 40. If the door 40 is executing the door closing command and the movement time required for the obstacles around the vehicle to move to the dangerous space of the door 40 is less than or equal to the remaining closing time of the door 40, continuing to close the door 40 will cause the obstacles around the vehicle to collide with the door 40 or be clamped by the door 40. Therefore, the door 40 should be controlled to stop moving and perform anti-collision protection. If the vehicle door 40 is executing a door closing command, and the movement time required for the obstacles around the vehicle to move to the dangerous space of the vehicle door 40 is greater than the remaining closing time of the vehicle door 40, continuing to close the vehicle door 40 will not cause the obstacles around the vehicle to collide with the vehicle door 40 or the obstacles around the vehicle to be clamped by the vehicle door 40. Therefore, the vehicle door 40 can be controlled to continue to execute the door closing command.

[0089] In some embodiments of the present invention, the anti-collision protection may include one or more of an alarm prompt, locking the seat belt, locking the vehicle door 40, and reverse opening and closing the door.

[0090] Specifically, please combine Figure 1 The door control system 100 may include an alarm protection device 50, which is connected to the controller 30. The alarm protection device 50 may include a turn signal, a horn, a seat belt, etc. The alarm prompt may include one or more of a sound alarm, a light alarm, and a text prompt. The sound alarm may be implemented through a horn (honking) or a speaker (voice broadcast prompt). The light alarm may be implemented through a turn signal or other lights to warn the user in the vehicle and / or other pedestrians or vehicles in the surrounding environment of the vehicle. The text prompt may be implemented through display screens inside and outside the vehicle.

[0091] During the process of opening and closing the door, if there is a safety hazard in the vehicle's surrounding environment, the seat belt and / or the door 40 can be forcibly locked to fix the user and prevent the user from opening and closing the door. The seat belt and the door 40 can be unlocked after the surrounding environment is confirmed to be safe, thereby preventing accidents and ensuring user safety.

[0092] In some embodiments of the present invention, the vehicle door control system 100 further includes a display 60, and the display 60 is connected to the controller 30, wherein the controller 30 is further used to control the display 60 to dynamically display the relationship of space state changes.

[0093] In this way, the spatial state change relationship of obstacles around the vehicle becomes visible and more vivid, which is conducive to improving the user experience.

[0094] Specifically, the spatial state change relationship can be dynamically displayed by displaying the door panoramic visual system and the simulated three-dimensional environment visual system on the display 60 (eg Figure 6 In this way, during the operation of the vehicle door 40, the user can timely learn about the obstacles around the vehicle through the display 60, and once danger is found, the user can also protect himself in the first time and take anti-collision protection measures manually.

[0095] In some embodiments, the vehicle can be wirelessly connected and communicated with the server, and then the vehicle can download road data, building data, etc. around the current vehicle from the server, and integrate the acquired road data and building data into the door panoramic viewing system, thereby further increasing the authenticity of the door panoramic viewing system.

[0096] In some embodiments, the controller 30 is also used to control the door based on the ultrasonic information of obstacles around the vehicle collected by the vehicle radar 20 when the ambient light is dim, the vehicle camera 10 is blocked by stains, or it is raining or snowing. In addition, at this time, the speed of opening or closing the door 40 can be reduced to prevent the door 40 from colliding with obstacles around the vehicle in the blind spot.

[0097] In some embodiments, the controller 30 is also used to control the door based on the image information of the obstacle around the vehicle collected by the vehicle camera 10 when the obstacle around the vehicle is in the detection blind spot of the vehicle radar 20. When it is detected that the obstacle around the vehicle is in a dangerous space, the door 40 is not opened or closed, and an alarm is issued. If the door is being opened or closed, emergency avoidance is immediately performed to perform anti-collision protection.

[0098] See also Figure 7 The present invention proposes a door control method, the method comprising:

[0099] S11: Acquire image information of obstacles around the vehicle, and acquire ultrasonic information of obstacles around the vehicle;

[0100] S13: fusing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle to obtain a spatial state change relationship between the obstacles around the vehicle and the vehicle door;

[0101] S15: Control the vehicle door according to the spatial state change relationship.

[0102] The vehicle door control method of the embodiment of the present invention controls the vehicle door according to the spatial state change relationship of the obstacles around the vehicle relative to the vehicle door, thereby improving the intelligence level of the vehicle door and enhancing the safety and reliability of the vehicle door.

[0103] It can be understood that, in the related technologies, obstacles around the vehicle are usually identified only based on ultrasonic information of obstacles around the vehicle collected by the vehicle-mounted radar, or only based on image information of obstacles around the vehicle collected by the vehicle-mounted camera. The data collected by the vehicle-mounted radar and the vehicle-mounted camera are not integrated to identify the obstacles around the vehicle based on the integrated data, and the identification of obstacles around the vehicle is not applied to the door control solution.

[0104] Furthermore, in the related technology of identifying obstacles around the vehicle based only on the ultrasonic information of obstacles around the vehicle collected by the vehicle-mounted radar, according to the ultrasonic principle, the vehicle-mounted radar sends ultrasonic waves, and after the ultrasonic waves hit the obstacles around the vehicle, a part of them is reflected back and detected by the vehicle-mounted radar, and the actual distance between the vehicle and the obstacles around the vehicle is calculated based on the detected reflected waves. However, due to the installation height angle, as well as the ultrasonic divergence angle and other issues, the vehicle-mounted radar has a working blind spot, and the vehicle-mounted radar cannot accurately identify the size and contour of the obstacles around the vehicle. In the related technology of identifying obstacles around the vehicle based only on the image information of obstacles around the vehicle collected by the vehicle-mounted camera, the vehicle-mounted camera is used to collect image information for intuitive analysis. However, due to the installation height and angle of the vehicle-mounted camera, after the image information is analyzed, calculated and restored, there is a certain error between the actual obstacle information around the vehicle and the displayed information. In addition, the vehicle-mounted camera is too dependent on environmental factors. For example, at night or in a dimly lit environment, or in rainy and snowy weather, the image information collected by the vehicle-mounted camera is of low quality, which is not conducive to analysis, calculation and restoration. At the same time, if the vehicle-mounted camera is accidentally contaminated or blocked, it will be even more difficult to collect real and reliable image information.

[0105] Moreover, the door control scheme in the related art only focuses on the realization of automatic door opening and closing, and does not recognize obstacles around the vehicle and automatically open and close the door according to the spatial state change relationship of the obstacles around the vehicle. In other words, the door control scheme in the related art has problems such as low intelligence and low reliability.

[0106] The door control method of this embodiment takes into account the advantage that the vehicle-mounted radar is less affected by the environment during operation, for example, it can detect obstacles around the vehicle at night or in dimly lit places, and the advantage that the vehicle-mounted camera can achieve blind spot detection based on the installation position, and fuses and analyzes the image information collected by the vehicle-mounted camera and the ultrasonic information collected by the vehicle-mounted radar, combining the advantages of both parties to compensate for the disadvantages of both parties, thereby ensuring the accuracy of obstacle recognition around the vehicle, so that the door control method can be implemented, and adds a related scheme for judging the spatial state change relationship between the obstacles around the vehicle and the door in the door control method, making the door control method of this embodiment more intelligent, reliable and safe. At the same time, the door control method implemented by the present invention is implemented based on the vehicle-mounted camera and vehicle-mounted radar already installed on the vehicle, and the cost can also be effectively controlled.

[0107] Specifically, the vehicle doors may include casement doors, double-opening doors, side-sliding doors, and the like.

[0108] The vehicle-mounted camera may include multiple cameras, each of which may face a different direction, so as to monitor the environment around the vehicle from multiple angles and collect image information of obstacles around the vehicle. The multiple vehicle-mounted cameras may be arranged to be distributed around the vehicle body, or may be arranged to be gathered on the top of the vehicle. When the multiple vehicle-mounted cameras are gathered on the top of the vehicle, the multiple cameras may be arranged to be fixedly mounted on the top of the vehicle, or may be arranged to be rotatably mounted on the top of the vehicle, and the multiple vehicle-mounted cameras may be arranged to rotate synchronously, or may be arranged to rotate individually.

[0109] The vehicle-mounted camera may also include one, and one vehicle-mounted camera may be arranged to be rotatably mounted on the top of the vehicle, so as to achieve the effect of monitoring the environment around the vehicle. The vehicle-mounted camera may shoot obstacles around the vehicle within its own direction range at a preset sampling frequency to obtain image information of obstacles around the vehicle, or may shoot obstacles around the vehicle within its own direction range in real time by recording to obtain image information of obstacles around the vehicle.

[0110] There may be multiple vehicle-mounted radars, each of which may be oriented in a different direction, so as to detect the environment around the vehicle from multiple angles and collect ultrasonic information of obstacles around the vehicle.

[0111] The obstacles around the vehicle may be movable persons or objects around the vehicle, such as pedestrians, vehicles, animals, etc. The spatial state change relationship of the obstacles around the vehicle can be understood as the change relationship of the spatial position of the obstacles around the vehicle relative to the vehicle door.

[0112] In some embodiments, before step S11, the door control method further includes: detecting whether a door opening and closing instruction is received; if a door opening and closing instruction is received, entering step S11, step S13 and step S15. In this way, computing space is saved and power consumption is reduced. The door opening and closing instruction can be used to indicate that the door is opened or to indicate that the door is closed. In some embodiments, the vehicle may include a voice recognition component, which can analyze the received user voice to determine whether the user voice includes a door opening and closing instruction. In some embodiments, the vehicle may include an input component, and the user can select to open and close the door on the input component to generate a door opening and closing instruction.

[0113] In some embodiments, before step S11, the door control method further includes: detecting the vehicle speed information and gear information of the vehicle; when the vehicle speed information and gear information of the vehicle meet the preset conditions, entering step S11, step S13 and step S15. In this way, it is ensured that the door control method can be executed in time, thereby being more intelligent and improving the user experience. Specifically, the preset conditions may include that the vehicle speed is less than or equal to the preset speed (for example, 3km / h), and the gear is the preset gear (for example, neutral gear, parking gear). It can be understood that when the vehicle speed is greater than the preset speed or the gear is the forward gear or the reverse gear, the user may not intend to open or close the door, and at this time there is no need to enter step S11, step S13 and step S15.

[0114] In one example, the process of fusing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle is as follows: Figure 2 As shown, by processing the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle acquired at multiple moments, the spatial state change relationship between the obstacles around the vehicle and the door can be determined.

[0115] See also Figure 8 In some embodiments of the present invention, step S13 includes:

[0116] S131: Establishing a spatial rectangular coordinate system according to the positional relationship between the vehicle-mounted camera and the vehicle-mounted radar;

[0117] S133: determining the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system according to the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle;

[0118] S135: Determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door according to the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system.

[0119] In this way, the spatial position of obstacles around the vehicle relative to the door can be determined more accurately.

[0120] Specifically, the spatial state change relationship can be divided based on different spatial regions, so that after determining the three-dimensional coordinates of the obstacles around the vehicle, the spatial region where the obstacles around the vehicle are located can be determined, and then the spatial state change relationship of the obstacles around the vehicle can be quickly determined.

[0121] In some embodiments of the present invention, the vehicle-mounted radar includes a first radar and a second radar, and the vehicle-mounted camera, the first radar and the second radar are located on the same straight line. Step S131 includes: establishing a spatial rectangular coordinate system with the vehicle-mounted camera as the origin and the straight line as the x-axis.

[0122] In this way, it is easy to determine the three-dimensional coordinates of obstacles around the vehicle.

[0123] Specifically, please combine Figure 3 , take the vehicle camera O as the origin, the straight line where the vehicle camera O, the first radar L1 and the second radar L2 are located as the x-axis, the x-axis is basically parallel to the horizontal plane, the straight line passing through the vehicle camera O and perpendicular to the horizontal plane is the y-axis, and the straight line passing through the vehicle camera O and parallel to the horizontal plane and perpendicular to the x-axis is the z-axis, to establish a spatial rectangular coordinate system.

[0124] See also Fig. 9 In some embodiments of the present invention, the three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component and a z-axis component, and step S133 includes:

[0125] S1331: Determine, based on the ultrasonic information collected by the first radar and the ultrasonic information collected by the second radar, a projection length of a line connecting the first radar and obstacles around the vehicle on the x-axis;

[0126] S1332: Determine a first x-axis component according to a distance between the first radar and the vehicle-mounted camera and a projection length;

[0127] S1333: Determine the two-dimensional coordinates of obstacles around the vehicle in the two-dimensional imaging image according to the image information;

[0128] S1334: Determine a first y-axis component and a z-axis component according to the first x-axis component and the two-dimensional coordinate;

[0129] S1335: Use the first x-axis component, the first y-axis component, and the z-axis component as the three-dimensional coordinates of the obstacles around the vehicle.

[0130] Please combine again Figure 3 , since the positions of the vehicle-mounted camera O, the first radar L1, and the second radar L2 are fixed in advance, the distance a1 between the vehicle-mounted camera O and the first radar L1 is a known quantity, and the distance a2 between the first radar L1 and the second radar L2 is also a known quantity. The distance b1 between the first radar L1 and the obstacle W around the vehicle can be determined based on the ultrasonic information collected by the first radar L1. The distance b2 between the second radar L2 and the obstacle W around the vehicle can be determined based on the ultrasonic information collected by the second radar L2.

[0131] In step S1331, in the triangle formed by the first radar L1, the second radar L2 and the obstacle W around the vehicle, since the lengths of the three sides of the triangle a2, b1 and b2 are all known quantities, the projection length a3 of the line L1W connecting the first radar L1 and the obstacle W around the vehicle on the x-axis, that is, the length of the line segment L1H, can be calculated based on the Pythagorean theorem.

[0132] In step S1332, the sum of the distance a1 between the first radar L1 and the vehicle-mounted camera O and the projection length a3 is taken as the first x-axis component x1.

[0133] In step S1333, the two-dimensional imaging image can be understood as a two-dimensional image generated according to the external light signal, i.e., image information, received by the vehicle-mounted camera within the actual shooting size range. When the obstacles around the vehicle in the actual space are within the shooting size range of the vehicle-mounted camera, the obstacles around the vehicle can be presented in the two-dimensional imaging image, and the two-dimensional coordinates corresponding to the obstacles around the vehicle in the two-dimensional imaging image are known. The size of the two-dimensional imaging image includes a first long side and a first wide side, wherein the first long side corresponds to the horizontal plane, and the first wide side is perpendicular to the first long side. An x-axis is established according to the first long side of the two-dimensional imaging image, and a y-axis is established according to the first wide side of the two-dimensional imaging image, so that the second x-axis component x0 and the second y-axis component y0 of the obstacles around the vehicle can be determined according to the position of the obstacles around the vehicle in the two-dimensional imaging image, thereby determining the two-dimensional coordinates of the obstacles around the vehicle as (x0, y0). In some embodiments, the size of the two-dimensional imaging image is the size of the display screen.

[0134] See also Fig.10 In some embodiments of the present invention, the two-dimensional coordinate includes a second x-axis component and a second y-axis component, and step S1334 includes:

[0135] S13341: Determine a proportional relationship between a shooting size of the vehicle-mounted camera and a size of the two-dimensional imaging image according to the first x-axis component and the second x-axis component;

[0136] S13342: Determine the first y-axis component according to the proportional relationship and the second y-axis component;

[0137] S13343: Determine the z-axis component according to the proportional relationship.

[0138] It can be understood that the shooting size of the vehicle-mounted camera includes the second long side and the second wide side, wherein the second long side corresponds to the horizontal plane, and the second wide side is perpendicular to the second long side. When the parameters and installation position of the vehicle-mounted camera are determined, there is a proportional relationship between the shooting size of the vehicle-mounted camera and the distance from the obstacles around the vehicle to the x-axis and the size of the two-dimensional imaging image. For example, when the parameters and installation position of the vehicle-mounted camera are determined, when the size of the display screen, i.e., the size of the two-dimensional imaging image, is 8 inches, the external environment at a distance of 5 meters from the x-axis is shot, and the shooting size is 13*9.8 meters. The range is displayed on the full screen on the display screen; the external environment at a distance of 10 meters from the x-axis is shot, and the shooting size is 26*19.5 meters. The range is displayed on the full screen on the display screen. Therefore, the proportional relationship between the shooting size of the vehicle-mounted camera and the size of the two-dimensional imaging image can be determined according to the ratio of the first x-axis component to the second x-axis component.

[0139] Furthermore, after determining the proportional relationship between the shooting size of the vehicle-mounted camera and the size of the two-dimensional imaging image, the first y-axis component y1 and the z-axis component z1 can be determined according to the proportional relationship, thereby determining the three-dimensional coordinates of the obstacles around the vehicle as (x1, y1, z1).

[0140] See also Fig.11 In some embodiments of the present invention, step S135 includes:

[0141] S1351: Determine the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system, and calculate the distance between the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system and the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system;

[0142] S1353: Determine the spatial state change relationship between the obstacles around the vehicle and the door according to the distance.

[0143] In this way, the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the door are determined based on the same spatial rectangular coordinate system, which makes it easy to determine the distance between the obstacles around the vehicle and the door according to the three-dimensional coordinates, and then determine the spatial state change relationship between the obstacles around the vehicle and the door.

[0144] Specifically, the three-dimensional coordinates of the door can be determined according to the installation position of the door. In some embodiments, the coordinates of the center point of the door frame can be used as the three-dimensional coordinates of the door. In an example, the three-dimensional coordinates of the obstacle around the vehicle are (x1, y1, z1), and the three-dimensional coordinates of the door are (x2, y2, z2). The distance D between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door can be calculated by solving the following formula: D 2 =(x1-x2) 2 +(y1-y2) 2 +(z1-z2) 2 .

[0145] See also Fig.12 In some embodiments of the present invention, step S1353 includes:

[0146] S13531: When the distance is less than a first threshold, determining that the obstacle around the vehicle is in a dangerous space of the door;

[0147] S13532: When the distance is greater than a second threshold, determining that the obstacle around the vehicle is in a safe space of the door;

[0148] S13533: When the distance is greater than or equal to the first threshold and less than or equal to the second threshold, it is determined that the obstacles around the vehicle are in the critical space of the door, and the first threshold is less than the second threshold.

[0149] In this way, after determining the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the vehicle door, by calculating the distance between the obstacles around the vehicle and the vehicle door, the spatial state change relationship of the obstacles around the vehicle relative to the vehicle door can be quickly determined.

[0150] Specifically, the spatial state change relationship may include a dangerous space at the vehicle door, a safe space at the vehicle door, and a critical space at the vehicle door.

[0151] The dangerous space of the door can be understood as the entire travel space of the door during the process of opening and closing the door. The obstacles around the vehicle within this space are effective obstacles around the vehicle. No matter whether the obstacles around the vehicle are in a moving state or a stationary state, the door cannot be opened and closed.

[0152] The safe space of the door can be understood as a space area that does not affect the opening and closing of the door. The obstacles around the vehicle in this space area are invalid obstacles around the vehicle, and there is no need to worry about whether the obstacles around the vehicle are in a moving state or a stationary state.

[0153] The critical space of the door is the space area between the safe space and the dangerous space. The obstacles around the vehicle in this space area are uncertain. If the obstacles around the vehicle in this space area have a tendency to move toward the dangerous space, they can be determined as valid obstacles around the vehicle. If the obstacles around the vehicle in this space area have a tendency to move toward the safe space, they can be determined as invalid obstacles around the vehicle.

[0154] In an example, the first threshold is 1m and the second threshold is 2m, that is, when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the vehicle door is less than 1m, it can be determined that the obstacles around the vehicle are in the dangerous space of the vehicle door; when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the vehicle door is greater than 2m, it can be determined that the obstacles around the vehicle are in the safe space of the vehicle door; when the distance between the three-dimensional coordinates of the obstacles around the vehicle and the three-dimensional coordinates of the vehicle door is greater than or equal to 1m and less than or equal to 2m, it can be determined that the obstacles around the vehicle are in the critical space of the vehicle door.

[0155] In some embodiments, the first threshold and the second threshold may include a tolerance range (e.g., ±20cm, ±30cm). In this way, the safety of the door when it is opened and closed can be further improved. For example, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is less than 1m±20cm, it can be determined that the obstacle around the vehicle is in the dangerous space of the door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is greater than 2m±20cm, it can be determined that the obstacle around the vehicle is in the safe space of the door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is greater than or equal to 1m±20cm and less than or equal to 2m±20cm, it can be determined that the obstacle around the vehicle is in the critical space of the door.

[0156] See also Fig.13 In some embodiments of the present invention, step S15 includes:

[0157] S151: When obstacles around the vehicle are in a dangerous space of the door and the door is in a stopped state, controlling the door to prohibit executing the received door opening and closing command;

[0158] S152: When obstacles around the vehicle are in the dangerous space of the door and the door is in motion, the door is controlled to stop moving and perform anti-collision protection;

[0159] S153: When the obstacles around the vehicle are in the safety space of the door and the door is in a stopped state, controlling the door to execute the received door opening and closing command;

[0160] S154: When the obstacles around the vehicle are in the safety space of the door and the door is in motion, controlling the door to execute the received door opening and closing command;

[0161] S155: When the obstacles around the vehicle are in the critical space of the door and the door is in a stopped state, controlling the door to delay the execution of the received door opening and closing command and perform anti-collision protection;

[0162] S156: When the obstacles around the vehicle are in the critical space of the door and the door is in motion, the door is controlled according to the movement time required for the obstacles around the vehicle to move to the dangerous space.

[0163] In this way, the relationship between the spatial state change and the state of the door is comprehensively considered, and the door control is performed according to the relationship between the spatial state change and the state of the door, which can make the opening and closing of the door more intelligent and improve safety at the same time. It can be understood that in the relevant technology, the door opening or closing action is immediately executed after receiving the door opening and closing command. If the door opening and closing command is issued by the user without paying attention to the vehicles and pedestrians in the front and rear, the direct door opening at this time may collide with pedestrians or vehicles, causing serious traffic accidents. In the door control method of the embodiment of the present invention, after receiving the door opening and closing command, the spatial state change relationship of the obstacles around the vehicle is first actively identified, and then it is determined whether to execute the received door opening and closing command based on the identification result, which can effectively warn the user in advance to prevent collisions caused by direct door opening and closing, and ensure user safety.

[0164] Specifically, the stop state may be a state where the door remains stationary after being fully opened, or a state where the door remains stationary after being fully closed, or a state where the door remains stationary at any travel position between fully opened and fully closed. The moving state may be a state where the door gradually opens, or a state where the door gradually closes. The moving state of the door may be generated by the door automatically moving according to a door opening and closing instruction, or by the user manually operating the door.

[0165] Further, step S151 can be understood as, when the obstacles around the vehicle are in the dangerous space of the door and the door is in a stopped state, if the door opening and closing command is detected, it will not be executed. Step S152 can be understood as, when the obstacles around the vehicle are in the dangerous space of the door, if the door is opening or closing, the door opening or closing action is stopped immediately and anti-collision protection is performed to prevent the user's personal safety and property from being harmed due to the continued opening or closing of the door.

[0166] Step S153 and step S154, that is to say, when the obstacles around the vehicle are in the safe space of the door, no matter the door is in a stopped state or in a moving state, the door can be opened and closed normally according to the received door opening and closing instructions.

[0167] Step S155 can be understood as, when the obstacles around the vehicle are in the critical space of the door and the door is in a stopped state, if a door opening and closing command is received, that is, the user intends to open and close the door, it will not be executed, and anti-collision protection will be performed at the same time, waiting for the obstacles around the vehicle to enter the safe space before executing the door opening and closing command.

[0168] Step S156 can be understood as, when the obstacles around the vehicle are in the critical space of the door, if the door is being opened or closed, how to control the door can be further determined based on the movement time required for the obstacles around the vehicle to move to the dangerous space of the door.

[0169] In some embodiments of the present invention, the vehicle doors are controlled according to the movement time required for obstacles around the vehicle to move to the dangerous space of the vehicle doors, including: when the vehicle doors are executing a door closing command and the movement time is less than or equal to the remaining closing time of the vehicle doors, the vehicle doors are controlled to stop moving and perform anti-collision protection; when the vehicle doors are executing a door closing command and the movement time is greater than the remaining closing time of the vehicle doors, the vehicle doors are controlled to continue executing the door closing command.

[0170] In this way, during the process of closing the car door, while ensuring safety, the door closing command is completed as much as possible, thereby avoiding the user waiting due to the inability to close the door and preventing the user's schedule from being delayed.

[0171] Specifically, please combine Figure 4 According to the ultrasonic information of the obstacles around the vehicle collected by the vehicle-mounted radar at different times, it is possible to infer whether the obstacles around the vehicle in the image information are the same obstacles around the vehicle, and the movement trend of the obstacles around the vehicle relative to the door. For example, if the distance between the obstacles around the vehicle detected by the radar and the vehicle at different times remains unchanged, it can be considered that the obstacles around the vehicle are in a stationary state; if the distance between the obstacles around the vehicle detected by the radar and the vehicle gradually decreases, it can be considered that the movement trend of the obstacles around the vehicle is approaching the door; if the distance between the obstacles around the vehicle detected by the radar and the vehicle gradually increases, it can be considered that the movement trend of the obstacles around the vehicle is moving away from the door.

[0172] Please combine Figure 5 According to the image information of the obstacles around the vehicle collected by the vehicle-mounted camera at different times, it is possible to infer whether the obstacles around the vehicle in the image information are the same obstacles around the vehicle, and the movement trend of the obstacles around the vehicle relative to the door. For example, multiple groups of two-dimensional imaging images can be obtained by using multiple vehicle-mounted cameras installed at different positions, and image analysis can be performed on the multiple groups of two-dimensional imaging images. Each group of two-dimensional imaging images is divided into multiple grids, and the outline of the obstacles around the vehicle in each group of two-dimensional imaging images is plotted. If the area of ​​the outline of the obstacle around the vehicle remains unchanged, it can be considered that the obstacles around the vehicle are in a stationary state; if the area of ​​the outline of the obstacle around the vehicle gradually increases, it can be considered that the movement trend of the obstacles around the vehicle is close to the door; if the area of ​​the outline of the obstacle around the vehicle gradually decreases, it can be considered that the movement trend of the obstacles around the vehicle is away from the door.

[0173] Further, in combination with the movement trend of the obstacles around the vehicle and the estimated movement speed of the obstacles around the vehicle, the movement time required for the obstacles around the vehicle to move to the dangerous space of the door can be calculated. The current position of the door and the closing speed of the door can be determined by the relevant sensor data and the current data, and then the remaining closing time of the door can be determined according to the current position of the door and the closing speed of the door. If the door is executing the door closing command, and the movement time required for the obstacles around the vehicle to move to the dangerous space of the door is less than or equal to the remaining closing time of the door, then continuing to close the door will cause the obstacles around the vehicle to collide with the door or the obstacles around the vehicle to be clamped by the door, so the door should be controlled to stop moving and perform anti-collision protection. If the door is executing the door closing command, and the movement time required for the obstacles around the vehicle to move to the dangerous space of the door is greater than the remaining closing time of the door, then continuing to close the door will not cause the obstacles around the vehicle to collide with the door or the obstacles around the vehicle to be clamped by the door, so the door can be controlled to continue to execute the door closing command.

[0174] In some embodiments of the present invention, the anti-collision protection may include one or more of an alarm prompt, locking a seat belt, locking a door, and reverse opening and closing of a door.

[0175] Specifically, the alarm prompt may include one or more of a sound alarm, a light alarm, and a text prompt. The sound alarm may be implemented through a horn (honking a whistle) or a speaker (voice broadcast prompt). The light alarm may be implemented through a turn signal or other lights to warn the user in the vehicle and / or other pedestrians or vehicles in the surrounding environment of the vehicle. The text prompt may be implemented through a display screen inside and outside the vehicle.

[0176] During the process of opening and closing the door, if there is a safety hazard in the vehicle's surrounding environment, the seat belt and / or the door can be forcibly locked to fix the user and prevent the user from opening and closing the door. The seat belt and door can be unlocked after the surrounding environment is confirmed to be safe, thereby preventing accidents and ensuring user safety.

[0177] See also Fig.14 In some embodiments of the present invention, after step S13, the method further includes:

[0178] S17: Dynamically display the spatial state change relationship.

[0179] In this way, the spatial state change relationship of obstacles around the vehicle becomes visible and more vivid, which is conducive to improving the user experience.

[0180] Specifically, the spatial state change relationship can be dynamically displayed by displaying the door panoramic visual system and the simulated three-dimensional environment visual system on the display (such as Figure 6In this way, during the operation of the door, the user can timely understand the obstacles around the vehicle through the display, and once danger is found, the user can also protect himself in the first time and take anti-collision protection measures manually.

[0181] In some embodiments, the vehicle can be wirelessly connected and communicated with the server, and then the vehicle can download road data, building data, etc. around the current vehicle from the server, and integrate the acquired road data and building data into the door panoramic viewing system, thereby further increasing the authenticity of the door panoramic viewing system.

[0182] In some embodiments, the door control method further includes: in the case of dim ambient light, or the vehicle camera is blocked by stains, or it is raining or snowing, the door is controlled based on the ultrasonic information of obstacles around the vehicle collected by the vehicle radar. In addition, at this time, the speed of opening or closing the door can be reduced to prevent the door from colliding with obstacles around the vehicle in the blind spot.

[0183] In some embodiments, the door control method further includes: when the obstacle around the vehicle is in the detection blind spot of the vehicle-mounted radar, the door control is performed based on the image information of the obstacle around the vehicle collected by the vehicle-mounted camera. When it is detected that the obstacle around the vehicle is in a dangerous space, the door opening and closing operation is not performed, and an alarm prompt is issued. If the door is being opened or closed, emergency avoidance is immediately performed to perform anti-collision protection.

[0184] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0185] In addition, the terms "first", "second", etc. used in the embodiments of the present invention are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present invention can explicitly or implicitly indicate that the embodiment includes at least one of the features. In the description of the present invention, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0186] In the present invention, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected" and "fixed" etc. in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can be understood that it can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements, or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific implementation situation.

[0187] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A door control system, characterized in that: include: A vehicle-mounted camera, which is used to collect image information of obstacles around the vehicle; A vehicle-mounted radar, the vehicle-mounted radar is used to collect ultrasonic information of obstacles around the vehicle; a controller, the controller being connected to the vehicle-mounted camera and the vehicle-mounted radar respectively, the controller being used to fuse the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, obtain the spatial state change relationship between the obstacles around the vehicle and the vehicle door, and control the vehicle door according to the spatial state change relationship; The vehicle-mounted radar includes a first radar and a second radar, and the vehicle-mounted camera, the first radar and the second radar are located on the same straight line, and the controller is further used to establish a spatial rectangular coordinate system with the vehicle-mounted camera as the origin and the straight line as the x-axis; The three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component and a z-axis component, wherein the controller is further used to determine the projection length of the line connecting the first radar and the obstacle around the vehicle on the x-axis according to the ultrasonic information collected by the first radar and the ultrasonic information collected by the second radar, and determine the first x-axis component according to the distance between the first radar and the vehicle-mounted camera and the projection length, and determine the two-dimensional coordinates of the obstacle around the vehicle in the two-dimensional imaging image according to the image information, and determine the first y-axis component and the z-axis component according to the first x-axis component and the two-dimensional coordinates, and use the first x-axis component, the first y-axis component and the z-axis component as the three-dimensional coordinates of the obstacle around the vehicle.

2. The door control system according to claim 1, characterized in that: The controller is also used to establish a spatial rectangular coordinate system based on the positional relationship between the vehicle-mounted camera and the vehicle-mounted radar, and to determine the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system based on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, and to determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door based on the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system.

3. The door control system according to claim 2, characterized in that: The controller is also used to determine the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system, calculate the distance between the three-dimensional coordinates of the obstacles around the vehicle in the spatial rectangular coordinate system and the three-dimensional coordinates of the vehicle door in the spatial rectangular coordinate system, and determine the spatial state change relationship between the obstacles around the vehicle and the vehicle door based on the distance.

4. The door control system according to claim 3, characterized in that: The controller is also used to: When the distance is less than a first threshold, determining that the obstacle around the vehicle is in a dangerous space of the door; When the distance is greater than a second threshold, determining that the obstacle around the vehicle is in the safety space of the door; When the distance is greater than or equal to a first threshold and less than or equal to a second threshold, it is determined that the obstacle around the vehicle is in the critical space of the door, and the first threshold is less than the second threshold.

5. The door control system according to claim 4, characterized in that: The controller is also used to: When obstacles around the vehicle are in a dangerous space of the door and the door is in a stopped state, controlling the door to prohibit executing a received door opening and closing instruction; When obstacles around the vehicle are in a dangerous space of the door and the door is in motion, controlling the door to stop moving and performing anti-collision protection; When the obstacles around the vehicle are in the safety space of the door and the door is in the stopped state, controlling the door to execute the received door opening and closing instruction; When the obstacles around the vehicle are in the safety space of the door and the door is in the moving state, controlling the door to execute the received door opening and closing instruction; When the obstacle around the vehicle is in the critical space of the door and the door is in a stopped state, controlling the door to postpone the execution of the received door opening and closing command and perform anti-collision protection; When the obstacle around the vehicle is in the critical space of the door and the door is in motion, the door is controlled according to the movement time required for the obstacle around the vehicle to move to the dangerous space of the door.

6. The door control system according to claim 5, characterized in that: The controller is also used to: When the vehicle door is executing a door closing instruction and the movement duration is less than or equal to the remaining closing duration of the vehicle door, controlling the vehicle door to stop moving and performing anti-collision protection; When the vehicle door is executing the door closing command and the movement duration is greater than the remaining closing duration of the vehicle door, the vehicle door is controlled to continue executing the door closing command.

7. The door control system according to claim 1, characterized in that: The two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller is further configured to: The proportional relationship between the shooting size of the vehicle-mounted camera and the size of the two-dimensional imaging image is determined according to the first x-axis component and the second x-axis component, and the first y-axis component is determined according to the proportional relationship and the second y-axis component, and the z-axis component is determined according to the proportional relationship.

8. The door control system according to claim 1, characterized in that: It also includes a display, which is connected to the controller, wherein the controller is further used to control the display to dynamically display the relationship of the spatial state changes.

9. A door control method, applied to the door control system according to any one of claims 1 to 8, characterized in that: The method comprises: Acquire image information of obstacles around the vehicle, and acquire ultrasonic information of the obstacles around the vehicle; fusing the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle to obtain a spatial state change relationship between the obstacle around the vehicle and the vehicle door; The vehicle door is controlled according to the spatial state change relationship.

Images