Vehicle door control method and program product

Abstract

The invention discloses a vehicle door control method and a program product, and the method comprises the steps: collecting a look-around aerial view of a target vehicle based on a look-around shooting device, collecting a vehicle side view of the target vehicle based on a side view shooting device, and collecting environment point cloud data of the target vehicle through a target sensor; determining type information and image position information of an obstacle in the environment where the target vehicle is located according to the look-around aerial view and the vehicle side view; determining a relative distance, a relative speed and a relative angle between the obstacle and a vehicle door according to the image position information, the type information and the environmental point cloud data of the obstacle and the vehicle door position information of the target vehicle door, wherein the relative angle is a normal included angle between the moving direction of the obstacle and the vehicle door; and in response to an opening request for the vehicle door, opening of the vehicle door is controlled according to the type information of the obstacle and the relative distance, the relative speed and the relative angle between the obstacle and the vehicle door, so that automatic obstacle avoidance in the vehicle door opening process is achieved.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to a door control method and program product. Background Technology

[0002] With the continuous growth of car ownership, vehicles often face interference from pedestrians, non-motorized vehicles, and other obstacles when opening their doors in densely parked scenarios such as parking lots and roadside parking spaces. Collisions caused by opening car doors can not only cause damage to the vehicle's exterior and obstacles, but also lead to safety accidents such as pedestrian injuries. Therefore, how to accurately detect obstacles within the door opening range and avoid collision risks has become a key issue in vehicle safety design.

[0003] Currently, vehicle occupants rely on observing their surroundings through car windows to determine the presence of obstacles. However, this manual observation method has significant limitations. The occupant's field of vision is easily restricted by factors such as the window's field of view, seat position, and weather conditions, making it difficult to fully cover potential risk areas when the door opens. Furthermore, manual judgment depends on the occupant's attention and reaction speed, which can lead to subjective misjudgments or negligence, especially when occupants are rushing to get in and out of the vehicle, significantly increasing safety risks. Therefore, there is an urgent need for a door control method to achieve accurate obstacle detection and improve the safety and reliability of the door opening process. Summary of the Invention

[0004] This invention provides a vehicle door control method and program product to solve the problem in related technologies where relying on the vehicle occupants' own observation can easily lead to incomplete observation, resulting in the risk of collision between the door and obstacles when it opens.

[0005] According to one aspect of the present invention, a door control method is provided, the method comprising:

[0006] The system acquires a bird's-eye view of the target vehicle using a surround-view camera, acquires a side view of the target vehicle using a side-view camera, and acquires environmental point cloud data of the target vehicle using a target sensor.

[0007] Based on the surround-view bird's-eye view and the vehicle side view, determine the type information of obstacles in the environment where the target vehicle is located and the image location information of the obstacles;

[0008] The relative distance, relative speed, and relative angle between the obstacle and the door are determined based on the image location information, type information, environmental point cloud data, and door location information of the target vehicle door. The relative angle is the angle between the moving direction of the obstacle and the normal of the door.

[0009] In response to a request to open the vehicle door, the opening of the vehicle door is controlled based on the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle.

[0010] According to another aspect of the present invention, a door control device is provided, the device comprising:

[0011] The target vehicle data acquisition module is used to acquire a bird's-eye view of the target vehicle based on the surround-view shooting device, acquire a side view of the target vehicle based on the side-view shooting device, and acquire environmental point cloud data of the target vehicle through the target sensor.

[0012] The obstacle type and image location determination module is used to determine the type information of obstacles and the image location information of obstacles in the environment where the target vehicle is located based on the surround-view bird's-eye view and the vehicle side view;

[0013] The obstacle relative information determination module is used to determine the relative distance, relative speed and relative angle between the obstacle and the door based on the image position information, type information, environmental point cloud data and door position information of the target vehicle door, wherein the relative angle is the angle between the moving direction of the obstacle and the normal of the door;

[0014] A door control opening module is used to respond to an opening request for the door by controlling the opening of the door based on the type information of the obstacle, the relative distance between the obstacle and the door, the relative speed, and the relative angle.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the door control method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the door control method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, embodiments of the present disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the door control method as described in any of the embodiments of the present disclosure.

[0021] The technical solution of this invention firstly involves acquiring a bird's-eye view of the target vehicle using a surround-view imaging device, acquiring a side view of the target vehicle using a side-view imaging device, and acquiring environmental point cloud data of the target vehicle using a target sensor. This multi-source data acquisition avoids the limitations of a single device's field of view or dimensionality, allowing for mutual supplementation and providing a richer data foundation for subsequent obstacle recognition. Next, the type information and image location information of obstacles in the target vehicle's environment are determined based on the bird's-eye view and the vehicle's side view. This allows for rapid location of the approximate location of obstacles using the bird's-eye view and accurate identification of obstacle types using the side view, improving recognition accuracy and simultaneously acquiring obstacle type and image location information. Finally, based on the obstacle's image location information, type information, and environmental point cloud data... The relative distance, relative speed, and relative angle between the obstacle and the door are determined based on the door position information of the target vehicle. The relative angle is the angle between the moving direction of the obstacle and the normal direction of the door. The relative information is calculated by integrating multi-dimensional information to reduce environmental interference and improve the accuracy of the relative information calculation, providing key quantitative indicators for subsequent risk assessment. Finally, in response to a request to open the door, the door is opened based on the obstacle type information, the relative distance, relative speed, and relative angle between the obstacle and the door. By integrating obstacle type, relative distance, relative speed, relative angle, and other multi-dimensional factors, the door opening control is made more aligned with actual safety scenarios. Through multi-dimensional data fusion and quantitative analysis, the door opening control is made more scientific and safer, effectively ensuring the safety of vehicle occupants and the surrounding environment.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1This is a flowchart of a door control method provided according to Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of a door control method provided according to Embodiment 2 of the present invention;

[0026] Figure 3 This is a flowchart of a door control method for an embodiment of the present invention, provided in Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a door control device according to Embodiment 4 of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the door control method of this invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0034] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0037] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0038] Example 1

[0039] Figure 1 This is a flowchart of a door control method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where obstacle avoidance is achieved during the door opening process. The method can be executed by a door control device, which can be implemented in hardware and / or software, optionally through an electronic device, such as a mobile terminal, PC, or server. Figure 1 As shown, the method may specifically include:

[0040] S110. Acquire a bird's-eye view of the target vehicle based on the surround-view shooting device, acquire a side view of the target vehicle based on the side-view shooting device, and acquire environmental point cloud data of the target vehicle through the target sensor.

[0041] In this embodiment of the invention, the target vehicle can be a vehicle that requires monitoring of obstacles around the vehicle to ensure the safety of the door opening process. The surround-view imaging device can be a device installed around the target vehicle, capable of acquiring 360° environmental images around the vehicle. The surround-view bird's-eye view can be a two-dimensional image of the vehicle and its surrounding environment generated from a top-down perspective after processing multiple sets of images around the vehicle acquired by the surround-view imaging device through image stitching, perspective transformation, and other techniques, which can intuitively show the distribution of obstacles around the vehicle. The side-view imaging device refers to a device installed on the side of the target vehicle to acquire images of the side environment of the vehicle. The vehicle side view can be an image of the side environment of the vehicle acquired directly by the side-view imaging device or after simple processing, presenting the side environment of the vehicle from a side-view perspective, which can clearly show obstacles on the side of the vehicle. The target sensor can be a device used to acquire point cloud data of the environment around the target vehicle. Environmental point cloud data can be understood as a dataset composed of a large number of three-dimensional coordinate points generated after the target sensor emits detection signals to the surrounding environment and receives reflected signals.

[0042] To ensure the consistency of data collection time for the target vehicle's surrounding environment and avoid environmental perception errors caused by timing discrepancies in data collection from different sensors, a hardware synchronization trigger signal can be sent to all sensors on the vehicle via a domain controller or high-performance electronic control unit (ECU). This assigns a unified timestamp to each frame of data collected by each sensor, enabling synchronized data collection from all sensors and providing a precise time reference for subsequent multi-source data fusion.

[0043] Furthermore, the vehicle's surrounding environment information can be collected based on the surround-view and side-view shooting devices. The images collected by the surround-view shooting devices can be preprocessed, such as stitching and distortion correction, to generate an environmental bird's-eye view of the target vehicle. The images collected by the side-view shooting devices can be preprocessed, such as distortion correction, to obtain a side view of the vehicle. At the same time, the environmental point cloud data of the target vehicle can be collected through the target sensor, and the three-dimensional spatial position information of objects in the environment can be obtained after preprocessing such as filtering and coordinate transformation.

[0044] Optionally, the target sensor can be a Direct Time of Flight (DTOF) sensor, which has strong environmental adaptability and can maintain stable and high-precision ranging capabilities under extreme visual conditions such as night, underground parking garages, tunnels, rain, snow, strong light, and backlight. It can be combined with the rich semantic information provided by the imaging device's sensor under good lighting conditions, and DTOF can make up for visual disadvantages under poor lighting conditions. The two complement each other to ensure the stability and reliability of obstacle detection.

[0045] S120. Determine the type information of obstacles in the environment where the target vehicle is located and the image location information of the obstacles based on the surround-view bird's-eye view and the vehicle side view.

[0046] The obstacle can be any object located in the target vehicle's environment that may obstruct the opening of the vehicle door or pose a collision risk. Obstacles can include, but are not limited to, pedestrians, non-motorized vehicles, motorized vehicles, and fixed structures. Type information can be attribute information obtained by classifying obstacles, such as categorizing them as pedestrians, motorized vehicles, non-motorized vehicles, or fixed obstacles. Image location information can be information used to identify the location of obstacles in a surround-view or vehicle side view, and can be presented in the form of pixel coordinates, bounding boxes, etc.

[0047] Based on the above scheme, optionally, determining the type information and image position information of obstacles in the environment where the target vehicle is located based on the surround-view bird's-eye view and the vehicle side view includes: determining the two-dimensional bounding boxes of obstacles in the surround-view bird's-eye view and the image coordinates of obstacles in the surround-view bird's-eye view based on the surround-view bird's-eye view and the first obstacle detection network; mapping the two-dimensional bounding boxes of obstacles to the vehicle side view based on the image coordinates of obstacles in the surround-view bird's-eye view to obtain a target environment map; and determining the type information and image position information of obstacles in the target environment map based on the target environment map and the second obstacle detection network.

[0048] The first obstacle detection network can be a pre-trained deep learning model for identifying and locating obstacles from a surround-view bird's-eye view. The two-dimensional bounding box can be a rectangular box used by the first obstacle detection network to select the area containing the obstacle in the surround-view bird's-eye view, reflecting the size and location range of the obstacle. The image coordinates can be the pixel coordinates of feature points in the surround-view bird's-eye view used to identify the obstacle, such as the obstacle center and bounding box vertices. The target environment map can be a side view image obtained by mapping the two-dimensional bounding boxes of the obstacles in the surround-view bird's-eye view to the vehicle side view, fusing obstacle location information. The second obstacle detection network can be a pre-trained deep learning model for identifying obstacle types and determining their image location information from the target environment map. The model structure of the second obstacle detection network can be the same as or different from that of the first obstacle detection network, and can be flexibly configured according to actual business recognition needs.

[0049] Optionally, the surround-view bird's-eye view can be input into the first obstacle detection network, outputting the two-dimensional bounding boxes and image coordinates of obstacles in the target vehicle's environment within the surround-view bird's-eye view, thus achieving a rough pixel-level localization of the obstacles. Further, a preset homography transformation matrix can be used to map the two-dimensional bounding boxes onto the vehicle's side view based on the obstacle's image coordinates, obtaining a target environment map. Based on the center point coordinates of the mapped two-dimensional bounding boxes in the target environment map, a preset-sized pixel region is cropped from the target environment map. This pixel region is then processed and analyzed by the second obstacle detection network, outputting obstacle type information and the obstacle's image position information in the target environment map, as well as its corresponding confidence level. The preset homography transformation matrix can be pre-determined using calibration parameters between the surround-view and side-view imaging devices.

[0050] Optionally, the network confidence threshold of the first obstacle detection network can be set to 0.5 to balance recall and false alarm rate. The first obstacle detection network can be pre-trained based on the sample panoramic bird's-eye view and its corresponding calibrated two-dimensional bounding boxes, and the second obstacle detection network can be pre-trained based on the sample target environment map and its corresponding calibrated target obstacle type information to ensure that the detection network has high recognition accuracy and generalization ability.

[0051] By employing dual-view images and a dual-detection network, the accuracy and reliability of obstacle recognition are significantly improved. Leveraging the global field of view of the bird's-eye view, rapid localization and preliminary bounding selection of obstacles are achieved, avoiding the problem of missing obstacles from local perspectives. Furthermore, by mapping image coordinates, the bounding boxes are transferred to the side view image to generate a target environment map. Utilizing the clear detail rendering capability of the side view, richer feature information is provided for the second obstacle detection network, enabling global localization and detail recognition. This provides accurate basic information for subsequent calculations of the relative relationship between obstacles and vehicle doors.

[0052] S130. Determine the relative distance, relative speed, and relative angle between the obstacle and the door based on the image position information, type information, environmental point cloud data, and door position information of the target vehicle door. The relative angle is the angle between the moving direction of the obstacle and the normal direction of the door.

[0053] The door position information can be the door's installation position in the target vehicle's body coordinate system, and its corresponding three-dimensional spatial coordinates, such as the initial position when the door is closed, the real-time position during the opening process, etc. Relative distance refers to the straight-line distance between the obstacle and the target vehicle's door. Relative speed refers to the speed of the obstacle relative to the target vehicle's door. Relative angle is the angle between the obstacle's direction of movement and the door's normal direction, enabling precise determination of whether the obstacle's movement is directed towards the door, further refining the dimensions of collision risk assessment.

[0054] Optionally, the relative distance, relative speed, and relative angle between the obstacle and the door can be calculated based on the obstacle's image location information, type information, environmental point cloud data, and the door location information of the target vehicle.

[0055] S140. In response to a request to open the vehicle door, control the opening of the vehicle door based on the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle.

[0056] Specifically, in response to a request to open a vehicle door, the type information of the identified obstacles, the relative distance, relative speed, and relative angle between the obstacles and the vehicle door can be correlated under a unified timestamp and coordinate system to generate a complete list of obstacles with spatiotemporal attributes. This list clearly indicates whether any obstacles affect door opening and determines the corresponding door opening method. Furthermore, for point cloud clusters detected by DTOF but not recognized by the side-view camera, their type information can be marked as unknown obstacles to avoid overlooking risks due to missing information.

[0057] Based on the above scheme, optionally, controlling the opening of the vehicle door according to the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle includes: determining the type weight of the obstacle according to the type information of the obstacle; determining the target risk value corresponding to the obstacle according to the type weight of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle; and controlling the opening of the vehicle door according to the target risk value and a preset threshold.

[0058] The type weight can be a weighting coefficient determined based on the obstacle type information to measure the risk level of different types of obstacles. For example, a pedestrian type weight can be set with a higher risk value, while a stationary obstacle type weight can be set with a lower risk value. The type weight value can be set based on a large amount of pre-existing real-vehicle test data or business needs. The target risk value can be a numerical value calculated by comprehensively considering the obstacle type weight, relative distance, relative speed, and relative angle to quantify the risk level of the obstacle to the door opening. The higher the target risk value, the greater the probability of a collision with the obstacle when the door opens. The preset threshold can be a pre-set value used to determine whether the target risk value is within a safe range.

[0059] By assigning different weights to obstacle types, the risk differences of different types of obstacles are quantified, avoiding misjudgments caused by treating all obstacles equally. By integrating type weights with multiple factors such as relative distance, relative speed, and relative angle, the target risk value is obtained through comprehensive calculation. Compared with judgment based on a single factor, this can more comprehensively and objectively reflect the actual risk level, avoiding safety hazards caused by ignoring a key factor. The door opening strategy is determined by comparing the target risk value with a preset threshold, ensuring that an appropriate opening method can be adopted in different risk scenarios, balancing safety and ease of use, and realizing intelligent and differentiated control of door opening.

[0060] Specifically, the target risk value corresponding to the obstacle is determined based on the following formula, according to the type weight of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle:

[0061] ;

[0062] in, This represents the target risk value corresponding to the obstacle. The type weight of the obstacle is indicated. This indicates the relative distance between the obstacle and the vehicle door. This indicates the minimum safe distance between the obstacle and the vehicle door. This indicates the magnitude of the relative velocity between the obstacle and the vehicle door. This indicates the relative angle between the obstacle and the vehicle door. The default value is a positive number to prevent division by zero. , These are adjustment coefficients, which can be calibrated through extensive real-vehicle testing in advance. The cosine of the angle between the relative velocity direction of the obstacle and the normal direction of the door can be used to determine the component of the relative velocity pointing towards the door.

[0063] Specifically, the preset threshold may include an upper threshold and a lower threshold; controlling the opening of the vehicle door according to the target risk value and the preset threshold includes: controlling the vehicle door to open to a first angle when the target risk value does not exceed the lower threshold; controlling the vehicle door to open to a second angle, where the second angle is smaller than the first angle, when the target risk value exceeds the lower threshold but not the upper threshold; and controlling the vehicle door to be in a closed state when the target risk value exceeds the upper threshold.

[0064] The upper limit threshold can be a critical value indicating the danger of the door opening. When the target risk value exceeds the upper limit threshold, it means that the risk of the door colliding with an obstacle after opening is extremely high, and the door can be prohibited from opening. For example, the upper limit threshold can be a situation where the relative distance between the obstacle and the door is less than the minimum safe distance.

[0065] The lower threshold can be a safe critical value for the risk of the door opening. When the target risk value does not exceed the lower threshold, it means that the risk of the door colliding with an obstacle after opening is extremely low, and the door can be allowed to open normally. The lower threshold can be defined as the relative distance between the obstacle and the door being greater than the maximum safe distance. The first angle can be the maximum safe angle for the door to open. Specifically, when the target risk value does not exceed the lower threshold, the door can be controlled to open to the first angle.

[0066] When the target risk value is between the lower and upper thresholds, it indicates that there is a certain risk in opening the door, and the door opening angle needs to be restricted. The second angle can be the limiting angle for door opening; that is, an angle that, while posing a certain risk, can both meet the temporary opening needs and reduce the risk of a collision.

[0067] Optionally, the second angle can be a preset angle value, or it can be determined by calculating the angle at which the relative distance between the obstacle and the vehicle first equals 0 during the opening of the door, so as to ensure that it adapts to the risk control needs of different scenarios.

[0068] Furthermore, after determining the door opening angle, the domain controller can generate a control message to send the door opening angle result to the body controller. The door actuator drives the electric servo motor integrated in the door lock mechanism or hinge, so that the motor can precisely control the door opening angle and speed with controlled torque and speed.

[0069] By dividing preset thresholds into upper and lower limits, different risk levels can be matched more precisely, and differentiated opening strategies can be formulated for different risk ranges, adapting to diverse usage scenarios. This achieves refined management of car door opening, effectively avoiding high-risk collisions and ensuring ease of use in low-risk scenarios.

[0070] To further enhance the safety and flexibility of the door opening process, optionally, obstacles can be continuously detected and the target risk value can be updated in real time during the door opening movement. If the target risk value decreases, it indicates that the obstacle is moving away, and the opening angle can be re-determined and increased according to the above steps. If the target risk value increases, the opening angle can be recalculated and decreased or the door can be locked according to the above steps.

[0071] To enhance the user experience, optionally, during door control, multimodal feedback methods such as instrument panel charts, audible warnings of different frequencies or rhythms, or color changes of LED light strips on the doors can be used to alert the driver to the currently identified obstacle risks and the corresponding door control methods. For example, when the left rear door is in the controlled open state, the instrument cluster can display a left rear door icon marked with an exclamation mark, while the in-vehicle speakers emit a warning sound, providing both visual and auditory alerts. When the door is about to open to its second angle, the motor output torque slightly increases, generating a damping sensation, providing occupants with a tactile cue that the door is about to open completely. If an occupant manually attempts to push the door further, the corresponding target risk value is calculated in real time, and reverse resistance is applied or a stronger warning is issued based on the actual risk situation, further ensuring the safety of the door opening process.

[0072] The technical solution of this invention firstly involves acquiring a bird's-eye view of the target vehicle using a surround-view imaging device, acquiring a side view of the target vehicle using a side-view imaging device, and acquiring environmental point cloud data of the target vehicle using a target sensor. This multi-source data acquisition avoids the limitations of a single device's field of view or dimensionality, allowing for mutual supplementation and providing a richer data foundation for subsequent obstacle recognition. Next, based on the bird's-eye view and the vehicle's side view, the type information of obstacles in the target vehicle's environment and the image location information of the obstacles are determined. This allows for rapid location of the approximate location of obstacles using the bird's-eye view and accurate identification of obstacle types using the side view, improving recognition accuracy and simultaneously acquiring the obstacle type. The system first obtains information and image location information. Then, based on the obstacle's image location information, type information, environmental point cloud data, and the target vehicle's door location information, it determines the relative distance, relative speed, and relative angle between the obstacle and the door. The relative angle is the angle between the obstacle's movement direction and the door's normal direction. This multi-dimensional information calculation reduces environmental interference, improves the accuracy of relative information measurement, and provides key quantitative indicators for subsequent risk assessment. Finally, in response to a door opening request, the system controls the door opening based on the obstacle's type information, the relative distance, relative speed, and relative angle between the obstacle and the door. By integrating obstacle type, relative distance, relative speed, and relative angle, the door opening control is made more aligned with actual safety scenarios. Through multi-dimensional data fusion and quantitative analysis, the door opening control becomes more scientific and safer, effectively ensuring the safety of vehicle occupants and the surrounding environment.

[0073] Example 2

[0074] Figure 2 This is a flowchart of a vehicle door control method provided in Embodiment 2 of the present invention, further describing the implementation process of determining the relative distance, relative speed, and relative angle between the obstacle and the vehicle door based on the image position information, type information, environmental point cloud data, and door position information of the target vehicle door. Detailed implementation methods can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 2 As shown, the method may specifically include:

[0075] S210. Acquire a bird's-eye view of the target vehicle based on the surround-view shooting device, acquire a side view of the target vehicle based on the side-view shooting device, and acquire environmental point cloud data of the target vehicle through the target sensor.

[0076] S220. Determine the type information of obstacles in the environment where the target vehicle is located and the image location information of the obstacles based on the surround-view bird's-eye view and the vehicle side view.

[0077] S230. Determine the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle based on the image position information and type information of the obstacle.

[0078] The vehicle body coordinate system can be a three-dimensional Cartesian coordinate system established with the target vehicle as the reference. It typically uses the vehicle's geometric center or front axle center as the origin, the vehicle's forward direction as the positive x-axis, the direction perpendicular to the ground upwards as the positive z-axis, and the direction perpendicular to the x-axis and pointing to the left of the vehicle as the positive y-axis. This system is used to uniformly describe the spatial positions of doors and obstacles. The three-dimensional spatial region can be a three-dimensional spatial range encompassing the entire obstacle, determined within the vehicle body coordinate system based on the obstacle's image position and type information. It represents the spatial area where the obstacle may exist.

[0079] Based on the above scheme, optionally, determining the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle according to the image position information and type information of the obstacle includes: determining the three-dimensional spatial coordinates of the obstacle in the vehicle body coordinate system of the target vehicle according to the image position information of the obstacle and the preset height corresponding to the obstacle, and determining the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle according to the three-dimensional spatial coordinates of the obstacle and the type information.

[0080] The preset height can be a fixed value or a value pre-set for different types of obstacles based on obstacle type information. The preset height can be obtained statistically from a large number of experimental samples and is used to help determine the three-dimensional spatial coordinates of the obstacle. The three-dimensional spatial coordinates can be the x, y, and z coordinates of the obstacle's feature points in the vehicle's coordinate system.

[0081] Specifically, the image position information of the obstacle can be converted back to the x and y coordinates in the vehicle body coordinate system using a preset coordinate transformation model, and the preset height can be used as the z coordinate of the obstacle in the vehicle body coordinate system. The three-dimensional spatial coordinates of the obstacle in the target vehicle's vehicle body coordinate system can then be obtained by combining these coordinates. The coordinate transformation model can be pre-trained and determined based on the parameters of the side-view shooting device and the vehicle calibration parameters.

[0082] Furthermore, using the three-dimensional spatial coordinates as the center, a cubic region is divided according to the size characteristics corresponding to the obstacle type information as the key search area for the obstacle's point cloud data, thereby determining the three-dimensional spatial region corresponding to the obstacle.

[0083] By using image location information of obstacles and corresponding preset heights, the three-dimensional spatial coordinates of obstacles in the vehicle coordinate system are quickly determined, giving coordinate positioning a three-dimensional attribute. Based on the three-dimensional spatial coordinates and combined with obstacle size features supplemented by type information, the boundaries of the three-dimensional spatial region are further refined. This ensures that the delineated region completely includes the obstacles without excessively expanding the range and causing irrelevant point clouds to be mixed in. This effectively improves the efficiency and accuracy of subsequent point cloud clustering, indirectly ensuring the accuracy of spatial location information measurement, realizing the scientific delineation of the three-dimensional spatial region, and laying a precise foundation for subsequent point cloud data processing.

[0084] S240. Determine the spatial location information of the obstacle based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data.

[0085] Among them, spatial location information can be data that can accurately represent the three-dimensional spatial location of the obstacle in the vehicle coordinate system.

[0086] Based on the above scheme, optionally, determining the spatial location information corresponding to the obstacle based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data includes: clustering multiple environmental points in the environmental point cloud data according to the three-dimensional spatial region corresponding to the obstacle to obtain the point cloud cluster corresponding to the obstacle; determining the geometric center of the point cloud cluster corresponding to the obstacle, and determining the geometric center as the spatial location information corresponding to the obstacle.

[0087] In this context, an environmental point can be a single 3D coordinate point in the environmental point cloud data. Each environmental point corresponds to a reflected signal from a tiny area in the environment detected by the target sensor. A large number of environmental points can collectively form the point cloud outline of objects such as obstacles and the ground. A point cloud cluster can be a set of all environmental points belonging to the same obstacle, obtained through a clustering algorithm. The environmental points in this set are spatially tightly clustered and can roughly reflect the 3D shape outline of the obstacle. The geometric center can be the average coordinate of all environmental points in the point cloud cluster, which can approximately represent the actual spatial location of the obstacle.

[0088] Specifically, within the three-dimensional spatial region corresponding to the obstacle, multiple environmental points located within the obstacle's three-dimensional spatial region in the environmental point cloud data can be clustered using the Euclidean clustering algorithm. Environmental points that are spatially close and belong to the same obstacle are grouped into a single point cloud cluster. For all environmental points in the clustered point cloud data, their geometric centers are calculated as the spatial location information corresponding to the obstacle. The Euclidean clustering distance threshold can be 0.05-0.15m, which can be flexibly adjusted according to the sensor accuracy and obstacle type.

[0089] By rapidly filtering environmental points belonging to the target obstacle using a preset 3D spatial region as the selection criterion, interference from point clouds such as the ground and distant irrelevant objects is eliminated, allowing point cloud processing to focus more on the target obstacle. A clustering algorithm aggregates the filtered environmental points into point cloud clusters, further eliminating discrete noise points and ensuring that the point cloud clusters accurately reflect the 3D contours of the obstacle. Using the geometric center of the point cloud cluster as the spatial location information of the obstacle, compared to directly selecting individual point clouds or edge points, the geometric center better represents the overall position of the obstacle, reducing positional deviations caused by uneven point cloud distribution. This makes the spatial location information more representative and reliable, providing accurate data support for subsequent relative relationship calculations and achieving precise extraction of obstacle spatial location information.

[0090] S250. Based on the spatial position information of the obstacle at multiple times and the door position information of the target vehicle door, determine the relative distance, relative speed, and relative angle between the obstacle and the door, wherein the relative angle is the angle between the moving direction of the obstacle and the normal direction of the door.

[0091] Optionally, the relative distance, relative speed, and relative angle between the obstacle and the door can be calculated based on the spatial position information of the obstacle and the door position information of the target vehicle at multiple consecutive time points. The relative angle is the angle between the direction of movement of the obstacle and the normal direction of the door.

[0092] Based on the above scheme, optionally, determining the relative distance, relative speed, and relative angle between the obstacle and the door based on the spatial position information of the obstacle at multiple times and the door position information of the target vehicle includes: determining the relative distance between the obstacle and the door at each time based on the spatial position information of the obstacle and the door position information of the target vehicle at each time; and determining the relative speed and relative angle of the obstacle relative to the door of the target vehicle based on the spatial position information of the obstacle at multiple times.

[0093] Specifically, the spatial location information of the obstacle and the door location information of the target vehicle can be calculated at each time point to determine the relative distance between the obstacle and the door at that time. The relative distance can be the straight-line distance between the geometric center coordinates of the obstacle's point cloud cluster and the door location information, or it can be the shortest distance between all environmental points in the point cloud cluster corresponding to the obstacle and the outer surface of the door.

[0094] Optionally, the spatial position information of obstacles can be continuously tracked at multiple time points. Kalman filtering is used to reduce the impact of sensor noise, measurement errors, and environmental interference on the position data. Furthermore, the relative distance change of the obstacle relative to the door of the target vehicle is estimated, and then the relative speed and relative angle of the obstacle relative to the door of the target vehicle are determined by combining the time intervals. The process noise and measurement noise parameters of the Kalman filter can be specifically optimized according to the accuracy level of the sensor and actual test data to ensure that the filtering effect matches the actual scene.

[0095] By calculating the relative distance between the obstacle and the door at each moment, reliable basic data is provided for subsequent dynamic analysis. Based on the spatial position information at multiple moments, the relative velocity is obtained by calculating the ratio of the change in position to the time interval, and the relative angle is obtained by analyzing the angle between the direction of the change in position and the normal of the door. This avoids the problem that data from a single moment cannot reflect the movement trend, and can accurately capture the movement state of the obstacle. The calculation of relative velocity and relative angle is more in line with the actual movement scenario, providing key quantitative indicators for the dynamic risk assessment of door opening control, and effectively improving the accuracy of dynamic risk judgment.

[0096] S260. In response to a request to open the vehicle door, control the opening of the vehicle door based on the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle.

[0097] The technical solution of this invention firstly determines the three-dimensional spatial region corresponding to the obstacle in the vehicle coordinate system of the target vehicle based on the image position information and the type information of the obstacle; combining the two-dimensional image position information with the obstacle type information avoids the limitations of planar positioning caused by relying solely on image coordinates, and provides three-dimensional spatial attributes through type information, making the spatial region division more closely match the actual shape of the obstacle; then, it determines the spatial position information corresponding to the obstacle based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data; and accurately extracts points belonging to the obstacle from the environmental point cloud data. Cloud clusters provide precise spatial positioning, more realistically reflecting the actual spatial coordinates of obstacles and significantly improving the accuracy of spatial position information, laying a reliable foundation for subsequent distance and speed calculations. Finally, by using the spatial position information of the obstacle at multiple moments and the door position information of the target vehicle, the relative distance, relative speed, and relative angle between the obstacle and the door are determined. By capturing the movement trend of the obstacle at multiple moments, the relative speed is calculated, avoiding the risk of ignoring moving obstacles due to static calculations. By associating the relative distance and angle with the door position information, the calculation results are more in line with the actual needs of door control.

[0098] Example 3

[0099] As an optional example of an embodiment of the present invention, the door control method of the present invention may specifically include:

[0100] Based on the fusion of surround view, side view and DTOF, the automatic door opening and obstacle avoidance of the car can be realized, which can realize all-round perception and intelligent decision-making of the environment behind the vehicle.

[0101] S1: Synchronous acquisition and preprocessing of multi-sensor data

[0102] 1. The domain controller or high-performance ECU sends a hardware synchronization trigger signal to all sensors to ensure that all data frames have a unified timestamp (microsecond-level synchronization).

[0103] 2. All triggered sensors synchronously acquire one frame of data, including images from surround-view and side-view cameras, and raw point cloud data from all DTOF sensors.

[0104] S2: Multi-source information fusion sensing

[0105] S2.1: Preliminary delineation of the Region of Interest (ROI) based on the surround view:

[0106] The preprocessed surround-view image is fed into a lightweight deep learning object detection network (such as YOLOv5s). This network is trained to detect general obstacles in the side and rear regions of the vehicle, outputting a 2D bounding box of the obstacle and its approximate coordinates in the bird's-eye view, i.e., the initial ROI. The network confidence threshold can be set to 0.5 to balance recall and false positive rate. For example, if the network identifies a bright patch in the left side of the image, it outputs a 2D bounding box and its confidence (85%), classifying it as an obstacle. This bounding box provides a coarse pixel-level location of the obstacle.

[0107] S2.2: Side-view camera for precise recognition:

[0108] The ROI obtained from the surround view is mapped to the corresponding side-view camera image using a pre-calibrated homography transformation matrix. Within this ROI region, a classification network is run to output the precise type of the obstacle (e.g., "pedestrian", "bicycle", "car") and its precise pixel coordinates in the image.

[0109] For example, the coordinates of the center point of the bounding box in a panoramic bird's-eye view. The points are precisely mapped onto the image coordinate system of the side-view camera to obtain the points. and with Centered on the image, a 300x300 pixel region of interest (ROI) is cropped from the side view image, and this ROI is fed into a more refined convolutional neural network specifically for classification. The network output is "car" with a confidence level of 92%.

[0110] S2.3: DTOF Precise Ranging and Point Cloud Clustering:

[0111] 1. Guided Clustering: Using obstacle types and 2D pixel coordinates provided by side-view cameras, the 3D spatial region (a cone-shaped space) where the object might exist in the vehicle's coordinate system is calculated backwards. For example, assuming a point on the obstacle (car) is 1.2 meters above the ground (a prior assumption), the pixel coordinates can be... A predicted 3D coordinate point in the vehicle body coordinate system is calculated back. ,by A cube measuring 1.5m × 1.5m × 2.0m was designated as the primary search area, centered on the area.

[0112] 2. Clustering Algorithm: Within the prediction area, the Euclidean clustering algorithm is used on the DTOF point cloud. A small distance threshold (e.g., 0.1m) is set to cluster spatially close points into one class, representing target obstacles.

[0113] 3. State Estimation: For clustered point cloud clusters, calculate their geometric centers as the positions of obstacles. Track the position changes of point cloud clusters across multiple frames, and estimate their velocity vectors (including magnitude and direction) relative to the vehicle using Kalman filtering to obtain the relative distance, relative velocity, and relative angle of the obstacle in the vehicle's coordinate system. The Euclidean clustering distance threshold can be 0.05-0.15m; the process noise and measurement noise parameters of the Kalman filter need to be optimized based on the sensor accuracy.

[0114] For example, to calculate the shortest distance between all points in the point cloud cluster of the obstacle and the outer surface of the left rear door panel, the Kalman filter receives the position information of the point cloud cluster in the current frame and the past 4 frames (a total of 5 frames). If the position estimate is 0.65m in 5 consecutive frames of data, the relative speed estimated by the filter is 0m / s (stationary).

[0115] S2.4: Spatiotemporal Synchronization and Information Fusion:

[0116] The obstacle semantic information output by S2.2 and the measurement information output by S2.3 are correlated under a unified timestamp and coordinate system to generate a complete obstacle list with spatiotemporal attributes. For point cloud clusters detected by DTOF but not recognized by side view, they are marked as unknown obstacles and assigned a preset high-risk weight, and recorded in the obstacle list.

[0117] S3: Dynamic Risk Assessment and Opening Decision

[0118] 1. Risk Modeling: Calculate a comprehensive risk value (Risk) for each obstacle.

[0119] ;

[0120] in, This represents the target risk value corresponding to the obstacle. The type weight of the obstacle is indicated. This indicates the relative distance between the obstacle and the vehicle door. This indicates the minimum safe distance between the obstacle and the vehicle door. This indicates the magnitude of the relative velocity between the obstacle and the vehicle door. This indicates the relative angle between the obstacle and the vehicle door. The default value is a positive number to prevent division by zero. , These are adjustment coefficients.

[0121] 2. Decision-making logic:

[0122] Calculated Compared with the preset threshold:

[0123] The preset thresholds include an upper threshold and a lower threshold; if the target risk value does not exceed the lower threshold, the door is controlled to open to a first angle; if the target risk value exceeds the lower threshold but does not exceed the upper threshold, the limit switch is activated, and the door is controlled to open to a second angle, which is smaller than the first angle; if the target risk value exceeds the upper threshold, the door is controlled to be in a closed state.

[0124] 3. Output command: Send the door opening angle decision result to the body controller.

[0125] S4: Execution and Dynamic Feedback

[0126] 1. The main circulation (S1-S3) runs continuously during the movement of the car door.

[0127] 2. If the vehicle door is in the limited-open position, continuously monitor the risk value. If the risk decreases (e.g., pedestrians move away), the opening angle can be increased; if the risk increases, the angle can be decreased or the door can be locked.

[0128] 3. Provide multimodal feedback: Inform users of the current door opening status and the reason for the decision through dashboard icons, sound warnings (different frequencies / rhythms), or color changes of LED light strips on the doors.

[0129] The technical solution of this invention firstly acquires data synchronously from multiple sensors, including images from surround-view and side-view cameras and point cloud data from all DTOF sensors, avoiding blind spots in single-sensor perception and providing complete data support for subsequent obstacle recognition and decision-making. Next, the surround-view cameras initially delineate regions of interest (ROIs) for obstacles, the side-view cameras perform detailed identification and classification of obstacles within these ROIs, and the DTOF sensors accurately measure distances and estimate speeds of obstacles. Tasks are assigned and relevant information is acquired based on the core strengths of different sensors, providing comprehensive input for subsequent risk assessment and avoiding decision-making biases due to missing information. Finally, a comprehensive risk value is calculated by combining multiple dimensions such as obstacle type, relative distance to the vehicle door, and relative speed. Based on this comprehensive risk value, a tiered decision-making process is implemented. This multi-dimensional risk assessment achieves scientific decision-making, and the tiered strategy avoids excessive intervention and optimizes the user experience.

[0130] Example 4

[0131] Figure 4 This is a schematic diagram of a vehicle door control device according to Embodiment 4 of the present invention. This device is used to execute the vehicle door control method provided in any of the above embodiments. This device and the vehicle door control methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the vehicle door control device can be referred to the embodiments of the above vehicle door control methods. Figure 4 As shown, the device includes: a target vehicle data acquisition module 410, an obstacle type and image position determination module 420, an obstacle relative information determination module 430, and a door control opening module 440.

[0132] The system includes: a target vehicle data acquisition module 410, used to acquire a surround-view bird's-eye view of the target vehicle based on a surround-view shooting device, acquire a side view of the target vehicle based on a side-view shooting device, and acquire environmental point cloud data of the target vehicle through a target sensor; an obstacle type and image position determination module 420, used to determine the type information and image position information of obstacles in the environment where the target vehicle is located based on the surround-view bird's-eye view and the side view of the vehicle; an obstacle relative information determination module 430, used to determine the relative distance, relative speed, and relative angle between the obstacle and the door based on the image position information, the type information, the environmental point cloud data, and the door position information of the target vehicle door, wherein the relative angle is the angle between the moving direction of the obstacle and the normal direction of the door; and a door control opening module 440, used to control the opening of the door in response to an opening request for the door, based on the type information of the obstacle, the relative distance, the relative speed, and the relative angle between the obstacle and the door.

[0133] The technical solution of this invention firstly involves the target vehicle data acquisition module 410 acquiring a panoramic bird's-eye view of the target vehicle using a surround-view shooting device, acquiring a side view of the target vehicle using a side-view shooting device, and acquiring environmental point cloud data of the target vehicle using a target sensor. This multi-source data acquisition avoids the limitations of a single device's field of view or dimensionality, allowing for mutual complementarity and providing a richer data foundation for subsequent obstacle recognition. Next, the obstacle type and image location determination module 420 determines the type information and image location information of obstacles in the environment where the target vehicle is located based on the panoramic bird's-eye view and the vehicle side view. This allows for rapid location of the approximate location of obstacles using the bird's-eye view and accurate identification of obstacle types using the side view, improving recognition accuracy and simultaneously acquiring obstacle data. The obstacle type information and image location information are used as inputs. Next, the obstacle relative information determination module 430 determines the relative distance, relative speed, and relative angle between the obstacle and the vehicle door based on the obstacle's image location information, type information, environmental point cloud data, and the target vehicle door's door location information. The relative angle is the angle between the obstacle's movement direction and the door's normal direction. This multi-dimensional information calculation reduces environmental interference, improves the accuracy of relative information measurement, and provides key quantitative indicators for subsequent risk assessment. Finally, the door opening control module 440 responds to the door opening request by controlling the door opening based on the obstacle's type information, the relative distance, relative speed, and relative angle between the obstacle and the door. By integrating obstacle type, relative distance, relative speed, and relative angle, the door opening control is more aligned with actual safety scenarios. Through multi-dimensional data fusion and quantitative analysis, the door opening control is made more scientific and safer, effectively ensuring the safety of vehicle occupants and the surrounding environment.

[0134] Based on the above scheme, optionally, the obstacle type and image position determination module 420 includes a surround-view bird's-eye view information determination module, a target environment map determination module, and an obstacle type and image position determination submodule. Specifically, the surround-view bird's-eye view information determination module is used to determine the two-dimensional bounding boxes of obstacles in the surround-view bird's-eye view and the image coordinates of the obstacles in the surround-view bird's-eye view based on the surround-view bird's-eye view and the first obstacle detection network; the target environment map determination module is used to map the two-dimensional bounding boxes of the obstacles to the vehicle side view based on the image coordinates of the obstacles in the surround-view bird's-eye view to obtain a target environment map; the obstacle type and image position determination submodule is used to determine the type information of the obstacles and the image position information of the obstacles in the target environment map based on the target environment map and the second obstacle detection network.

[0135] Based on the above scheme, optionally, the obstacle relative information determination module 430 includes a three-dimensional spatial region determination submodule, a spatial position information determination submodule, and an obstacle relative information determination submodule. The three-dimensional spatial region determination submodule is used to determine the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle based on the image position information and type information of the obstacle; the spatial position information determination submodule is used to determine the spatial position information corresponding to the obstacle based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data; the obstacle relative information determination submodule is used to determine the relative distance, relative speed, and relative angle between the obstacle and the vehicle door based on the spatial position information of the obstacle at multiple times and the door position information of the target vehicle door.

[0136] Based on the above scheme, optionally, the three-dimensional spatial region determination submodule includes a three-dimensional spatial region determination unit. The three-dimensional spatial region determination unit is used to determine the three-dimensional spatial coordinates of the obstacle in the target vehicle's body coordinate system based on the obstacle's image position information and the obstacle's corresponding preset height, and to determine the three-dimensional spatial region corresponding to the obstacle in the target vehicle's body coordinate system based on the obstacle's three-dimensional spatial coordinates and type information.

[0137] Based on the above scheme, optionally, the spatial location information determination submodule includes an obstacle point cloud cluster determination unit and a spatial location information determination unit. The obstacle point cloud cluster determination unit is used to cluster multiple environmental points in the environmental point cloud data according to the three-dimensional spatial region corresponding to the obstacle to obtain the point cloud cluster corresponding to the obstacle; the spatial location information determination unit is used to determine the geometric center of the point cloud cluster corresponding to the obstacle, and determine the geometric center as the spatial location information corresponding to the obstacle.

[0138] Based on the above scheme, optionally, the obstacle relative information determination submodule includes a relative distance determination unit and a relative speed and angle determination unit. The relative distance determination unit is used to determine the relative distance between the obstacle and the vehicle door at each given time based on the spatial position information of the obstacle and the door position information of the target vehicle at each given time. The relative speed and angle determination unit is used to determine the relative speed and relative angle of the obstacle relative to the door of the target vehicle based on the spatial position information of the obstacle at multiple time points.

[0139] Optionally, based on the above solution, the door control opening module 440 includes a door control opening submodule. This submodule is used to determine the type weight of the obstacle based on its type information, determine a target risk value corresponding to the obstacle based on its type weight, the relative distance between the obstacle and the door, the relative speed, and the relative angle, and control the opening of the door based on the target risk value and a preset threshold.

[0140] Based on the above solution, optionally, the door control opening submodule is used to determine the target risk value corresponding to the obstacle based on the type weight of the obstacle, the relative distance between the obstacle and the door, the relative speed, and the relative angle, according to the following formula:

[0141] ;

[0142] in, This represents the target risk value corresponding to the obstacle. The type weight of the obstacle is indicated. This indicates the relative distance between the obstacle and the vehicle door. This indicates the minimum safe distance between the obstacle and the vehicle door. This indicates the magnitude of the relative velocity between the obstacle and the vehicle door. This indicates the relative angle between the obstacle and the vehicle door. The default value is a positive number to prevent division by zero. , These are adjustment coefficients.

[0143] Based on the above solution, optionally, the preset threshold includes an upper threshold and a lower threshold; the door control opening submodule includes a first opening unit, a second opening unit, and a third opening unit. The first opening unit is used to control the door to open to a first angle when the target risk value does not exceed the lower threshold; the second opening unit is used to control the door to open to a second angle, where the second angle is smaller than the first angle, when the target risk value exceeds the lower threshold but does not exceed the upper threshold; the third opening unit is used to control the door to remain closed when the target risk value exceeds the upper threshold.

[0144] The door control device provided in the embodiments of the present invention can execute the door control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0145] Example 5

[0146] Figure 5 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0147] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0148] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0149] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as door control methods.

[0150] In some embodiments, the door control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded into and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the door control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the door control method by any other suitable means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0155] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0156] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0157] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0158] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A door control method, characterized in that, include: The system acquires a bird's-eye view of the target vehicle using a surround-view camera, acquires a side view of the target vehicle using a side-view camera, and acquires environmental point cloud data of the target vehicle using a target sensor. Based on the surround-view bird's-eye view and the vehicle side view, determine the type information of obstacles in the environment where the target vehicle is located and the image location information of the obstacles; The relative distance, relative speed, and relative angle between the obstacle and the door are determined based on the image location information, type information, environmental point cloud data, and door location information of the target vehicle door. The relative angle is the angle between the moving direction of the obstacle and the normal of the door. In response to a request to open the vehicle door, the opening of the vehicle door is controlled based on the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle.

2. The door control method according to claim 1, characterized in that, The step of determining the type information of obstacles in the environment where the target vehicle is located and the image location information of the obstacles based on the surround-view bird's-eye view and the vehicle side view includes: Based on the surround-view bird's-eye view and the first obstacle detection network, the two-dimensional bounding boxes of obstacles in the environment where the target vehicle is located in the surround-view bird's-eye view and the image coordinates of the obstacles in the surround-view bird's-eye view are determined. The two-dimensional bounding box of the obstacle is mapped to the vehicle side view based on the image coordinates of the obstacle in the surround view bird's-eye view to obtain a target environment map; The type information of the obstacle and the image location information of the obstacle in the target environment map are determined based on the target environment map and the second obstacle detection network.

3. The door control method according to claim 1, characterized in that, Determining the relative distance, relative speed, and relative angle between the obstacle and the vehicle door based on the obstacle's image location information, type information, environmental point cloud data, and the target vehicle door's door location information includes: Based on the image location information and type information of the obstacle, determine the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle; The spatial location information of the obstacle is determined based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data. The relative distance, relative speed, and relative angle between the obstacle and the door are determined based on the spatial position information of the obstacle at multiple times and the door position information of the target vehicle.

4. The door control method according to claim 3, characterized in that, Determining the three-dimensional spatial region corresponding to the obstacle in the vehicle body coordinate system of the target vehicle based on the image position information and type information of the obstacle includes: The three-dimensional spatial coordinates of the obstacle in the target vehicle's body coordinate system are determined based on the obstacle's image position information and the obstacle's corresponding preset height. The three-dimensional spatial region corresponding to the obstacle in the target vehicle's body coordinate system is determined based on the obstacle's three-dimensional spatial coordinates and the obstacle's type information.

5. The door control method according to claim 3, characterized in that, The step of determining the spatial location information corresponding to the obstacle based on the three-dimensional spatial region corresponding to the obstacle and the environmental point cloud data includes: Clustering multiple environmental points in the environmental point cloud data according to the three-dimensional spatial region corresponding to the obstacle, to obtain the point cloud cluster corresponding to the obstacle; Determine the geometric center of the point cloud cluster corresponding to the obstacle, and use the geometric center as the spatial location information corresponding to the obstacle.

6. The door control method according to claim 3, characterized in that, The step of determining the relative distance, relative speed, and relative angle between the obstacle and the vehicle door based on the spatial position information of the obstacle at multiple times and the door position information of the target vehicle door includes: The relative distance between the obstacle and the vehicle door at each given time moment is determined based on the spatial location information of the obstacle and the door location information of the target vehicle at each given time moment; and, The relative speed and relative angle of the obstacle relative to the door of the target vehicle are determined based on the spatial position information of the obstacle at multiple times.

7. The door control method according to claim 1, characterized in that, The method of controlling the opening of the vehicle door based on the type information of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle includes: The type weight of the obstacle is determined based on the type information of the obstacle. The target risk value corresponding to the obstacle is determined based on the type weight of the obstacle, the relative distance between the obstacle and the door, the relative speed, and the relative angle. The opening of the door is controlled based on the target risk value and a preset threshold.

8. The door control method according to claim 7, characterized in that, Based on the following formula, the target risk value corresponding to the obstacle is determined according to the type weight of the obstacle, the relative distance between the obstacle and the vehicle door, the relative speed, and the relative angle: ； in, This represents the target risk value corresponding to the obstacle. The type weight of the obstacle is indicated. This indicates the relative distance between the obstacle and the vehicle door. This indicates the minimum safe distance between the obstacle and the vehicle door. This indicates the magnitude of the relative velocity between the obstacle and the vehicle door. This indicates the relative angle between the obstacle and the vehicle door. The default value is a positive number to prevent division by zero. , These are adjustment coefficients.

9. The door control method according to claim 7, characterized in that, The preset threshold includes an upper threshold and a lower threshold; controlling the opening of the car door based on the target risk value and the preset threshold includes: If the target risk value does not exceed the lower threshold, control the door to open to the first angle; If the target risk value exceeds the lower threshold but does not exceed the upper threshold, the door is controlled to open to a second angle, where the second angle is smaller than the first angle. If the target risk value exceeds the upper limit threshold, the door is controlled to be closed.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the door control method as described in any one of claims 1-9.

Images