Obstacle avoidance control method and system for vehicle door opening and vehicle-mounted terminal

Abstract

The invention provides an obstacle avoidance control method and system for vehicle door opening and a vehicle-mounted terminal.The method comprises the steps that obstacle information outside a vehicle door is obtained, and the obstacle information comprises the obstacle distance between a target obstacle and the vehicle door and the first height corresponding to the target obstacle; according to the obstacle distance, the first height and preset vehicle door opening data, collision risk assessment between the vehicle door and the target obstacle is carried out; and if the collision risk exists between the vehicle door and the target obstacle, the vehicle door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height so that the vehicle door can avoid the target obstacle in the opening process. According to the obstacle condition outside the vehicle door, the vehicle door opening angle and / or the vehicle suspension height are / is intelligently adjusted so as to ensure that the vehicle door can avoid the obstacle and be opened, and therefore the convenience and safety of the vehicle for a user are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle safety technology, and in particular to an obstacle avoidance control method and system for opening a vehicle door, and a vehicle-mounted terminal. Background Art

[0002] With the continuous development of science and technology, cars have become an essential means of transportation in human life. With the development of the automobile industry and the advancement of technology, users' demand for intelligent vehicles is also increasing day by day. Therefore, modern vehicles have been equipped with a variety of assisted driving and comfort functions. Despite these advances, vehicle safety is still one of the core issues that has received much attention, among which the safety of door opening is very important. During the opening process of a car door, a certain area will be formed. If there are obstacles in this area, there is a risk of door collision, which will cause certain damage to the vehicle and even threaten the personal safety of the driver and passengers.

[0003] In the related art, obstacles beside the car door are usually detected by ultrasonic radar or camera, or anti-collision device is set to control the door lock to avoid the risk of collision when the door is opened. Although this avoids the collision of the car door, it undoubtedly brings inconvenience to the need to open the car door, that is, there is a lack of obstacle avoidance control method when the car door is opened, and the car door cannot be opened smoothly while avoiding obstacles, which makes it difficult to provide users with a good user experience.

[0004] Therefore, it is necessary to propose a method for preventing a vehicle door from colliding with an obstacle during the vehicle door opening process, so as to ensure that the vehicle door can avoid the obstacle and open, thereby improving the convenience and safety of the vehicle for the user. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present application discloses an obstacle avoidance control method, system and vehicle-mounted terminal for opening a vehicle door, which are used to solve the technical problem in the prior art that the vehicle door cannot be opened smoothly while avoiding collision with obstacles.

[0006] In a first aspect, the present application provides an obstacle avoidance control method for opening a vehicle door, the method comprising: obtaining obstacle information outside the vehicle door, the obstacle information comprising an obstacle distance between a target obstacle and the vehicle door and a first height corresponding to the target obstacle; performing a collision risk assessment between the vehicle door and the target obstacle based on the obstacle distance, the first height and preset door opening data, the door opening data comprising a first opening distance corresponding to a preset opening angle of the door, a second height of a lower edge of the door from the ground and a third height of an upper edge of the door from the ground; if there is a collision risk between the vehicle door and the target obstacle, adjusting the door opening angle according to the obstacle distance and / or adjusting the vehicle suspension height according to the first height, so as to avoid the target obstacle during the door opening process.

[0007] In one embodiment of the present application, if there is a risk of collision between the vehicle door and the target obstacle, the door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height, including: comparing the obstacle distance with a second opening distance corresponding to the minimum opening angle of the door, the minimum opening angle being the minimum door opening angle required for the driver and passengers to get on and off the vehicle normally, and the minimum opening angle is less than the preset opening angle; if the obstacle distance is greater than the second opening distance, the door opening angle is adjusted according to the obstacle distance; if the obstacle distance is less than or equal to the second opening distance, the vehicle suspension height is adjusted according to the first height, or, adjusting the door opening angle according to the obstacle distance and adjusting the vehicle suspension height according to the first height.

[0008] In one embodiment of the present application, adjusting the door opening angle according to the obstacle distance includes: obtaining the door width and the horizontal distance of the door hinge center relative to the door hinge side; determining the door opening distance range according to the second opening distance and the obstacle distance, the distance range being greater than or equal to the second opening distance and less than the obstacle distance; obtaining the door opening angle range according to the distance range, the door width and the horizontal distance; and adjusting the preset opening angle to within the angle range.

[0009] In one embodiment of the present application, adjusting the vehicle suspension height according to the first height includes: obtaining the current vehicle suspension height, the maximum height of the vehicle suspension and the minimum height of the vehicle suspension; if the target obstacle only blocks the lower edge of the door, determining the first suspension height range for door opening according to the maximum height of the vehicle suspension and the first height, and adjusting the current vehicle suspension height to within the first suspension height range, the first height being the height of the target obstacle itself; if the target obstacle only blocks the upper edge of the door, determining the second suspension height range for door opening according to the minimum height of the vehicle suspension and the first height, and lowering the current vehicle suspension height to within the second suspension height range, the first height being the height of the target obstacle from the ground.

[0010] In one embodiment of the present application, a method for obtaining the target obstacle and the first height includes: obtaining an environmental image set outside the vehicle door, the environmental image set including images of various obstacles outside the vehicle door under different environmental conditions; marking the obstacles on each environmental image in the environmental image set, the marking content including the actual position coordinates of the obstacle, the actual type of the obstacle and the actual height of the obstacle, the actual position coordinates of the obstacle are the two-dimensional coordinates of the obstacle in the environmental image coordinate system, and the actual height of the obstacle is the height of the obstacle in the world coordinate system; constructing an initial model, and using the labeled environmental image set to train the initial model to obtain an obstacle detection model, the initial model is a target detection algorithm model under the convolutional neural network architecture; obtaining a real-time environmental image outside the vehicle door, and inputting the real-time environmental image into the obstacle detection model to identify the target obstacle and obtain the first height of the target obstacle.

[0011] In one embodiment of the present application, the method of training the initial model using the annotated environment image set to obtain an obstacle detection model includes: constructing a loss function of the initial model, wherein the loss function includes a bounding box regression loss function, a classification loss function, and a height loss function; generating a training set according to the annotated environment image set, and inputting the training set into the initial model to obtain obstacle prediction position coordinates, obstacle prediction type, and obstacle prediction height; calculating a bounding box regression loss between the obstacle prediction position coordinates and the obstacle actual position coordinates, a type loss between the obstacle prediction type and the obstacle actual type, and a height loss between the obstacle prediction height and the obstacle actual height; performing weighted summation on the bounding box regression loss, the type loss, and the height loss, adjusting the parameters of the initial detection model, and performing convergence of the initial detection model, wherein the parameters include convolution kernel weights, fully connected layer weights, and fully connected layer biases; when the result of the weighted summation reaches a preset loss value, the target detection model is obtained.

[0012] In one embodiment of the present application, the method for obtaining the actual height of the obstacle includes: obtaining the camera focal length, the camera intrinsic parameter matrix, the camera extrinsic parameters, the obstacle position coordinates and the ground height of the ground in the world coordinate system, the camera extrinsic parameters including a rotation matrix and a translation vector; according to the camera focal length and the camera intrinsic parameter matrix, converting the obstacle position coordinates into a first three-dimensional coordinate in the camera coordinate system; according to the rotation matrix, the translation vector and the camera intrinsic parameter matrix, calculating the first three-dimensional coordinates to obtain the second three-dimensional coordinates of the obstacle in the world coordinate system; calculating the difference between the height coordinate in the second three-dimensional coordinate and the ground height to obtain the actual height of the obstacle.

[0013] In one embodiment of the present application, the collision risk assessment between the vehicle door and the target obstacle is performed based on the obstacle distance, the first height and the preset door opening data, including: if the obstacle distance is less than or equal to the first opening distance, and the own height is greater than or equal to the second height, there is a collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the height above the ground is less than or equal to the third height, there is a collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the own height is less than the second height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the own height is less than the second height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the height above the ground is greater than the third height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is greater than the first opening distance, there is no collision risk between the vehicle door and the target obstacle.

[0014] In a second aspect, the present application provides an obstacle avoidance control system for opening a vehicle door, the system comprising: a data monitoring module, for acquiring obstacle information outside the vehicle door, the obstacle information comprising an obstacle distance between the target obstacle and the vehicle door and a first height corresponding to the target obstacle; a collision risk assessment module, for performing a collision risk assessment between the vehicle door and the target obstacle based on the obstacle distance, the first height and preset door opening data, the door opening data comprising a first opening distance corresponding to a preset opening angle of the door, a second height of a lower edge of the door from the ground and a third height of an upper edge of the door from the ground; an obstacle avoidance control module, for adjusting the door opening angle according to the obstacle distance and / or adjusting the vehicle suspension height according to the first height if there is a collision risk between the vehicle door and the target obstacle, so as to avoid the target obstacle during the door opening process.

[0015] In a third aspect, the present application provides a vehicle-mounted terminal, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the vehicle-mounted terminal implements the obstacle avoidance control method for opening a vehicle door as described in the first aspect.

[0016] As described above, the obstacle avoidance control method, system and vehicle-mounted terminal for door opening provided by the embodiments of the present application have the following beneficial effects:

[0017] First, obstacle information outside the door is obtained, and the obstacle information includes an obstacle distance between the target obstacle and the door and a first height corresponding to the target obstacle. According to the obstacle distance, the first height and preset door opening data, a collision risk assessment is performed between the door and the target obstacle. The door opening data includes a first opening distance corresponding to a preset opening angle of the door, a second height of a lower edge of the door from the ground and a third height of an upper edge of the door from the ground. If there is a collision risk between the door and the target obstacle, the door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height. According to the obstacle situation outside the door, the door opening angle and / or the vehicle suspension height are intelligently adjusted, and the system is fully integrated with the vehicle's suspension control system to realize intelligent height adjustment to assist the door opening, so that the door can be opened to avoid obstacles and avoid collisions. The door will not be directly locked or prohibited from opening due to the presence of obstacles. For users, this brings convenience in getting on and off the vehicle, improves the safety of the entire vehicle, and enhances the user experience.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 is a schematic diagram of an implementation environment of an obstacle avoidance control system for opening a vehicle door, shown in an exemplary embodiment of the present application;

[0021] Figure 2 is a flow chart of an obstacle avoidance control method for opening a vehicle door, shown in an exemplary embodiment of the present application;

[0022] Figure 3is a block diagram of an obstacle avoidance control system for door opening shown in an exemplary embodiment of the present application;

[0023] Figure 4 It is a structural schematic diagram of a vehicle-mounted terminal provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0025] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the form, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0026] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0027] During the opening process of a car door, a certain area will be formed. If there is an obstacle in this area, there is a risk of door collision, which will cause certain damage to the vehicle and even threaten the personal safety of the driver and passengers. At present, obstacles next to the car door are usually detected by ultrasonic radar and camera, or anti-collision devices are set to control the locking of the car door to avoid the risk of collision during the opening process. However, the inventor of this application has found that although this avoids the collision of the car door, it undoubtedly brings inconvenience to the need to open the car door, that is, there is a lack of obstacle avoidance control method when the car door is opened, and the car door cannot be opened smoothly while avoiding obstacles, making it difficult to provide users with a good user experience.

[0028] Therefore, see Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an implementation environment of an obstacle avoidance control system for door opening, shown in an exemplary embodiment of the present application. Figure 1As shown, the implementation environment includes a vehicle 110 and an obstacle avoidance control system 120 for opening the door, wherein the obstacle avoidance control system 120 for opening the door is embedded in the vehicle 110, and is used to implement obstacle avoidance control for opening the door of the vehicle 110. The obstacle avoidance control system 120 for opening the door includes but is not limited to a vehicle computer system, a vehicle-mounted computer, etc., and intelligently adjusts the door opening angle and / or the vehicle suspension height according to the obstacle situation outside the door, and is fully integrated with the vehicle's suspension control system to achieve intelligent height adjustment to assist the door opening, so that the door can be opened to avoid obstacles and avoid collisions with the door, and the door will not be directly locked or prohibited from opening due to the presence of obstacles. For users, it brings convenience to getting on and off the vehicle, improves the safety of the entire vehicle, and enhances the user experience.

[0029] See also Figure 2 , Figure 2 is a flow chart of an obstacle avoidance control method for opening a vehicle door, shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is a schematic diagram. It should be understood that the method may also be applicable to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0030] like Figure 2 As shown, in an exemplary embodiment, the obstacle avoidance control method for opening the door includes at least steps S210 to S230, which are described in detail as follows:

[0031] Step S210: Obstacle information outside the vehicle door is obtained, where the obstacle information includes an obstacle distance between the target obstacle and the vehicle door and a first height corresponding to the target obstacle.

[0032] As a possible embodiment, the obstacle distance may be measured by a door anti-collision radar, and the first height may be measured by a vehicle-mounted camera, such as a surround-view monitoring camera.

[0033] Among them, the on-board camera and door anti-collision radar enable the vehicle to better understand the external environment of the door, and use the automatic adjustment suspension, on-board camera and door anti-collision radar to assist the door in opening in an obstacle environment, realizing intelligent obstacle avoidance control of door opening.

[0034] In this embodiment, when the current car triggers a door opening event, the on-board camera is activated to perform a 360-degree environment scan to identify obstacles outside the door, and uses image recognition technology to locate the obstacles to determine the position and height of the obstacles, i.e., the first height H. obstacle At the same time, the door anti-collision radar is activated to monitor the distance to the obstacle, that is, the obstacle distance D doorobstacle .

[0035] It should also be noted that before the door is opened, the obstacle information outside the door will be monitored. During the door opening process, the obstacle information monitoring is also a continuous process. If new obstacle information is detected during the opening process, it will be determined whether a collision occurs based on the current state of the door. If so, the door opening angle and / or vehicle suspension height will be readjusted to avoid the door from colliding during the opening process. In addition, there is at least one target obstacle, and a collision risk assessment will be performed for each target obstacle. If there are multiple target obstacles with collision risks, obstacle avoidance control will be performed in combination with the distances of multiple obstacles and the first heights of multiple target obstacles.

[0036] In a possible embodiment, when it is detected that the current car triggers a door opening event, the target door to be opened is first determined, and then obstacle detection is performed based on the target door, so that the target door can avoid obstacles and be opened.

[0037] In one embodiment, a method for obtaining a target obstacle and a first height includes: obtaining an environmental image set outside a vehicle door, the environmental image set including images of various obstacles outside the vehicle door under different environmental conditions; marking obstacles on each environmental image in the environmental image set, wherein the marking content includes the actual position coordinates of the obstacle, the actual type of the obstacle and the actual height of the obstacle, the actual position coordinates of the obstacle are the two-dimensional coordinates of the obstacle in the environmental image coordinate system, and the actual height of the obstacle is the height of the obstacle in the world coordinate system; constructing an initial model, and training the initial model with the labeled environmental image set to obtain an obstacle detection model, wherein the initial model is a target detection algorithm model under a convolutional neural network architecture; obtaining a real-time environmental image outside the vehicle door, and inputting the real-time environmental image into the obstacle detection model to identify the target obstacle and obtain the height data of the target obstacle.

[0038] Among them, the environmental image set includes images of various obstacles outside the car door under different environmental conditions, such as different lighting conditions, different weather conditions, different environmental scenes, etc. Various obstacles, that is, obstacle types can be obstacles blocking the lower edge of the car door or the upper edge of the car door. For example, stone pillars and stairs on the ground can be obstacles blocking the lower edge of the car door, and branches on trees can be obstacles blocking the upper edge of the car door. In this way, the diversity of the data set is guaranteed, so that obstacle detection model training can be carried out, which can improve the robustness of the model algorithm.

[0039] In this embodiment, an obstacle detection model is used to identify and locate the target obstacle and extract the first height of the target obstacle. The obstacle detection model is a target detection algorithm model under a CNN (Convolutional Neural Network) architecture, wherein the target detection algorithm, such as YOLO (You Only Look Once), combines the convolutional neural network architecture with the target detection algorithm. When obstacle features are extracted based on the convolutional neural network architecture so that the obstacle in the image can be identified and the obstacle height can be extracted, the target detection algorithm is continued to be used to use the classification task to identify the category of the obstacle. In this way, the reliability of the identification and positioning of the target obstacle and the extraction of the height data is ensured.

[0040] In this embodiment, the annotated content includes the actual position coordinates of the obstacle, the actual type of the obstacle and the actual height of the obstacle, so that it can accurately identify and locate the obstacle, identify the type and extract the height, which can significantly improve the performance of the training model.

[0041] It should be noted that, because obstacles include the heights that block the lower edge of the door and the heights that block the upper edge of the door, when the target obstacle is an obstacle that blocks the lower edge of the door, the first height is the target obstacle's own height, that is, the height of the target obstacle's highest point from the ground; when the target obstacle is an obstacle that blocks the upper edge of the door, the first height is the target obstacle's height from the ground, that is, the height of the obstacle's lowest point from the ground. Therefore, the marked actual height of the obstacle will correspond to the actual type of the obstacle, which may be the actual height of the obstacle itself or the actual height of the obstacle from the ground.

[0042] It should also be noted that the actual position coordinates of the obstacle are the two-dimensional coordinates (x, y) of the obstacle in the environment image coordinate system, which can be annotated with a bounding box (x min ,y min ,x max ,y max ) indicates that the actual height of the obstacle needs to be annotated in 3D, which is the height of the obstacle in the world coordinate system. Among them, the environmental image coordinate system is the coordinate system of the image plane, which describes the pixel position in the image. For example, the origin is the lower left corner of the image, the horizontal direction from left to right is the x-axis, and the vertical direction from top to bottom is the y-axis; the world coordinate system is a three-dimensional coordinate system in the real world, which describes the position and posture of the obstacle in three-dimensional space. You can choose any position as the origin, and define three orthogonal directions in three-dimensional space as the x-axis, y-axis, and z-axis.

[0043] As a possible embodiment, before using the labeled environmental image set to train the initial model, each environmental image is preprocessed, wherein the preprocessing includes at least one of noise removal, grayscale, normalization, standardization, and image enhancement.

[0044] As a possible embodiment, the convolutional neural network architecture includes an input layer, a convolution layer, a pooling layer and a fully connected layer.

[0045] The input layer receives the preprocessed image data. The image dimension data of the input layer is (batch_size, channels, height, width), where batch_size is the number of environment images in each batch, channels is the number of color channels of the environment image, for example, for RGB (Red, Green, Blue) images, it is 3, and height and width are the size of the image; for label input, the label data (actual position coordinates of obstacles, actual type of obstacles and actual height of obstacles) is an additional input, which can be attached to the image dimension data or as an independent input layer. The label data dimension is (Num_Objects, Num_Features), where Num_Objects is the number of obstacles in the environment image, and Num_Features is the feature of each obstacle, namely, the actual position coordinates of the obstacle, the actual type of obstacle and the actual height of the obstacle.

[0046] Convolutional layer, using multiple convolutional layers to automatically extract image features, the calculation formula of the convolution operation is:

[0047]

[0048] Among them, f is the input feature map, g is the convolution kernel, (i, j) is the position index on the output feature map, (m, n) is the position index on the convolution kernel, f(i+m, j+n) is the pixel value of the input feature map at position (m, n), g(m, n) is the weight of the convolution kernel at position (m, n), and the sum ∑m and ∑n is the accumulation of all weights of the convolution kernel. H g and W g are the height and width of the convolution kernel.

[0049] The pooling layer uses maximum pooling to reduce the spatial dimension of the feature and increase the invariance to image displacement. The purpose is to reduce the spatial size of the feature map, thereby reducing the computational complexity and the number of parameters. The calculation formula of the pooling operation is:

[0050] max pooling(x)=max i,j (x(i,j)) Formula (2)

[0051] Among them, max pooling() is a pooling operation, x is the input feature map, x(i,j) represents the position of the sliding window on the input feature map, max i,j Represents the maximum pooling operation, which selects the maximum value in the window as the output.

[0052] Fully connected layer, a fully connected layer is used at the end of the convolutional neural network to perform classification or regression tasks. The output of the fully connected layer can be expressed as:

[0053] y=Wx+b Formula (3)

[0054] Where y is the output vector, W is the weight matrix, b is the bias term, and x is the input vector.

[0055] It should be noted that the weights of the convolution kernel at each position of the image, the weight matrix W and the bias term b are learnable parameters of the network, namely the above-mentioned convolution kernel weights, fully connected layer weights and fully connected layer biases. The initial values ​​of these parameters are usually randomly initialized and can be adjusted through the back propagation algorithm and gradient descent during training. In addition, the input x of the fully connected layer can be a one-dimensional feature vector from the previous layer. In the convolutional neural network, there is usually a Flatten operation before the fully connected layer to convert the multi-dimensional feature map into a one-dimensional vector.

[0056] In one embodiment, the initial model is trained using the annotated environmental image set to obtain an obstacle detection model, including: constructing a loss function of the initial model, the loss function including a bounding box regression loss function, a classification loss function and a height loss function; generating a training set according to the annotated environmental image set, and inputting the training set into the initial model to obtain obstacle prediction position coordinates, obstacle prediction type and obstacle prediction height; calculating the bounding box regression loss between the obstacle prediction position coordinates and the obstacle actual position coordinates, the type loss between the obstacle prediction type and the obstacle actual type, and the height loss between the obstacle prediction height and the obstacle actual height; performing weighted summation on the bounding box regression loss, the type loss and the height loss, adjusting the parameters of the initial detection model, and performing convergence of the initial detection model, the parameters including the convolution kernel weight, the fully connected layer weight and the fully connected layer bias; when the result of the weighted summation reaches a preset loss value, the target detection model is obtained.

[0057] In this embodiment, the parameters of the initial model are updated by comprehensively considering the bounding box regression loss, type loss and height loss, which can significantly improve the performance and robustness of the obstacle detection model, so that the obstacle detection model can reliably predict the position, type and height of the obstacle, thereby ensuring the accuracy of the door opening obstacle avoidance control.

[0058] In target detection, the classification loss takes into account the problem of category imbalance. If the number of obstacles of different categories in the data set is quite different, the weighted cross entropy loss function is used to train the classification task. The classification loss function is:

[0059]

[0060] Among them, L 1 is the classification loss, M is the number of categories, c is the obstacle category index (indicating category c), β c is the weight of category c, y o,c is the true category label, is the probability that the model predicts that it belongs to class c.

[0061] It should be noted that y o,c is the true category label, which is set to the inverse of the category frequency. Assuming that the data set has multiple categories, for category c, the category frequency refers to the proportion of the number of samples belonging to category c in the data set to the total number of samples. If n is used c represents the number of samples belonging to category c, N represents the total number of samples, then the frequency f of category c c =n c / N, the reciprocal of the category frequency is 1 / fc=N / n c .

[0062] In object detection, the bounding box is described by four parameters, corresponding to the center point coordinates (x, y) and width and height (w, h) of the bounding box. Smooth L1 Loss is applied to these four parameters respectively and then summed up. This can comprehensively consider the errors of the bounding box in different aspects, so that the model can optimize these four parameters at the same time during the training process, thereby more accurately predicting the position and size of the bounding box. The bounding box regression loss function is:

[0063]

[0064] Among them, L 2 is the bounding box regression loss, 4 is the number of bounding box parameters, is the i-th parameter of the ground-truth bounding box. i is the i-th parameter of the model’s predicted bounding box, α i is the weight of the i-th bounding box parameter.

[0065] It should be noted that the smooth L1 loss function is expressed as L2 loss when the error value is small, and as L1 loss when the error value is large, which is specifically reflected as follows:

[0066]

[0067] Among them, τ is a dynamic threshold, which can be set according to the distribution of errors in the training data.

[0068] In target detection, the height loss is determined by the square error between the predicted height and the true height. The height loss function is:

[0069] L 3 =(h pred -h true ) 2 Formula (7)

[0070] Among them, L 3 is the height loss, h pred is the model predicted height, h true is the actual height.

[0071] In this embodiment, a weighted sum is performed on the bounding box regression loss, the type loss, and the height loss, that is, the total loss function is:

[0072] L=λ 1 L 1 +λ 2 L 2 +λ 3 L 3 Formula (8)

[0073] Among them, L is the total loss, L 1 is the classification loss, L 2 is the bounding box regression loss, L 3 is the height loss, λ 1 , 2 With λ 3 It is an adaptive weight system used to characterize the importance of different loss terms.

[0074] In this embodiment, a weighted sum is performed on the bounding box regression loss, type loss, and height loss, and the parameters of the initial detection model are adjusted. When the result of the weighted sum reaches a preset loss value, it indicates that the initial model has reached the predetermined standard. At this time, the target detection model is obtained.

[0075] In a possible embodiment, after the model is output, post-processing is performed to improve the quality of the prediction results, wherein the post-processing includes at least one of non-maximum suppression and confidence filtering, non-maximum suppression is used to remove overlapping prediction bounding boxes, and confidence filtering is used to filter low-quality predictions through a confidence threshold.

[0076] In a possible embodiment, the mAP (average precision) indicator is used to evaluate the model performance, and the model parameters are optimized according to the evaluation results. Finally, the trained model is deployed on the vehicle's computing platform to process the environmental images captured by the camera in real time to identify obstacles and extract height features.

[0077] In one embodiment, the method for obtaining the actual height of an obstacle includes: obtaining the camera focal length, the camera intrinsic parameter matrix, the camera extrinsic parameters, the obstacle position coordinates and the ground height in the world coordinate system, the camera extrinsic parameters include the rotation matrix and the translation vector; according to the camera focal length and the camera intrinsic parameter matrix, the obstacle position coordinates are converted into the first three-dimensional coordinates in the camera coordinate system; according to the rotation matrix, the translation vector and the camera intrinsic parameter matrix, the first three-dimensional coordinates are calculated to obtain the second three-dimensional coordinates of the obstacle in the world coordinate system; the difference between the height coordinate in the second three-dimensional coordinate and the ground height is calculated to obtain the actual height of the obstacle.

[0078] In this embodiment, the obstacle height feature H is extracted obstacle This is achieved through 3D reconstruction, that is, combining depth information and 2D images, and using 3D reconstruction technology to estimate the height of obstacles. Specifically, a vision-based method is used to estimate the true height of obstacles. By using the internal and external parameters of the camera and the position information of the obstacle in the image, the pixel coordinates in the image can be more accurately mapped to the three-dimensional space, and the height information of the obstacle can be obtained more accurately without the need for additional sensors. In this way, the accuracy of the obtained obstacle height is guaranteed.

[0079] It should be noted that the focal length of the camera determines the proportional relationship between the size of the object in the image and its actual size; the camera intrinsic parameter matrix contains information such as the focal length of the camera and the coordinates of the image center point, which is used to project the points in the image coordinate system to the world coordinate system; the camera extrinsic parameters, including the rotation matrix and translation vector, describe the position and posture of the camera relative to the world coordinate system; the obstacle position coordinates are the position information of the obstacles detected in the image coordinate system; the ground height is the height of the ground in the world coordinate system.

[0080] In this embodiment, according to the camera focal length and the camera intrinsic parameter matrix, the obstacle position coordinates are converted into the first three-dimensional coordinates in the camera coordinate system. The conversion formula is specifically:

[0081]

[0082]

[0083]

[0084] Among them, X c , Y c , Z c The overall representation is the first three-dimensional coordinate in the camera coordinate system, K is the camera internal parameter matrix, x and y overall represent the obstacle position coordinates, f is the camera focal length, [1,0,0] T is a unit vector along the X-axis, [0,1,0]T is a unit vector along the Y axis, [0,0,1] T is a unit vector along the Z axis.

[0085] The camera intrinsic parameter matrix is:

[0086]

[0087] Among them, f x 、f y It is the pixel representation of the focal length in the x-axis and y-axis directions of the image, which determines the degree of magnification of the obstacle in the image. x 、c y are the coordinates of the image center.

[0088] In this embodiment, the first three-dimensional coordinates are calculated according to the rotation matrix and the translation vector to obtain the second three-dimensional coordinates of the obstacle in the world coordinate system. The specific conversion formula is:

[0089]

[0090] Among them, X world , Y world , Z world The overall representation is the second three-dimensional coordinate in the world coordinate system, R is the rotation matrix, t is the translation vector, K is the camera intrinsic parameter matrix, X c , Y c , Z c The overall representation is the first three-dimensional coordinate in the camera coordinate system.

[0091] In this embodiment, the difference between the height coordinate in the second three-dimensional coordinate and the ground height is calculated to obtain the obstacle height, and the calculation formula is:

[0092] H obstacle =Z world -Z ground Formula (14)

[0093] Among them, H obstacle is the obstacle height, Z world is the Z coordinate of the highest point of the obstacle in the world coordinate system, Z ground is the Z coordinate of the ground in the world coordinate system.

[0094] In addition, the difference between the height coordinate in the second three-dimensional coordinate and the height above the ground is calculated to obtain the actual height of the obstacle, including: when the obstacle is an obstacle blocking the lower edge of the vehicle door, the difference between the maximum value of the height coordinate in the second three-dimensional coordinate and the height above the ground is calculated to obtain the actual height of the obstacle itself; when the obstacle is an obstacle blocking the upper edge of the vehicle door, the difference between the minimum value of the height coordinate in the second three-dimensional coordinate and the height above the ground is calculated to obtain the actual height of the obstacle from the ground.

[0095] Step S220, performing a collision risk assessment between the vehicle door and the target obstacle based on the obstacle distance, the first height and the preset door opening data, the door opening data including a first opening distance corresponding to a preset opening angle of the door, a second height of the lower edge of the door from the ground and a third height of the upper edge of the door from the ground.

[0096] The door opening data is the data corresponding to the preset door opening, including the first opening distance corresponding to the preset opening angle of the door, the second height of the lower edge of the door from the ground, and the third height of the upper edge of the door from the ground. The preset opening angle refers to the door opening angle set by the user, and the first opening distance refers to the distance from the vehicle body to the side of the door away from the door hinge after the door is opened according to the preset opening angle.

[0097] In one embodiment, a collision risk assessment is performed between the vehicle door and the target obstacle based on the obstacle distance, the first height and the preset door opening data, including: if the obstacle distance is less than or equal to the first opening distance, and the own height is greater than or equal to the second height, there is a collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the height from the ground is less than or equal to the third height, there is a collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the own height is less than the second height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the own height is less than the second height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is less than or equal to the first opening distance, and the height from the ground is greater than the third height, there is no collision risk between the vehicle door and the target obstacle; if the obstacle distance is greater than the first opening distance, there is no collision risk between the vehicle door and the target obstacle.

[0098] It should be understood that if the obstacle data is greater than the first opening distance, there is no risk of collision between the vehicle door and the target obstacle, that is, as long as the obstacle data is greater than the first opening distance, no matter whether the obstacle is blocking the upper edge or the lower edge of the vehicle door, no matter what the relationship between the first height and the second height and the third height is, there is no risk of collision between the vehicle door and the target obstacle.

[0099] In this embodiment, the obstacle distance, the first height and the preset door opening data are combined to comprehensively evaluate the collision risk between the door and the target obstacle, thereby ensuring the reliability of the evaluation result.

[0100] As a possible embodiment, in terms of distance, the collision risk assessment formula is:

[0101]

[0102] Among them, Risk 1Indicates the collision risk at distance, 1 means there is a collision risk at distance, 0 means there is no collision risk at distance, D obstacle is the obstacle distance, D dooropen is the first opening distance.

[0103] When the target obstacle is an obstacle blocking the lower edge of the door, the collision risk assessment formula in terms of height is:

[0104]

[0105] Among them, Risk 2 Indicates the collision risk at height, 1 means there is a collision risk at height, 0 means there is no collision risk at height, H obstacle is the first height, H dooropen It is the second highest.

[0106] When the target obstacle is an obstacle blocking the lower edge of the door, the collision risk assessment formula in terms of height is:

[0107]

[0108] Among them, Risk d Indicates the collision risk at height, 1 means there is a collision risk at height, 0 means there is no collision risk at height, H obstacle is the first height, H dooropen It is the third height.

[0109] In this embodiment, only (Risk 1 ,Risk 2 ) is (1,1), or (Risk 1 ,Risk 3 ) is (1,1), it is determined that there is a collision risk between the door and the target obstacle, and (Risk 1 ,Risk 2 ) is (0,1), (1,0), (0,0), or (Risk 1 ,Risk 3 ) is (0,1), (1,0), (0,0), it is determined that there is no collision risk between the door and the target obstacle.

[0110] Step S230: If there is a risk of collision between the vehicle door and the target obstacle, the vehicle door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height, so that the target obstacle is avoided during the door opening process.

[0111] Specifically, if there is a risk of collision between the vehicle door and the target obstacle, the door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height, including: comparing the obstacle distance with the second opening distance corresponding to the minimum opening angle of the door, the minimum opening angle being the minimum door opening angle required for the driver and passengers to get on and off the vehicle normally, and the minimum opening angle is less than the preset opening angle; if the obstacle distance is greater than the second opening distance, the door opening angle is adjusted according to the obstacle distance; if the obstacle distance is less than or equal to the second opening distance, the vehicle suspension height is adjusted according to the first height, or, the door opening angle is adjusted according to the obstacle distance and the vehicle suspension height is adjusted according to the first height.

[0112] In this embodiment, when there is a risk of collision between the vehicle door and the target obstacle, if the collision can be avoided by adjusting the door opening angle, the door opening angle is adjusted only according to the obstacle distance; if the collision cannot be avoided by adjusting the door opening angle, the vehicle suspension height can be adjusted according to the first height, or both the vehicle suspension height is adjusted according to the first height and the door opening angle is adjusted according to the obstacle distance.

[0113] In this way, by combining the adjustment of the door opening angle and the adjustment of the vehicle suspension height, obstacle avoidance control of the door opening can be achieved together, and the door can be opened smoothly while avoiding collisions, which brings convenience to users when getting on and off the vehicle and improves the safety of the entire vehicle.

[0114] In one embodiment, adjusting the door opening angle according to the obstacle distance includes: obtaining the door width and the horizontal distance of the door hinge center relative to the door hinge side; determining the door opening distance range according to the second opening distance and the obstacle distance, the distance range being greater than or equal to the second opening distance and less than the obstacle distance; obtaining the door opening angle range according to the distance range, the door width and the horizontal distance; and adjusting the preset opening angle to within the angle range.

[0115] In this embodiment, the relationship between the door opening distance and the door opening angle is as follows:

[0116] D dooroaen =(L hinge +W door )sinθ dooropen Formula (18)

[0117] Among them, D dooropen is the opening distance, L hinge is the horizontal distance of the door hinge center relative to the door hinge side, W door is the door width, θ dooropen is the door opening angle.

[0118] In this embodiment, firstly, the distance range for obstacle avoidance when the door is opened is determined based on the second opening distance and the obstacle distance, that is, the distance range is [second opening distance, obstacle distance), that is, the opening distance after the door is opened is within this distance range and will not collide with the target obstacle. Then, the distance range is converted into the angle range for the door opening, that is, according to the above formula (18), the angle range is obtained by calculation based on the distance range, the door width and the horizontal distance. Finally, the door opening angle is adjusted to within the angle range to realize obstacle avoidance control of the door opening.

[0119] In this way, the horizontal distance of the door hinge center relative to the door hinge side is fully considered, and the angle range of door opening obstacle avoidance is determined in combination with the door width, the second opening distance and the obstacle distance, thereby ensuring the reliability of door opening obstacle avoidance control.

[0120] For example, the minimum opening angle is when the door is half open, that is, the second opening distance is D halfopen , then the minimum opening angle is θ halfopen , the preset opening angle is that the door is fully open, that is, the first opening distance is D fullopen , then the preset opening angle is θ fullopen Then, the distance range is [D halfopen , obstacle distance), the angle range [θ halfopen , the opening angle corresponding to the obstacle distance), and finally the door opening angle needs to be controlled at [θ halfopen , the opening angle corresponding to the obstacle distance).

[0121] In a possible embodiment, before detecting that the current car triggers a door opening event, the body size of the driver and passenger corresponding to each door is evaluated, and the target opening angle of each door is determined based on the mapping relationship between the body size and the door opening angle. After obtaining the door opening obstacle avoidance angle range, it is determined whether the target opening angle range is within the door opening obstacle avoidance angle range, and if so, the door opening angle is adjusted to the target opening angle.

[0122] As a possible embodiment, this can avoid collision of the vehicle doors, enable the vehicle doors to be opened smoothly, and enable the corresponding drivers and passengers to get on and off the vehicle smoothly.

[0123] In one embodiment, the vehicle suspension height is adjusted according to the first height, including: obtaining the current vehicle suspension height, the maximum height of the vehicle suspension and the minimum height of the vehicle suspension; if the target obstacle only blocks the lower edge of the door, then according to the maximum height of the vehicle suspension and the first height, the first suspension height range for door opening is determined, and the current vehicle suspension height is adjusted to within the first suspension height range, and the first height is the target obstacle's own height; if the target obstacle only blocks the upper edge of the door, then according to the minimum height of the vehicle suspension and the first height, the second suspension height range for door opening is determined, and the current vehicle suspension height is lowered to within the second suspension height range, and the first height is the height of the target obstacle from the ground.

[0124] In this embodiment, if the target obstacle only blocks the lower edge of the door, the first suspension height range for door opening is determined according to the maximum height of the vehicle suspension and the self-height of the target obstacle, that is, the first suspension height range is (self-height, maximum height of vehicle suspension), and finally the current vehicle suspension height is adjusted to the first suspension height range; if the target obstacle only blocks the upper edge of the door, the second suspension height range for door opening is determined according to the minimum height of the vehicle suspension and the height from the ground of the target obstacle, that is, the second suspension height range is [minimum height of vehicle suspension, height from the ground), and finally the current vehicle suspension height is adjusted to the second suspension height range. In this way, considering different obstruction situations, combining the extreme values ​​of the door suspension height (maximum height of vehicle suspension, minimum height of vehicle suspension) and the first height, the obstacle avoidance control of door opening is realized, and the reliability of the obstacle avoidance control of door opening is ensured.

[0125] In the embodiment of the present application, it should be noted that adjusting the door opening angle according to the obstacle data and / or adjusting the vehicle suspension height according to the first height is applicable to getting on or getting off the vehicle.

[0126] For example, the existing vehicle suspension adjustment system usually returns to the default height after the vehicle is powered off. When getting in the vehicle, the door may not be opened normally due to obstacles. When the vehicle is powered on again, the intelligent door opening angle and / or vehicle suspension height will be adjusted according to the obstacles outside the vehicle body to facilitate the opening of the door.

[0127] In one possible embodiment, the user is informed through a vehicle information display screen or a mobile application that the vehicle suspension is being adjusted and the door opening status, and a manual intervention option is provided, allowing the user to manually adjust the door opening angle or the vehicle suspension height according to actual conditions, or cancel the automatic adjustment of the door opening angle or the vehicle suspension height.

[0128] In a possible embodiment, data information of each door opening angle and vehicle suspension height adjustment is recorded, and the data information includes obstacle information, door opening data, height before and after suspension adjustment, etc. The recorded data information is used to optimize at least one of the obstacle detection model, door opening angle adjustment strategy, and vehicle suspension height adjustment strategy, so as to further improve the reliability of door opening obstacle avoidance control.

[0129] The above-mentioned obstacle avoidance control method for door opening first obtains obstacle information outside the door, and the obstacle information includes the obstacle distance between the target obstacle and the door and the first height corresponding to the target obstacle. According to the obstacle distance, the first height and the preset door opening data, a collision risk assessment is performed between the door and the target obstacle. The door opening data includes the first opening distance corresponding to the preset opening angle of the door, the second height of the lower edge of the door from the ground and the third height of the upper edge of the door from the ground. If there is a collision risk between the door and the target obstacle, the door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height. According to the obstacle situation outside the door, the door opening angle and / or the vehicle suspension height are intelligently adjusted, and the vehicle is fully integrated with the suspension control system of the vehicle to realize intelligent height adjustment to assist the door opening, so that the door can be opened to avoid obstacles and avoid collisions with the door. The door will not be directly locked or prohibited from opening due to the presence of obstacles. For users, it brings convenience to getting on and off the vehicle, improves the safety of the entire vehicle, and enhances the user experience.

[0130] See also Figure 3 , Figure 3 is a block diagram of an obstacle avoidance control system for vehicle door opening shown in an exemplary embodiment of the present application. The system can be applied to Figure 1 The implementation environment shown is a schematic diagram. It should be understood that the system may also be applicable to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the system is applicable.

[0131] like Figure 3 As shown, in an exemplary embodiment, the obstacle avoidance control system 300 for opening the vehicle door includes at least a data monitoring module 310, a collision risk assessment module 320 and an obstacle avoidance control module 330, which are described in detail as follows:

[0132] The data monitoring module 310 is used to obtain obstacle information outside the vehicle door, where the obstacle information includes an obstacle distance between the target obstacle and the vehicle door and a first height corresponding to the target obstacle;

[0133] A collision risk assessment module 320, for performing a collision risk assessment between the vehicle door and the target obstacle according to the obstacle distance, the first height and preset vehicle door opening data, wherein the vehicle door opening data includes a first opening distance corresponding to a preset opening angle of the vehicle door, a second height of the lower edge of the vehicle door from the ground and a third height of the upper edge of the vehicle door from the ground;

[0134] The obstacle avoidance control module 330 is used to adjust the door opening angle according to the obstacle distance and / or the vehicle suspension height according to the first height if there is a collision risk between the door and the target obstacle, so as to avoid the target obstacle during the door opening process.

[0135] The above-mentioned obstacle avoidance control system for door opening first obtains obstacle information outside the door, and the obstacle information includes the obstacle distance between the target obstacle and the door and the first height corresponding to the target obstacle. According to the obstacle distance, the first height and the preset door opening data, the collision risk between the door and the target obstacle is evaluated. The door opening data includes the first opening distance corresponding to the preset opening angle of the door, the second height of the lower edge of the door from the ground and the third height of the upper edge of the door from the ground. If there is a collision risk between the door and the target obstacle, the door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height. According to the obstacle situation outside the door, the door opening angle and / or the vehicle suspension height are intelligently adjusted, and the vehicle is fully integrated with the suspension control system of the vehicle to realize intelligent height adjustment to assist the door opening, so that the door can be opened to avoid obstacles and avoid collisions. The door will not be directly locked or prohibited from opening due to the presence of obstacles. For users, it brings convenience to getting on and off the vehicle, improves the safety of the entire vehicle, and enhances the user experience.

[0136] It should be noted that the obstacle avoidance control system for opening the vehicle door provided in the above embodiment and the obstacle avoidance control method for opening the vehicle door provided in the above embodiment belong to the same concept, and the contents of the operations performed by each module have been described in detail in the method embodiment and will not be repeated here.

[0137] See also Figure 4 , Figure 4 It is a structural schematic diagram of a vehicle-mounted terminal provided in one embodiment of the present application. Figure 4 The structure diagram of the computer system of the vehicle terminal suitable for implementing the embodiment of the present application is shown. It should be noted that: Figure 4 The computer system 400 of the vehicle-mounted terminal shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0138] like Figure 4As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through the bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0139] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0140] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.

[0141] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0143] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. An obstacle avoidance control method for opening a vehicle door, characterized in that: The method comprises: Obtaining obstacle information outside the vehicle door, the obstacle information including an obstacle distance between the target obstacle and the vehicle door and a first height corresponding to the target obstacle; performing a collision risk assessment between the vehicle door and the target obstacle according to the obstacle distance, the first height and preset vehicle door opening data, wherein the vehicle door opening data includes a first opening distance corresponding to a preset opening angle of the vehicle door, a second height of a lower edge of the vehicle door from the ground and a third height of an upper edge of the vehicle door from the ground; If there is a risk of collision between the vehicle door and the target obstacle, the vehicle door opening angle is adjusted according to the obstacle distance and / or the vehicle suspension height is adjusted according to the first height so that the target obstacle is avoided during the door opening process.

2. The door opening obstacle avoidance control method according to claim 1, characterized in that: If there is a collision risk between the vehicle door and the target obstacle, adjusting the door opening angle according to the obstacle distance and / or adjusting the vehicle suspension height according to the first height includes: Comparing the obstacle distance with a second opening distance corresponding to a minimum door opening angle, wherein the minimum door opening angle is the minimum door opening angle required for a driver or passenger to normally get on or off the vehicle, and the minimum door opening angle is smaller than the preset opening angle; If the obstacle distance is greater than the second opening distance, adjusting the door opening angle according to the obstacle distance; If the obstacle distance is less than or equal to the second opening distance, the vehicle suspension height is adjusted according to the first height, or the door opening angle is adjusted according to the obstacle distance and the vehicle suspension height is adjusted according to the first height.

3. The obstacle avoidance control method for door opening according to claim 2, characterized in that: The adjusting the door opening angle according to the obstacle distance comprises: Get the door width and the horizontal distance of the door hinge center relative to the door hinge side; Determining a door opening distance range according to the second opening distance and the obstacle distance, wherein the distance range is greater than or equal to the second opening distance and less than the obstacle distance; Obtaining an angle range of door opening according to the distance range, the door width and the horizontal distance; The preset opening angle is adjusted to within the angle range.

4. The obstacle avoidance control method for door opening according to claim 1, characterized in that: The adjusting the vehicle suspension height according to the first height comprises: Get the current vehicle suspension height, the maximum vehicle suspension height and the minimum vehicle suspension height; If the target obstacle only blocks the lower edge of the door, a first suspension height range for opening the door is determined according to the maximum suspension height of the vehicle and the first height, and the current vehicle suspension height is adjusted to within the first suspension height range, where the first height is the height of the target obstacle itself; If the target obstacle only blocks the upper edge of the door, the second suspension height range for door opening is determined based on the minimum height of the vehicle suspension and the first height, and the current vehicle suspension height is lowered to within the second suspension height range, where the first height is the height of the target obstacle from the ground.

5. The obstacle avoidance control method for door opening according to claim 1, characterized in that: The method for obtaining the target obstacle and the first height includes: Acquire an environmental image set outside the vehicle door, wherein the environmental image set includes images of various obstacles outside the vehicle door under different environmental conditions; Annotating obstacles on each environment image in the environment image set, wherein the annotation content includes the actual position coordinates of the obstacle, the actual type of the obstacle, and the actual height of the obstacle, wherein the actual position coordinates of the obstacle are the two-dimensional coordinates of the obstacle in the environment image coordinate system, and the actual height of the obstacle is the height of the obstacle in the world coordinate system; Constructing an initial model, and training the initial model using the annotated environment image set to obtain an obstacle detection model, wherein the initial model is a target detection algorithm model under a convolutional neural network architecture; A real-time environment image outside the vehicle door is acquired, and the real-time environment image is input into the obstacle detection model to identify the target obstacle and obtain the first height of the target obstacle.

6. The obstacle avoidance control method for door opening according to claim 5, characterized in that: The step of training the initial model using the labeled environment image set to obtain an obstacle detection model includes: Constructing a loss function of the initial model, wherein the loss function includes a bounding box regression loss function, a classification loss function, and a height loss function; Generate a training set according to the annotated environment image set, and input the training set into the initial model to obtain obstacle prediction position coordinates, obstacle prediction type and obstacle prediction height; Calculating a bounding box regression loss between the predicted position coordinates of the obstacle and the actual position coordinates of the obstacle, a type loss between the predicted type of the obstacle and the actual type of the obstacle, and a height loss between the predicted height of the obstacle and the actual height of the obstacle; Performing a weighted summation on the bounding box regression loss, the type loss, and the height loss, adjusting parameters of the initial detection model, and performing convergence of the initial detection model, wherein the parameters include convolution kernel weights, fully connected layer weights, and fully connected layer biases; When the weighted summation result reaches a preset loss value, the target detection model is obtained.

7. The obstacle avoidance control method for door opening according to claim 5, characterized in that: The method for obtaining the actual height of the obstacle includes: Obtaining the focal length of the camera, the camera intrinsic parameter matrix, the camera extrinsic parameters, the obstacle position coordinates, and the ground height of the ground in the world coordinate system, wherein the camera extrinsic parameters include a rotation matrix and a translation vector; According to the camera focal length and the camera intrinsic parameter matrix, the obstacle position coordinates are converted into first three-dimensional coordinates in the camera coordinate system; Calculating the first three-dimensional coordinates according to the rotation matrix, the translation vector and the camera intrinsic parameter matrix to obtain the second three-dimensional coordinates of the obstacle in the world coordinate system; The difference between the height coordinate in the second three-dimensional coordinate and the ground height is calculated to obtain the actual height of the obstacle.

8. The obstacle avoidance control method for door opening according to claim 4, characterized in that: The performing collision risk assessment between the door and the target obstacle according to the obstacle distance, the first height and the preset door opening data includes: If the obstacle distance is less than or equal to the first opening distance, and the own height is greater than or equal to the second height, there is a collision risk between the vehicle door and the target obstacle; If the obstacle distance is less than or equal to the first opening distance, and the height from the ground is less than or equal to the third height, there is a collision risk between the vehicle door and the target obstacle; If the obstacle distance is less than or equal to the first opening distance, and the own height is less than the second height, there is no collision risk between the vehicle door and the target obstacle; If the obstacle distance is less than or equal to the first opening distance, and the height from the ground is greater than the third height, there is no collision risk between the vehicle door and the target obstacle; If the obstacle distance is greater than the first opening distance, there is no collision risk between the vehicle door and the target obstacle.

9. An obstacle avoidance control system for opening a vehicle door, characterized in that: The system comprises: A data monitoring module, used to obtain obstacle information outside the vehicle door, wherein the obstacle information includes an obstacle distance between the target obstacle and the vehicle door and a first height corresponding to the target obstacle; a collision risk assessment module, configured to perform a collision risk assessment between the vehicle door and the target obstacle according to the obstacle distance, the first height and preset vehicle door opening data, wherein the vehicle door opening data includes a first opening distance corresponding to a preset opening angle of the vehicle door, a second height of a lower edge of the vehicle door from the ground and a third height of an upper edge of the vehicle door from the ground; The obstacle avoidance control module is used to adjust the door opening angle according to the obstacle distance and / or the vehicle suspension height according to the first height if there is a risk of collision between the door and the target obstacle, so as to avoid the target obstacle during the door opening process.

10. A vehicle-mounted terminal, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the vehicle terminal to implement the obstacle avoidance control method for opening the vehicle door as described in any one of claims 1 to 8.

Images