Vehicle door obstacle avoidance method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a vehicle door obstacle avoidance method, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a semantic segmentation image and a depth image corresponding to a to-be-detected image; wherein the semantic segmentation image comprises semantic categories corresponding to image areas, the depth image comprises depth information corresponding to pixels, and obstacles matched with the multiple semantic categories comprise step obstacles and non-step obstacles; determining a detection height corresponding to the step type obstacle based on coordinates matched with a plurality of pixels in the image area, and determining a detection height corresponding to a non-step type obstacle based on coordinates matched with pixels at a specified position in the image area and depth information included in the depth image; based on the detection heights corresponding to the various obstacles and the reference height corresponding to the vehicle door, target door opening parameters corresponding to the vehicle door are determined; wherein the target door opening parameter is used for avoiding obstacles higher than the reference height when the vehicle door is opened. In this way, the vehicle door obstacle avoidance precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle door obstacle avoidance method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of automobile intelligence, automatically opening and closing car doors have become an important branch of vehicle intelligence, but car doors are very likely to touch obstacles and be damaged during the automatic opening and closing process. In the prior art, the collected images are usually sent to the vehicle terminal for the user to confirm autonomously, or the distance between the car door and the obstacle is detected by radar to determine whether the obstacle can be avoided. However, the relevant technologies are difficult to accurately determine the specific height of different types of obstacles, resulting in low accuracy of door obstacle avoidance. In view of this, how to improve the accuracy of door obstacle avoidance has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a vehicle door obstacle avoidance method, an electronic device and a computer-readable storage medium, which can improve the accuracy of vehicle door obstacle avoidance.

[0004] To solve the above technical problems, the first aspect of the present application provides a door obstacle avoidance method, comprising: obtaining a semantic segmentation image and a depth image corresponding to an image to be detected; wherein the semantic segmentation image includes a semantic category corresponding to an image area, the depth image includes depth information corresponding to pixels, and obstacles matching multiple semantic categories include step obstacles and non-step obstacles; based on the coordinates of a plurality of pixel matches in the image area, determining a detection height corresponding to the step obstacles, and based on the coordinates of pixel matches at a specified position in the image area and the depth information included in the depth image, determining a detection height corresponding to the non-step obstacles; based on the detection heights corresponding to each type of obstacle and a reference height corresponding to the door, determining a target door opening parameter corresponding to the door; wherein the target door opening parameter is used to avoid obstacles higher than the reference height when the door is opened.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] The above scheme obtains the image to be detected, determines the semantic segmentation image and depth image corresponding to the image to be detected, determines the semantic categories corresponding to different image regions from the semantic segmentation image, and obtains the depth information corresponding to the pixels from the depth image. Multiple semantic categories correspond to different categories of obstacles, and the multiple categories of obstacles include step obstacles and non-step obstacles. Based on the coordinates of multiple pixel matches in the image area, the height of the step obstacle is analyzed to determine the detection height corresponding to the step obstacle, so that when the height difference of the step obstacle does not change much and the depth information is not obvious, the detection height of the obstacle is determined by multi-point sampling, and the height of the non-step obstacle is analyzed based on the coordinates of the pixel match at the specified position in the image area and the depth information included in the depth image, and the detection height corresponding to the non-step obstacle is determined. When the height difference of the non-step obstacle changes greatly, the height of the non-step obstacle in three-dimensional space is detected in combination with the depth information, and finally the accuracy of the detection height corresponding to obstacles with different height differences is improved. Based on the detection heights corresponding to various obstacles and the reference heights corresponding to the car doors, how to set the car door parameters is analyzed, and the target door opening parameters corresponding to the car doors are determined, so that when the car doors are opened according to the target door opening parameters, obstacles higher than the reference height can be avoided, thereby reducing the probability of collision with obstacles after the car doors are opened and improving the accuracy of the car door obstacle avoidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 It is a flow chart of an implementation method of a vehicle door obstacle avoidance method of the present application; Figure 2 It is a flow chart of another implementation method of the vehicle door obstacle avoidance method of the present application; Figure 3 This is a schematic diagram of an application scenario of an implementation method of obtaining the detection height of a step-like obstacle in the present application; Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application; Figure 5 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0009] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0010] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0011] The vehicle door obstacle avoidance method provided in the present application is used to set target door opening parameters for automatically opening and closing vehicle doors so as to avoid obstacles that may damage the vehicle doors, and its corresponding execution subject is a processing unit capable of performing image processing.

[0012] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a vehicle door obstacle avoidance method of the present application, the method comprising: S101: Obtain a semantic segmentation image and a depth image corresponding to the image to be detected; wherein the semantic segmentation image includes the semantic category corresponding to the image area, the depth image includes the depth information corresponding to the pixel, and obstacles matching multiple semantic categories include step obstacles and non-step obstacles.

[0013] Specifically, an image to be detected is obtained, and a semantic segmentation image and a depth image corresponding to the image to be detected are determined. The semantic categories corresponding to different image areas can be determined from the semantic segmentation image, and the depth information corresponding to the pixels can be obtained from the depth image. Multiple semantic categories correspond to different categories of obstacles, and the multiple categories of obstacles include step obstacles and non-step obstacles.

[0014] In some implementation scenarios, an image to be detected is obtained, and a semantic segmentation model is used to perform semantic segmentation on pixels in the image to be detected to obtain a semantic segmentation image corresponding to the image to be detected, and a depth detection model is used to perform depth analysis on pixels in the image to be detected to obtain a depth image corresponding to the image to be detected. The semantic segmentation model is trained using images with semantic category labels, and the depth detection model is trained using images with depth information labels.

[0015] In some implementation scenarios, an image to be detected is obtained, and a multi-task detection model is used to determine the semantic category and depth information matched by the pixels in the image to be detected, wherein the multi-task detection model first performs semantic segmentation on the pixels in the image to be detected, and then performs depth analysis on pixels belonging to different semantic categories based on the results of the semantic segmentation. The multi-task detection model is trained using images with semantic category labels and depth information labels.

[0016] It should be noted that the semantic categories include various types of obstacles and non-obstacles. Various types of obstacles are divided into step obstacles and non-step obstacles according to the standard heights of the obstacles, wherein the standard height of non-step obstacles is higher than that of step obstacles.

[0017] For ease of explanation, the semantic categories include drivable area, curb, special target and background as an example, where the drivable area corresponds to the area level with the plane of the vehicle, that is, non-obstacles, the curb corresponds to the area perpendicular to the ground, the bottom is level with the drivable area and the top is level with the walking area, that is, step-type obstacles, special targets correspond to targets close to the height of the door, special targets include but are not limited to ground locks and jacks, and background corresponds to other categories of targets, including but not limited to buildings, trees and vehicles. Among them, in other specific implementation scenarios, semantic categories can also be specifically set based on actual scenarios, and this application does not impose specific restrictions on this.

[0018] S102: Determine the detection height corresponding to the step-type obstacle based on the coordinates of the multiple pixel matches in the image area, and determine the detection height corresponding to the non-step-type obstacle based on the coordinates of the pixel matches at a specified position in the image area and the depth information included in the depth image.

[0019] Specifically, based on the coordinates of multiple pixel matches in the image area, the height of the step-type obstacles is analyzed to determine the detection height corresponding to the step-type obstacles. Therefore, when the height difference of the step-type obstacles does not change much and the depth information is not obviously different, the detection height of the obstacle is determined through multi-point sampling.

[0020] In some implementation scenarios, the installation height of the acquisition device that captures the image to be detected and the calibrated coordinates of the acquisition device in the vehicle coordinate system are obtained, the pixel coordinates of multiple pixel matches corresponding to the upper end are obtained from the image area corresponding to the step-type obstacle, the pixel coordinates are converted to the vehicle coordinate system, and the conversion coordinates are obtained. Based on the installation height, the calibrated coordinates and the conversion coordinates, the height of the pixel in the vehicle coordinate system is fitted to determine the pixel matching height, and based on the heights of multiple pixel matches, the detection height corresponding to the step-type obstacle is determined.

[0021] In some implementation scenarios, multiple groups of pixel pairs are obtained from the image area corresponding to the step-type obstacle, wherein the pixel pair includes two pixels located at the upper edge and the lower edge in the vertical position, and based on the image size and the pixel difference corresponding to the pixels in the pixel pair, the height difference of the multiple groups of pixel pair matching is obtained, and based on the multiple groups of height differences, the detection height corresponding to the step-type obstacle is determined.

[0022] It can be understood that based on the coordinates of the pixel matching at the specified position in the image area and the depth information included in the depth image, the height of the non-step obstacles is analyzed to determine the detection height corresponding to the non-step obstacles. Therefore, when the height difference of the non-step obstacles varies greatly, the height of the non-step obstacles in three-dimensional space is detected in combination with the depth information, ultimately improving the accuracy of the detection height corresponding to obstacles with different height differences.

[0023] In some implementation scenarios, the highest point in the image area corresponding to the non-step obstacle is taken as the designated position, and the pixel coordinates matching the pixel at the designated position are determined. Based on the depth information of the designated pixel and the device parameters of the acquisition device, the pixel coordinates of the designated pixel are converted to the vehicle coordinate system corresponding to the vehicle, and the target conversion coordinates corresponding to the designated pixel are obtained. Based on the target conversion coordinates, the detection height corresponding to the non-step obstacle is determined.

[0024] In some implementation scenarios, the highest point and the lowest point in the image area corresponding to the non-step obstacle are used as designated positions, the installation height of the acquisition device that acquires the image to be detected is obtained, the plane where the designated position is located is determined based on the depth information, and the height difference between the highest point and the lowest point is determined based on the plane distance between the plane where the designated position is located and the plane where the acquisition device is located, as well as the installation height of the acquisition device, and the corresponding height difference is used as the detection height corresponding to the non-step obstacle.

[0025] S103: Determine target door opening parameters corresponding to the door based on the detection heights corresponding to various obstacles and the reference heights corresponding to the door; wherein the target door opening parameters are used to avoid obstacles higher than the reference height when the door is opened.

[0026] Specifically, based on the detection heights corresponding to various obstacles and the reference heights corresponding to the car doors, the method of setting the car door parameters is analyzed to determine the target door opening parameters corresponding to the car doors, so that when the car doors are opened according to the target door opening parameters, obstacles higher than the reference height can be avoided, thereby reducing the probability of the car doors colliding with obstacles after they are opened and improving the accuracy of the car door obstacle avoidance.

[0027] In some implementation scenarios, based on the detection heights corresponding to various obstacles and the reference heights corresponding to the vehicle door, the area corresponding to the image to be detected in the vertical projection plane is divided into a collision area and a non-collision area, wherein the detection height of the collision area exceeds the reference height, and the height of the non-collision area does not exceed the reference height; based on the collision area, the non-collision area and the area swept on the projection plane when the vehicle door is opened, the target door opening parameters corresponding to the vehicle door are determined.

[0028] In some implementation scenarios, based on the detection height and depth images corresponding to various types of obstacles, a three-dimensional model corresponding to the scene to be matched with the detected image is constructed, and the opening and closing area swept from the three-dimensional model when the door is from closed to fully opened is obtained. Based on the detection height of the obstacles in the opening and closing area and the reference height corresponding to the door, the target door opening parameters corresponding to the door are determined.

[0029] It can be understood that the reference height corresponding to the door corresponds to the height of the lowest point of the door from the plane where the vehicle is located, and the target door opening parameter at least includes the final door opening angle.

[0030] The above scheme obtains the image to be detected, determines the semantic segmentation image and depth image corresponding to the image to be detected, determines the semantic categories corresponding to different image regions from the semantic segmentation image, and obtains the depth information corresponding to the pixels from the depth image. Multiple semantic categories correspond to different categories of obstacles, and the multiple categories of obstacles include step obstacles and non-step obstacles. Based on the coordinates of multiple pixel matches in the image area, the height of the step obstacle is analyzed to determine the detection height corresponding to the step obstacle, so that when the height difference of the step obstacle does not change much and the depth information is not obvious, the detection height of the obstacle is determined by multi-point sampling, and the height of the non-step obstacle is analyzed based on the coordinates of the pixel match at the specified position in the image area and the depth information included in the depth image, and the detection height corresponding to the non-step obstacle is determined. When the height difference of the non-step obstacle changes greatly, the height of the non-step obstacle in three-dimensional space is detected in combination with the depth information, and finally the accuracy of the detection height corresponding to obstacles with different height differences is improved. Based on the detection heights corresponding to various obstacles and the reference heights corresponding to the car doors, how to set the car door parameters is analyzed, and the target door opening parameters corresponding to the car doors are determined, so that when the car doors are opened according to the target door opening parameters, obstacles higher than the reference height can be avoided, thereby reducing the probability of collision with obstacles after the car doors are opened and improving the accuracy of the car door obstacle avoidance.

[0031] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the door obstacle avoidance method of the present application, the method comprising: S201: Obtain a semantic segmentation image and a depth image corresponding to the image to be detected; wherein the semantic segmentation image includes the semantic category corresponding to the image area, the depth image includes the depth information corresponding to the pixel, and obstacles matching multiple semantic categories include step obstacles and non-step obstacles.

[0032] Specifically, an image to be detected is obtained, and a semantic segmentation image and a depth image corresponding to the image to be detected are determined. The semantic categories corresponding to different image areas can be determined from the semantic segmentation image, and the depth information corresponding to the pixels can be obtained from the depth image. Multiple semantic categories correspond to different categories of obstacles, and the multiple categories of obstacles include step obstacles and non-step obstacles.

[0033] S202: Determine the detection height corresponding to the step-type obstacle based on the coordinates of the multiple pixel matches in the image area, and determine the detection height corresponding to the non-step-type obstacle based on the coordinates of the pixel matches at the specified position in the image area and the depth information included in the depth image.

[0034] Specifically, based on the coordinates of multiple pixel matches in the image area, the height of step-type obstacles is analyzed to determine the detection height corresponding to the step-type obstacles. Based on the coordinates of pixel matches at specified positions in the image area and the depth information included in the depth image, the height of non-step-type obstacles is analyzed to determine the detection height corresponding to the non-step-type obstacles.

[0035] In some implementation scenarios, the detection height corresponding to the step-type obstacle is determined based on the coordinates of multiple pixel matches in the image area, including: obtaining the boundary pixels at the upper and lower ends of the step-type obstacle in the image area, and determining the coordinates of the boundary pixel matches; obtaining the installation height corresponding to the acquisition device that acquires the image to be detected, and determining the detection height corresponding to the step-type obstacle based on the installation height and the coordinates of the boundary pixel matches.

[0036] Specifically, the boundary pixels located at the upper and lower ends of the vertical section of the step-type obstacle in the image area are obtained, the pixel coordinates of the boundary pixels in the image to be detected are determined and converted into the vehicle coordinate system to obtain the coordinates of the boundary pixel matching.

[0037] It should be noted that the acquisition device corresponds to device parameters, which include the intrinsic parameter matrix and distortion coefficient of the acquisition device, as well as the extrinsic parameter matrix and translation matrix of the acquisition device. Based on the intrinsic parameter matrix and distortion coefficient of the acquisition device, the pixel coordinates are converted to the normalized device coordinate system. Based on the extrinsic parameter matrix and translation matrix of the acquisition device, the plane coordinates in the normalized device coordinate system are converted to the vehicle coordinate system.

[0038] Furthermore, the installation height corresponding to the acquisition device that acquires the image to be detected is determined, and the height of the vertical plane of the step-type obstacle is fitted based on the coordinates of the installation height and the boundary pixel matching to obtain the detection height corresponding to the step-type obstacle. When the height difference of the step-type obstacle does not change much and the depth information is not obviously different, the height of the step-type obstacle can be detected in combination with the installation height of the acquisition device, thereby improving the accuracy of the detection height of the step-type obstacle.

[0039] In a specific implementation scenario, the installation height corresponding to the acquisition device that acquires the image to be detected is obtained, and the detection height corresponding to the step-type obstacle is determined based on the installation height and the coordinates of the boundary pixel matching, including: determining the plane where the step-type obstacle is located based on the coordinates of the boundary pixel matching at the lower end, determining the installation height of the acquisition device relative to the plane and the projection coordinates of the acquisition device projected onto the plane; determining the intersection of the line between the acquisition device and the boundary pixel at the upper end and the plane based on the installation height and the coordinates of the boundary pixel matching at the upper end, and obtaining the coordinates of the intersection matching; determining the detection height corresponding to the step-type obstacle based on the installation height, the projection coordinates, the coordinates of the boundary pixel matching at the lower end, and the coordinates of the intersection.

[0040] Specifically, the coordinates of the boundary pixel match at the lower end are obtained, the plane where the step-type obstacle is located is determined based on the corresponding coordinates, the installation height of the acquisition device compared to the plane where the obstacle is located is determined to make the installation height more accurate, and the projection coordinates of the acquisition device projected onto the plane are determined.

[0041] Furthermore, based on the installation height of the acquisition device and the coordinates of the matching boundary pixels located at the upper end, a connecting line between the acquisition device and the boundary pixels located at the upper end is determined, and the intersection of the connecting line and the plane where the obstacle is located and the coordinates of the matching intersection are obtained. Then, based on the installation height, projection coordinates, the coordinates of the matching boundary pixels located at the lower end and the coordinates of the intersection, the height of the step-type obstacle is fitted, the detection height corresponding to the step-type obstacle is determined, and the accuracy of the detection height corresponding to the step-type obstacle is improved.

[0042] For ease of explanation, Figure 3 This is a schematic diagram of an application scenario of an implementation method of obtaining the detection height of step-like obstacles in this application. Indicates the installation height of the acquisition equipment. Indicates the detection height of step obstacles. The projection point of the acquisition device on the ground is , the projection point of the step obstacle on the ground is , the intersection of the line connecting the acquisition device and the vertex of the step obstacle and the ground is In the vehicle coordinate system, assume that the coordinates of each point are: , , , then the detection height can be obtained based on the following formula: (1) Where D represents the distance between coordinates.

[0043] In some implementation scenarios, the detection height corresponding to the non-step obstacle is determined based on the coordinates of the pixel match at the specified position in the image area and the depth information included in the depth image, including: obtaining the specified pixel of the non-step obstacle at the specified position in the image area, and determining the pixel coordinates of the specified pixel in the image to be detected; obtaining the device parameters of the acquisition device that acquires the image to be detected, and based on the depth information and device parameters of the specified pixel, converting the pixel coordinates of the specified pixel to the vehicle coordinate system corresponding to the vehicle, to obtain the target conversion coordinates corresponding to the specified pixel; based on the target conversion coordinates, determining the detection height corresponding to the non-step obstacle.

[0044] Specifically, a specified pixel located at a specified position is obtained from an image area corresponding to the non-step obstacle, the pixel coordinates of the specified pixel in the image to be detected are determined, and the device parameters matched by the acquisition device that acquires the image to be detected are obtained. The pixel coordinates of the specified pixel in the depth image are converted to the vehicle coordinate system corresponding to the vehicle using the depth information and device parameters of the specified pixel. When the height difference of the non-step obstacle varies greatly, the specified pixel at the specified position is converted to the vehicle coordinate system in combination with the depth information to obtain the target conversion coordinates, so that the values ​​of the upper and lower spaces in the target conversion coordinates can feedback the height corresponding to the specified pixel in the three-dimensional space.

[0045] It can be understood that the detection height corresponding to the non-step obstacle can be determined based on the values ​​of the upper and lower spaces in the target conversion coordinates, that is, based on the values ​​of the Z axis.

[0046] It should be noted that the designated position may be the position corresponding to the highest point in the image area, so that the pixels in the corresponding image area are uniformly identified as the height corresponding to the highest point, so as to adapt to background targets such as walls or vehicles. Alternatively, the designated position may be the position corresponding to some identification points in the target contour in the image area, so that the shape of the target in the image area and the target of the corresponding shape are detected at the heights of different identification points, so as to adapt to special targets such as ground locks or jacks.

[0047] In a specific implementation scenario, the device parameters include the intrinsic parameter matrix and distortion coefficient of the acquisition device, as well as the extrinsic parameter matrix and translation matrix of the acquisition device; based on the depth information and device parameters of the specified pixel, the pixel coordinates of the specified pixel are converted to the vehicle coordinate system corresponding to the vehicle, and the target conversion coordinates corresponding to the specified pixel are obtained, including: based on the intrinsic parameter matrix and distortion coefficient of the acquisition device, the pixel coordinates are converted to the normalized device coordinate system to obtain the plane conversion coordinates and the incident angle; based on the plane conversion coordinates, the incident angle and the depth information of the specified pixel, the three-dimensional conversion coordinates of the specified pixel converted to the device coordinate system are determined; based on the three-dimensional conversion coordinates and the extrinsic parameter matrix and translation matrix of the acquisition device, the target conversion coordinates of the specified pixel converted to the vehicle coordinate system are determined.

[0048] Specifically, assume that the pixel coordinates corresponding to the specified pixel are According to the intrinsic parameter matrix and distortion coefficient, the plane transformation coordinates of the point in the normalized device coordinate system can be obtained. and angle of incidence , the depth value of the specified pixel included in the known depth image is D. Then the three-dimensional transformed coordinates of the point in the device coordinate system are for: (2) (3) (4) Among them, sqrt means to find the square root.

[0049] Furthermore, according to the external parameter matrix and translation matrix , the target transformation coordinates in the vehicle coordinate system can be calculated ,in, , based on the target transformation coordinates The value can determine the detection height corresponding to the non-step obstacle, so as to obtain the accurate detection height of the non-step obstacle by combining the depth information and the equipment parameters of the acquisition device. The target conversion coordinates in the vehicle coordinate system can feedback the relative position relationship between the non-step obstacle and the door, so as to facilitate the judgment of whether a collision will occur.

[0050] S203: Based on the detection heights corresponding to various obstacles and the reference heights corresponding to the vehicle door, determine the collision area and the non-collision area in the image to be detected; wherein the detection height of the collision area exceeds the reference height, and the height of the non-collision area does not exceed the reference height.

[0051] Specifically, a reference height corresponding to the vehicle door is obtained, and based on the height difference between the detection height of various obstacles and the reference height corresponding to the vehicle door, the collision area and non-collision area in the image to be detected are determined, thereby obtaining a collision area where the detection height exceeds the reference height, and a non-collision area where the detection height does not exceed the reference height, so as to judge whether a collision will occur when the door is opened.

[0052] In some implementation scenarios, based on the detection heights corresponding to various obstacles and the reference heights corresponding to the vehicle doors, the collision areas and non-collision areas in the image to be detected are determined, including: based on the detection heights corresponding to various obstacles, matching height values ​​are set for pixels in the corresponding image areas to generate a height map corresponding to the image to be detected; based on the matching height values ​​of the pixels in the height map and the reference height, the collision areas and non-collision areas in the image to be detected are determined.

[0053] Specifically, based on the detection heights corresponding to various types of obstacles, matching height values ​​are set for pixels in the corresponding image area to generate a height map that matches the image to be detected, wherein the height map corresponds to the distance between the obstacle and the vehicle in the vehicle coordinate system, thereby generating the heights of different targets in the space corresponding to the image to be detected.

[0054] Furthermore, the pixels in the height map are traversed, the height values ​​matched by the pixels in the height map are compared with the reference height, the area where the detected height exceeds the reference height is taken as the collision area, and the area where the detected height does not exceed the reference height is taken as the non-collision area, so as to improve the accuracy of area division.

[0055] S204: Obtain a reference door opening area corresponding to the vehicle door, and determine a target door opening parameter corresponding to the vehicle door based on an intersection of the reference door opening area and the collision area; wherein the reference door opening area corresponds to a maximum door opening parameter corresponding to the vehicle door.

[0056] Specifically, a reference door opening area corresponding to when the vehicle door is opened according to the maximum door opening parameters is obtained, and the intersection of the reference door opening area and the collision area is obtained. When the intersection is an empty set, the maximum door opening parameters are used as target door opening parameters. When the intersection is a non-empty set, based on the intersection of the reference door opening area and the collision area, the target door opening parameters that will not collide with the collision area when the vehicle door is opened are re-determined. In this way, the target door opening parameters can be determined based only on the intersection area, thereby improving the efficiency of obtaining the target door opening parameters.

[0057] It can be understood that when the reference door opening area is mapped on the projection plane, it corresponds to the fan-shaped area swept by the door when it is opened from closed to the maximum.

[0058] It should be noted that the obstacle in the reference door opening area corresponds to the obstacle to be confirmed; after obtaining the reference door opening area corresponding to the vehicle door, and determining the target door opening parameters corresponding to the vehicle door based on the intersection of the reference door opening area and the collision area, it also includes: in response to the absolute difference between the detection height of the obstacle to be confirmed and the reference height being less than the difference threshold, generating parameter display information matching the semantic category and detection height corresponding to the obstacle to be confirmed, and generating execution prompt information corresponding to the target door opening parameters.

[0059] Specifically, obstacles located within the reference door opening area need to be paid special attention to. The obstacles within the reference door opening area are marked as obstacles to be confirmed, and the absolute difference between the detection height of the obstacles to be confirmed and the reference height is determined. When the absolute difference is less than the difference threshold, the semantic category and detection height corresponding to the obstacles to be confirmed are used to generate parameter display information, so as to display the category corresponding to the obstacles to be confirmed and the detection height of the obstacles to be confirmed through the parameter display information, wherein the parameter display information is usually displayed on the vehicle screen.

[0060] Furthermore, execution prompt information corresponding to the target door opening parameters is generated, so that the execution prompt information prompts whether to execute the current target door opening parameters, thereby reducing the probability of damage to the vehicle door due to forced door opening due to inaccurate detection height.

[0061] In this embodiment, boundary pixels located at the upper and lower ends of the vertical section of the step-type obstacle in the image area are obtained, the pixel coordinates of the boundary pixels in the image to be detected are determined and converted into the vehicle coordinate system, the coordinates of the boundary pixels are obtained, and the installation height corresponding to the acquisition device for acquiring the image to be detected is determined. Based on the installation height and the coordinates of the boundary pixel matching, the height of the vertical plane of the step-type obstacle is fitted to obtain the detection height corresponding to the step-type obstacle, so that when the height difference of the step-type obstacle does not change much and the depth information is not obviously different, the height of the step-type obstacle can be detected in combination with the installation height of the acquisition device, thereby improving the accuracy of the detection height of the step-type obstacle. The designated pixel at the designated position is obtained from the image area corresponding to the non-step obstacle, the pixel coordinates of the designated pixel in the image to be detected are determined, the device parameters matched by the acquisition device for acquiring the image to be detected are obtained, and the pixel coordinates of the designated pixel in the depth image and the device parameters are used to convert the pixel coordinates of the designated pixel into the vehicle coordinate system corresponding to the vehicle, so that when the height difference of the non-step obstacle changes greatly, the designated pixel at the designated position is converted into the vehicle coordinate system in combination with the depth information to obtain the target conversion coordinates, so that the values ​​of the upper and lower spaces in the target conversion coordinates can feedback the height corresponding to the designated pixel in the three-dimensional space. Based on the detection heights corresponding to various obstacles and the reference heights corresponding to the car door, the collision area and the non-collision area in the image to be detected are determined, the reference door opening area corresponding to the door when it is opened according to the maximum door opening parameter is obtained, and the intersection of the reference door opening area and the collision area is obtained. The target door opening parameters can be determined based on the intersection area alone, thereby improving the efficiency of obtaining the target door opening parameters.

[0062] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 30 includes a memory 301 and a processor 302 coupled to each other, wherein the memory 301 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0063] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0064] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0065] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0066] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0067] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A vehicle door obstacle avoidance method, characterized in that: The method comprises: Obtain a semantic segmentation image and a depth image corresponding to the image to be detected; wherein the semantic segmentation image includes the semantic category corresponding to the image area, the depth image includes the depth information corresponding to the pixel, and obstacles matching multiple semantic categories include step obstacles and non-step obstacles; Determine the detection height corresponding to the step obstacle based on the coordinates of the multiple pixel matches in the image area, and determine the detection height corresponding to the non-step obstacle based on the coordinates of the pixel matches at a specified position in the image area and the depth information included in the depth image; Based on the detection heights corresponding to various obstacles and the reference heights corresponding to the vehicle doors, target door opening parameters corresponding to the vehicle doors are determined; wherein the target door opening parameters are used to avoid obstacles higher than the reference height when the vehicle doors are opened.

2. The vehicle door obstacle avoidance method according to claim 1, characterized in that: The determining the detection height corresponding to the step obstacle based on the coordinates of the plurality of pixel matches in the image area includes: Acquire boundary pixels of the step-like obstacle at the upper and lower ends in the image region, and determine the coordinates of the boundary pixel matches; The installation height corresponding to the acquisition device that acquires the image to be detected is obtained, and the detection height corresponding to the step-type obstacle is determined based on the installation height and the coordinates of the boundary pixel matching.

3. The vehicle door obstacle avoidance method according to claim 2, characterized in that: The obtaining of the installation height corresponding to the acquisition device for acquiring the image to be detected, and determining the detection height corresponding to the step obstacle based on the installation height and the coordinates matched by the boundary pixels, includes: Based on the coordinates of the boundary pixel matching at the lower end, determine the plane where the step-like obstacle is located, determine the installation height of the acquisition device relative to the plane, and determine the projection coordinates of the acquisition device projected onto the plane; Based on the coordinates of the installation height and the boundary pixel at the upper end, determine the intersection of the line between the acquisition device and the boundary pixel at the upper end and the plane, and obtain the coordinates of the intersection; The detection height corresponding to the step-type obstacle is determined based on the installation height, the projection coordinates, the coordinates of the boundary pixel matching at the lower end, and the coordinates of the intersection.

4. The vehicle door obstacle avoidance method according to claim 1, characterized in that: The determining the detection height corresponding to the non-step obstacle based on the coordinates of the pixel matching at the designated position in the image area and the depth information included in the depth image includes: Acquire a designated pixel of the non-step obstacle in the image region at a designated position, and determine the pixel coordinates of the designated pixel in the image to be detected; Acquire device parameters of a collection device that collects the image to be detected, and based on the depth information and device parameters of the designated pixel, convert the pixel coordinates of the designated pixel into a vehicle coordinate system corresponding to the vehicle to obtain a target conversion coordinate corresponding to the designated pixel; Based on the target conversion coordinates, a detection height corresponding to the non-step obstacle is determined.

5. The vehicle door obstacle avoidance method according to claim 4, characterized in that: The device parameters include the intrinsic parameter matrix and distortion coefficient of the acquisition device, and the extrinsic parameter matrix and translation matrix of the acquisition device; The step of converting the pixel coordinates of the designated pixel into a vehicle coordinate system corresponding to the vehicle based on the depth information and device parameters of the designated pixel to obtain a target conversion coordinate corresponding to the designated pixel includes: Based on the intrinsic parameter matrix and distortion coefficient of the acquisition device, the pixel coordinates are converted into a normalized device coordinate system to obtain a plane conversion coordinate and an incident angle; Determine a three-dimensional transformation coordinate of the designated pixel transformed into a device coordinate system based on the plane transformation coordinate, the incident angle and the depth information of the designated pixel; Based on the three-dimensional transformation coordinates and the external parameter matrix and the translation matrix of the acquisition device, the target transformation coordinates of the designated pixel in the vehicle coordinate system are determined.

6. The vehicle door obstacle avoidance method according to claim 1, characterized in that: The determining of the target door opening parameters corresponding to the door based on the detection heights corresponding to the various obstacles and the reference heights corresponding to the door includes: Based on the detection heights corresponding to various obstacles and the reference height corresponding to the vehicle door, determining the collision area and the non-collision area in the image to be detected; wherein the detection height of the collision area exceeds the reference height, and the height of the non-collision area does not exceed the reference height; A reference door opening area corresponding to the vehicle door is obtained, and a target door opening parameter corresponding to the vehicle door is determined based on an intersection of the reference door opening area and the collision area; wherein the reference door opening area corresponds to a maximum door opening parameter corresponding to the vehicle door.

7. The vehicle door obstacle avoidance method according to claim 6, characterized in that: The determining of the collision area and the non-collision area in the image to be detected based on the detection heights corresponding to the various obstacles and the reference height corresponding to the vehicle door comprises: Based on the detection heights corresponding to various types of obstacles, matching height values ​​are set for pixels in the corresponding image area to generate a height map corresponding to the image to be detected; Based on the height values ​​of the pixel matches in the height map and the reference height, a collision area and a non-collision area in the image to be detected are determined.

8. The vehicle door obstacle avoidance method according to claim 6, characterized in that: Obstacles to be confirmed corresponding to the obstacles in the reference door opening area; After obtaining the reference door opening area corresponding to the vehicle door and determining the target door opening parameter corresponding to the vehicle door based on the intersection of the reference door opening area and the collision area, the method further includes: In response to the absolute difference between the detected height of the obstacle to be confirmed and the reference height being less than a difference threshold, parameter display information matching the semantic category and detected height corresponding to the obstacle to be confirmed is generated, and execution prompt information corresponding to the target door opening parameter is generated.

9. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images