Obstacle recognition and positioning method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an obstacle recognition and positioning method and device, electronic equipment and a readable storage medium. The method comprises the steps that an environment image shot by a vehicle is taken; extracting feature data from the environment image by using a pre-trained obstacle detection model; obstacle grounding point detection is performed on the feature data by using a grounding point detection network included in the obstacle detection model to obtain first category information and grounding point information of an obstacle in the environment image, and the grounding point information represents a contact position of the obstacle and the ground; performing obstacle detection on the feature data by using an obstacle detection network included in the obstacle detection model to obtain second category information and position information of the obstacle in the environment image; and performing drivable area detection on the feature data by using an obstacle segmentation network included in the obstacle detection model to obtain drivable area information. According to the scheme, the safety of automatic parking can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to an obstacle identification and positioning method and device, an electronic device, and a readable storage medium. Background Art

[0002] In recent years, the pace of automobile intelligence has been accelerating, and automatic parking technology has also been widely used and developed. Automatic parking technology can help people complete parking more quickly, safely and reliably. The on-board camera detection it uses can identify parking spaces, while the algorithm can accurately avoid obstacles and plan a suitable driving path, and then update the control instructions to complete parking. It can be seen that accurate obstacle avoidance is a key part of automatic parking technology, that is, it is particularly important to accurately identify and locate various dynamic and static obstacles in the surrounding environment during the process of parking in and out of the vehicle, so as to provide a more reliable spatial area for downstream planning.

[0003] Existing technical solutions are constrained between accuracy and efficiency. Some solutions take a long time to achieve the best efficiency, but perform poorly in terms of accuracy. Some solutions can achieve the best accuracy, but their efficiency needs to be improved. It is difficult to meet the real-time optimization of the vehicle side and delayed response may occur.

[0004] Therefore, there is an urgent need to provide a new solution to overcome the above shortcomings, thereby improving the safety and reliability of the automatic parking process. Summary of the invention

[0005] The main technical problem solved by the present application is to provide an obstacle identification and positioning method and device, electronic equipment, and readable storage medium to improve the safety and reliability of the automatic parking process.

[0006] In order to solve the above technical problems, the first aspect of the present application provides an obstacle identification and positioning method, the method comprising: obtaining an environment image taken by a self-vehicle; extracting feature data from the environment image using a pre-trained obstacle detection model; performing obstacle grounding point detection on the feature data using a grounding point detection network included in the obstacle detection model to obtain first category information and grounding point information of the obstacle in the environment image, wherein the grounding point information represents the contact position between the obstacle and the ground; performing obstacle detection on the feature data using the obstacle detection network included in the obstacle detection model to obtain second category information and position information of the obstacle in the environment image; performing drivable area detection on the feature data using the obstacle segmentation network included in the obstacle detection model to obtain drivable area information.

[0007] Optionally, the first category information includes vehicle obstacles and common obstacles, and the vehicle obstacles and the common obstacles use different methods to calculate the grounding point loss.

[0008] Optionally, the vehicle obstacle and the common obstacle use different methods to calculate the ground point loss, including: the vehicle obstacle uses a cross entropy loss function to calculate the ground point loss, and the common obstacle uses a mean square error loss function to calculate the ground point loss.

[0009] Optionally, after obtaining the second category information of the obstacle in the environment image, the method further includes: the second category information includes static obstacles, pedestrians, vehicles and wheels, and the loss of the obstacle is calculated respectively according to the second category information.

[0010] Optionally, after the obstacle detection network performs obstacle detection on the feature data, the method further includes: obtaining an occlusion attribute and a truncation attribute of the obstacle in the environment image.

[0011] Optionally, the obstacle segmentation network performs drivable area detection on the feature data to obtain the drivable area information, including: obtaining the lower edge contour and height information of the obstacle, and dividing the drivable area according to the height information.

[0012] Optionally, dividing the drivable area according to the height information includes: marking the area with the height information as a non-drivable area, and marking the area without the height information as a drivable area.

[0013] In order to solve the above technical problems, the second aspect of the present application provides an obstacle identification and positioning device, which includes: an acquisition module, used to acquire an environment image taken by a self-vehicle; an extraction module, used to extract feature data from the environment image using a pre-trained obstacle detection model; a grounding point detection module, used to perform obstacle grounding point detection on the feature data using a grounding point detection network included in the obstacle detection model, and obtain first category information and grounding point information of the obstacle in the environment image, wherein the grounding point information indicates the contact position between the obstacle and the ground; an obstacle detection module, used to perform obstacle detection on the feature data using the obstacle detection network included in the obstacle detection model, and obtain second category information and position information of the obstacle in the environment image; an obstacle segmentation module, used to perform drivable area detection on the feature data using the obstacle segmentation network included in the obstacle detection model, and obtain drivable area information.

[0014] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the obstacle identification and positioning method as described above.

[0015] In order to solve the above technical problems, the fourth aspect of the present application provides a readable storage medium, on which program instructions that can be executed by a processor are stored, and when the program instructions are executed by the processor, the obstacle identification and positioning method as described above is implemented.

[0016] The beneficial effects of the present application are as follows: the present application proposes a general obstacle recognition and positioning method, which detects dynamic and static obstacles in the automatic parking process with an end-to-end model architecture, models them according to different obstacle characteristics, and saves computing power by sharing features, thereby improving recognition efficiency and accuracy. At the same time, the present application designs a suitable network architecture, uses a feature extraction network to extract multi-scale features from the input image, and outputs them as output features for multi-task training, uses a ground point detection network to output the first category information and ground point information of the obstacle, uses an obstacle detection network to output the second category information and location information of the obstacle, and uses an obstacle segmentation network to output the drivable area information. Based on the above scheme, the present application can significantly improve the safety and reliability of the automatic parking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a structural diagram of an embodiment of an obstacle detection model of the present application;

[0018] Figure 2 It is a flowchart of an embodiment of the obstacle identification and positioning method of the present application;

[0019] Figure 3 It is a structural diagram of an embodiment of a ground point detection network of the present application;

[0020] Figure 4 This is a simplified schematic diagram of an embodiment of the present application for detecting and training the contact point of a vehicle tire;

[0021] Figure 5 This is a simplified schematic diagram of an embodiment of the present application for obstacle detection training;

[0022] Figure 6 It is a schematic diagram of the framework of an embodiment of the obstacle identification and positioning device of the present application;

[0023] Figure 7 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0024] Figure 8 It is a schematic diagram of the framework of an embodiment of a readable storage medium of the present application. DETAILED DESCRIPTION

[0025] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0026] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0027] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0028] See also Figure 1 , Figure 1 It is a structural diagram of an embodiment of an obstacle detection model of the present application.

[0029] like Figure 1 As shown, the present application inputs the environment image into the feature extraction network, which includes a backbone network and a neck network. After the environment image is input into the backbone network, the extracted features are output to the feature pyramid module of the neck network. After multi-scale feature fusion, the output is used as the output feature of multi-task training. The obstacle detection model of the present application has three detection networks that will output features. These three detection networks are the ground point detection network, the obstacle detection network, and the obstacle segmentation network. The ground point detection network outputs the first category information and ground point information of the obstacle, the obstacle detection network outputs the second category information and location information of the obstacle, and the obstacle segmentation network outputs the drivable area information.

[0030] Specifically, this application first needs to manually annotate the training environment images, and its labels include: obstacle frame coordinates, obstacle frame category, obstacle grounding point coordinates, obstacle grounding point confidence, vehicle grounding point frame coordinates, vehicle grounding point category, vehicle grounding point coordinates, vehicle grounding point confidence, and drivable area (walls, pillars, curbs, vehicles) contour annotations. Then, this application uniformly converts the label format of the data set into the format required for the modeling target. Finally, this application constructs Figure 1 The obstacle detection model shown.

[0031] Based on this, the present application proposes a general obstacle recognition and positioning method, which detects dynamic and static obstacles in the automatic parking process with an end-to-end model architecture, models them according to different obstacle characteristics, and saves computing power by sharing features, thereby improving recognition efficiency and accuracy. At the same time, the present application designs a suitable network architecture, uses a feature extraction network to extract multi-scale features from the input image, and outputs them as output features for multi-task training, uses a ground point detection network to detect the first category information and ground point information of obstacles, uses an obstacle detection network to output the second category information and location information of obstacles, and uses an obstacle segmentation network to output drivable area information, thereby improving the safety and reliability of the automatic parking process.

[0032] See also Figure 2 , Figure 2 It is a flowchart of an embodiment of the obstacle identification and positioning method of the present application.

[0033] like Figure 2 As shown, the present application provides a method for obstacle identification and positioning, the method comprising the following steps:

[0034] S11: Acquire the environment image taken by the vehicle.

[0035] In one embodiment of the present application, a camera on a vehicle captures the environment in which the vehicle is located to obtain an environment image.

[0036] It can be understood that the environment image can be a single image taken by a camera, or an image stitched together from multiple images, such as a panoramic image; the camera can be a fisheye camera, a depth camera, etc.

[0037] S12: Extract feature data from the environment image using the pre-trained obstacle detection model.

[0038] In one embodiment of the present application, the obstacle detection model may be a software module of a detection algorithm for executing multiple tasks. The obstacle detection model may be constructed based on a neural network model of various structures related to the art.

[0039] Specifically, the obstacle detection model may include a backbone network and a neck network, which together constitute the feature extraction network of the present application. The backbone network may be a network with a structure such as CSPNet (Cross Stage Partial Network), and the neck network may be a network with a structure such as PAN (Pyramid Attention Network). The backbone network can extract basic feature maps from environmental images, and the neck network can fuse feature maps of different sizes and semantic levels, integrate the global features and local features of environmental images, and obtain feature data of large and small targets at the same time.

[0040] Furthermore, in one embodiment of the present application, the backbone network uses a residual network to extract and output feature maps of different sizes from the input surround fisheye image. The size of each feature map gradually decreases, the depth gradually deepens, and the perception continues to improve. The neck network fuses and outputs feature maps of different levels through a feature pyramid structure, which can improve the generalization of the model and the detection effect of objects of different sizes. That is, the feature map output by the backbone network is further input into the feature pyramid structure, and a feature map rich in semantic information and detail information is obtained through convolution and upsampling. Feature maps of different levels can integrate the shallow and deep features of the fisheye image, and obtain feature information of large and small targets at the same time.

[0041] S13: using a grounding point detection network included in the obstacle detection model to perform obstacle grounding point detection on the feature data, and obtain first category information and grounding point information of the obstacle in the environment image, where the grounding point information indicates the contact position between the obstacle and the ground.

[0042] Due to the distortion characteristics of fisheye images, the center of the traditional bottom obstacle frame cannot well reflect the location of the obstacle. The obstacle detection model includes a ground point detection network, which can run a ground point detection algorithm to predict the location of the ground point of the obstacle. The ground point detection network can determine the boundary and ground area of ​​the obstacle based on the input feature data, and then determine multiple ground points from the boundary of the obstacle and the area in contact with the ground.

[0043] The grounding point information indicates the contact position between the obstacle and the ground. One obstacle may correspond to multiple grounding points, and each grounding point may indicate the contact point between the obstacle and the ground.

[0044] In order to improve the prediction accuracy, the present application will classify and mark the obstacles in the environment image according to the feature data while obtaining the touchdown point information, and then obtain the first category information of the obstacles.

[0045] like Figure 3 As shown, in one embodiment of the present application, the first category of information includes vehicle obstacles and common obstacles, and the vehicle obstacles and the common obstacles use different methods to calculate the grounding point loss.

[0046] For example, the vehicle obstacle uses the cross entropy loss function to calculate the grounding point loss, and the ordinary obstacle uses the mean square error loss function to calculate the grounding point loss.

[0047] Specifically, a separate obstacle box is designed for the grounding point of the vehicle obstacle for classification and regression. Specifically, assuming Figure 4It is a vehicle obstacle-tire grounding point in a fisheye image, that is, for each tire of the vehicle obstacle, the grounding area at the center of the tire is used as the grounding point. This application classifies the tires of the vehicle obstacle into left front, right front, right rear, and left rear. The classification results are used to determine the direction of the vehicle. At the same time, a grounding point frame is designed to place positive samples. The grounding point loss calculation of the vehicle obstacle is as follows:

[0048] Vehicle grounding point classification loss:

[0049]

[0050] Vehicle ground point frame loss:

[0051]

[0052] Among them, N represents the number of positive sample points, α and γ represent weights, and y i represents the category label, i.e. the four categories of tires: left front, right front, right rear, and left rear. i represents the predicted probability value, l * 、r * ,t * , b * They respectively represent the offsets from the predicted tire contact point to the four corner points, and l, r, t, and b respectively represent the true value offsets of the four corner points.

[0053] For common obstacles, the grounding point is the grounding area of ​​the obstacle. For example, for static obstacles such as cones and no-stop signs, the grounding point is the center of the bottom obstacle frame. For example, for pedestrians, the grounding point is the left and right feet of the pedestrian. In terms of training methods, this application will learn the offset between the predicted point and the actual grounding point.

[0054] In terms of the loss function, for the common obstacle ground point offset regression task, the mean square error loss is used to narrow the gap between the coordinate offset and the true ground point coordinates.

[0055] Common obstacle grounding point offset loss:

[0056]

[0057] Among them, N represents the number of positive sample points, dx * ,dy * They represent the predicted offsets in the x and y directions, respectively, and dx and dy represent the true offsets in the x and y directions, respectively.

[0058] Based on this, this application designs a grounding point detection network to accurately locate the position of obstacles. At the same time, according to the grounding point characteristics of vehicle obstacles and ordinary obstacles, separate classification and regression methods are designed respectively, so that the type and position of the grounding point can be predicted more accurately, and the position and direction of the vehicle obstacle can be determined more accurately.

[0059] S14: using the obstacle detection network included in the obstacle detection model to perform obstacle detection on the feature data, and obtain second category information and position information of the obstacles in the environment image.

[0060] The obstacle detection model includes an obstacle detection network, which can run an obstacle detection algorithm to predict the location and category of obstacles. Generally, obstacle information includes an obstacle detection frame indicating the location of the obstacle and second category information indicating the category of the obstacle. The obstacle detection frame is a frame diagram (usually a rectangular frame) containing an obstacle in the environment image. The second category information indicates the category of the obstacle in the obstacle detection frame.

[0061] It should be understood that the obstacle information obtained in this step may represent the positions of multiple obstacles and the second category information, that is, multiple obstacle detection frames and the second category information corresponding to each obstacle detection frame may be obtained.

[0062] In a specific embodiment of the present application, the second category information includes static obstacles, pedestrians, vehicles and wheels, and the loss of obstacles is calculated respectively according to the second category information, that is, the obstacles are first classified, and then the loss of each type of obstacle is calculated respectively.

[0063] For example, static obstacles can be objects such as switch ground locks, ice cream cones, no parking signs, and trash cans; pedestrians can be people of various shapes; vehicles can be two-wheeled vehicles, three-wheeled vehicles, four-wheeled vehicles, trucks, etc.; wheels can be four types including left front wheel, right front wheel, right rear wheel, and left rear wheel. In terms of training methods, regression learning target positive sample point to the four corner points offset, while for each major category respectively put positive samples to ensure that each category of objects can be trained in the case of overlap.

[0064] See also Figure 5 , Figure 5 It is a simplified schematic diagram of an embodiment of the present application for obstacle detection training.

[0065] The loss function includes obstacle category classification loss and obstacle detection box loss. The classification loss and obstacle detection box loss each include the following four category losses:

[0066] Static obstacle category classification loss:

[0067]

[0068] Pedestrian classification loss:

[0069]

[0070] Vehicle obstacle classification loss:

[0071]

[0072] Wheel category classification loss:

[0073]

[0074] Static obstacle detection box loss:

[0075]

[0076] Pedestrian detection box loss:

[0077]

[0078] Vehicle obstacle detection box loss:

[0079]

[0080] Wheel detection box loss:

[0081]

[0082] Among them, N represents the number of positive sample points, α and γ represent weights, and y i represents the category label, p i represents the predicted probability value, l * 、r * ,t * , b * They represent the offsets from the predicted points to the four corner points, and l, r, t, and b represent the true value offsets of the four corner points.

[0083] Furthermore, after the obstacle detection network performs obstacle detection on the feature data, the method further includes: obtaining an occlusion attribute and a truncation attribute of the obstacle in the environment image.

[0084] That is, in order to solve the problem of poor obstacle detection within the occlusion and truncation range, this application designs a loss function to classify the occlusion and truncation attributes of the obstacle, thereby assisting in obstacle distance judgment. The following introduces the design made on the annotation end for occlusion and truncation situations:

[0085] When marking occlusion, you need to imagine the occluded part of the object and mark the occlusion level. The occlusion level is divided into three levels: 0, 1, and 2 according to the visible ratio:

[0086] 1. The visible ratio is 0%-30%, that is, [0.1, 0.3), and the label is given as "0".

[0087] 2. The visible ratio is 30%-60%, that is, [0.3, 0.6), and the label is given as "1".

[0088] 3. The visible ratio is 60%-100%, that is, [0.6, 1.0], and the label is given as "2".

[0089] Among them, the visible ratio = the area of ​​the visible part of the rectangular box / the area of ​​the entire rectangular box completed in the mind.

[0090] When marking truncation, if truncation occurs in the annotation, that is, the annotation object is outside the fisheye lens image, you need to mark the truncation level, but you only need to drag the frame to the boundary. The truncation level is divided into three levels: 0, 1, and 2 according to the remaining ratio:

[0091] 1. The remaining proportion after truncation is 0%-30%, that is, [0.1, 0.3), and the label is given as "0".

[0092] 2. The remaining proportion after truncation is 30%-60%, that is, [0.3, 0.6), and the label is given as "1".

[0093] 3. The remaining ratio after truncation is 60%-100%, that is, [0.6, 1.0], and the label is given as "2".

[0094] The remaining ratio after truncation = the area of ​​the visible rectangular frame in the picture / the area of ​​the complete rectangular frame that is not truncated by the edge of the image.

[0095] After obtaining the occlusion level and truncation level, the occlusion attribute and truncation attribute of the obstacle are output by predicting the classification loss, as follows:

[0096] Obstacle occlusion attribute classification loss:

[0097]

[0098] Obstacle truncation attribute classification loss:

[0099]

[0100] Among them, N represents the number of positive sample points, α and γ represent weights, and y i represents the category label, p i Represents the predicted probability value.

[0101] Based on this, this application addresses the problem that existing algorithms have poor detection capabilities when overlapping objects are occluded. In the obstacle detection network, all obstacles are divided into four major categories and positive samples are used for training respectively to solve the aforementioned problem. At the same time, the occlusion and truncation properties of the obstacles are output, so that the category information and location information of the obstacles can be obtained more accurately.

[0102] S15: Utilizing the obstacle segmentation network included in the obstacle detection model, a drivable area detection is performed on the feature data to obtain drivable area information.

[0103] The obstacle detection model includes an obstacle segmentation network, which can run an obstacle segmentation algorithm to predict the drivable area around the vehicle. The obstacle segmentation network can obtain the lower edge contour and height information of the obstacle based on the input feature data, and divide the drivable area based on the height information. That is, the area with height information is marked as a non-drivable area, and the area without height information is marked as a drivable area.

[0104] In a specific embodiment of the present application, for three-dimensional static obstacles such as walls, pillars, curbs, and vehicle outlines, since traditional obstacle detection frames are more used to detect obstacles of regular sizes, in order to address the serious imbalance in the aspect ratio and unstable regression of the detection frame offset, the present application conducts targeted modeling based on the characteristics of the obstacles in the fisheye image, and uses image segmentation to predict the lower edge contour of each category in the image pixel by pixel. In terms of annotation, the present application draws the lower edge contour of obstacles such as walls and pillars on the fisheye image, and uses the unique ground and height information and semantic edge information of the obstacles to make it easier for the network model to learn the characteristic information of such obstacles, and finally extract the lower edge point sequence of the contour as a distance judgment.

[0105] In terms of training methods, classification loss is used to output the category information of each pixel. The specific calculation formula of classification loss is as follows:

[0106]

[0107] Among them, N represents the number of samples of feature points, α and γ represent weights, and y i represents the category label, p i Represents the predicted probability value.

[0108] In addition, this application will also use the intersection over union loss (IoU Loss) and the hard example mining loss (Hard Example Mining Loss) to restore more accurate segmentation results. The intersection over union loss calculates the loss based on the overlapping area between the predicted box and the true box. In this application, the difference between the predicted value and the true value is sorted to obtain the pixel order and the true value order of the difference, thereby calculating the similarity of the intersection and the union, and using this as the loss value gradient to pass back and update the parameters. The hard example mining loss is a method of dynamically selecting samples with high losses (i.e., hard examples) for focused training during the training process. Specifically in this application, the hard example mining loss obtains the index and score of the highest and second highest probability categories on each pixel, and checks whether the two scores are close. For close pixels, a higher weight is used to calculate the cross entropy loss.

[0109] Based on this, the present application can design an obstacle segmentation network according to different obstacle attributes, design a label true value containing specific semantic information, and output the drivable area information using semantic segmentation for walls, pillars, curbs, etc., thereby significantly improving the accuracy of obstacle identification and positioning. Among them, the label true value containing specific semantic information can be a label made according to the height information of the obstacle.

[0110] In summary, the present application provides a general obstacle recognition and positioning method, which detects dynamic and static obstacles in the automatic parking process with an end-to-end model architecture, models them according to different obstacle characteristics, and saves computing power by sharing features. At the same time, a suitable network architecture is designed, and a feature extraction network is used to extract multi-scale features from the input image, and output them as output features for multi-task training. A ground point detection network is used to model the ground points of vehicle obstacles and ordinary obstacle ground points, and output the first category information and ground point information of the obstacles; an obstacle detection network is used to divide obstacles into four major categories to solve the problem of missed detection caused by overlapping object occlusion, and finally output the second category information and location information of the obstacles, and output the occlusion attributes and truncation attributes of the obstacles at the same time, so as to assist in distance judgment; an obstacle segmentation network is used to output the drivable area information. Based on the end-to-end model architecture described above, the present application achieves an optimal balance between accuracy and time consumption, greatly improving the safety and reliability of the automatic parking process.

[0111] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of the obstacle identification and positioning device of the present application.

[0112] like Figure 6 As shown, the present application also provides an obstacle identification and positioning device, the device comprising:

[0113] The acquisition module 21 is used to acquire the environment image taken by the vehicle.

[0114] The extraction module 22 is used to extract feature data from the environment image using a pre-trained obstacle detection model.

[0115] The grounding point detection module 23 is used to use the grounding point detection network included in the obstacle detection model to perform obstacle grounding point detection on the feature data to obtain the first category information and grounding point information of the obstacle in the environment image, and the grounding point information indicates the contact position between the obstacle and the ground.

[0116] Optionally, the first category of information obtained by the touchdown point detection module 23 specifically includes vehicle obstacles and common obstacles, and the touchdown point losses of the vehicle obstacles and the common obstacles are calculated in different ways.

[0117] Optionally, in the touchdown point detection module 23, a cross entropy loss function is used to calculate the touchdown point loss for a vehicle obstacle, and a mean square error loss function is used to calculate the touchdown point loss for a common obstacle.

[0118] The obstacle detection module 24 is used to perform obstacle detection on the feature data using the obstacle detection network included in the obstacle detection model to obtain second category information and position information of the obstacles in the environment image.

[0119] Optionally, the second category information acquired by the obstacle detection module 24 specifically includes static obstacles, pedestrians, vehicles and wheels, and the loss of the obstacles is calculated respectively according to the second category information.

[0120] Optionally, after the obstacle detection module 24 performs obstacle detection on the feature data, the step further includes: obtaining an occlusion attribute and a truncation attribute of the obstacle in the environment image.

[0121] The obstacle segmentation module 25 is used to perform drivable area detection on the feature data using the obstacle segmentation network included in the obstacle detection model to obtain drivable area information.

[0122] Optionally, the obstacle segmentation module 25 obtains the lower edge contour and height information of the obstacle, and divides the drivable area according to the height information.

[0123] Optionally, the obstacle segmentation module 25 marks the area with height information as a non-drivable area, and marks the area without height information as a drivable area.

[0124] See also Figure 7 , Figure 7 It is a schematic diagram of the framework of an embodiment of an electronic device 30 of the present application.

[0125] like Figure 7As shown, the present application also provides an electronic device 30, which includes a memory 31 and a processor 32 coupled to each other, and the processor 32 is used to execute program instructions stored in the memory to implement the obstacle identification and positioning method as described above.

[0126] See also Figure 8 , Figure 8 It is a schematic diagram of the framework of an embodiment of a readable storage medium 40 of the present application.

[0127] like Figure 8 As shown, the present application further provides a readable storage medium 40 on which program instructions 41 that can be executed by a processor are stored. When the program instructions 41 are executed by the processor, the obstacle identification and positioning method as described above is implemented.

[0128] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0129] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0130] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An obstacle identification and positioning method, characterized in that: The method comprises: Obtain the environment image taken by the vehicle; Extracting feature data from the environment image using a pre-trained obstacle detection model; Using a grounding point detection network included in the obstacle detection model, performing obstacle grounding point detection on the feature data to obtain first category information and grounding point information of the obstacle in the environment image, wherein the grounding point information indicates a contact position between the obstacle and the ground; Using the obstacle detection network included in the obstacle detection model, perform obstacle detection on the feature data to obtain second category information and position information of the obstacle in the environment image; The obstacle segmentation network included in the obstacle detection model is used to perform drivable area detection on the feature data to obtain drivable area information.

2. The obstacle identification and positioning method according to claim 1, characterized in that: The first category of information includes vehicle obstacles and common obstacles, and the vehicle obstacles and the common obstacles use different methods to calculate the grounding point loss.

3. The obstacle identification and positioning method according to claim 2, characterized in that: The vehicle obstacle and the common obstacle use different methods to calculate the grounding point loss, including: The vehicle obstacle uses a cross entropy loss function to calculate the ground contact point loss, and the common obstacle uses a mean square error loss function to calculate the ground contact point loss.

4. The obstacle identification and positioning method according to claim 1, characterized in that: After obtaining the second category information of the obstacle in the environment image, the method further includes: The second category information includes static obstacles, pedestrians, vehicles and wheels, and the losses of the obstacles are calculated respectively according to the second category information.

5. The obstacle identification and positioning method according to claim 4, characterized in that: After the obstacle detection network performs obstacle detection on the feature data, the method further includes: Obtaining the occlusion attribute and the truncation attribute of the obstacle in the environment image.

6. The obstacle identification and positioning method according to claim 1, characterized in that: The obstacle segmentation network performs drivable area detection on the feature data to obtain drivable area information, including: The lower edge contour and height information of the obstacle are obtained, and the drivable area is divided according to the height information.

7. The obstacle identification and positioning method according to claim 6, characterized in that: The dividing the drivable area according to the height information includes: The area with the height information is marked as a non-drivable area, and the area without the height information is marked as a drivable area.

8. An obstacle identification and positioning device, characterized in that: The device comprises: An acquisition module is used to acquire the environment image taken by the vehicle; An extraction module, used to extract feature data from the environment image using a pre-trained obstacle detection model; a grounding point detection module, configured to perform obstacle grounding point detection on the feature data using a grounding point detection network included in the obstacle detection model, and obtain first category information and grounding point information of the obstacle in the environment image, wherein the grounding point information indicates a contact position between the obstacle and the ground; An obstacle detection module, configured to perform obstacle detection on the feature data using an obstacle detection network included in the obstacle detection model, and obtain second category information and position information of the obstacle in the environment image; The obstacle segmentation module is used to use the obstacle segmentation network included in the obstacle detection model to perform drivable area detection on the feature data to obtain drivable area information.

9. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the obstacle identification and positioning method according to any one of claims 1 to 7.

10. A readable storage medium having stored thereon program instructions that can be executed by a processor, characterized in that: When the program instructions are executed by a processor, the obstacle identification and positioning method according to any one of claims 1 to 7 is implemented.

Images