Obstacle identification method and device

Abstract

The invention discloses an obstacle identification method and device, and relates to the technical field of big data. A specific embodiment of the method comprises the steps of determining a background region in a road image; obtaining road boundary information, and projecting the road boundary information to the road image to obtain a target road boundary area; separating an obstacle from the background area based on the target road boundary area; according to the embodiment, the detection rate and accuracy of visual low obstacles can be improved, and the low obstacles crossing the road boundary can be detected.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to an obstacle recognition method and device. Background Art

[0002] Existing obstacle recognition is mainly based on occupancy networks, image semantic segmentation algorithms or image segmentation anomaly detection algorithms. The occupancy network determines whether each position in the three-dimensional space is occupied by an obstacle through three-dimensional reconstruction; the image semantic segmentation algorithm labels all obstacles on the road with a general category for identification; the image segmentation anomaly detection algorithm first uses the image segmentation algorithm to segment the road, and then trains the anomaly detection algorithm on the road area.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the related art:

[0004] The existing technology has a low detection rate for low visual obstacles and a high probability of false detection, and is unable to detect low obstacles that cross the road boundary. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides an obstacle recognition method and device, which can improve the detection rate and accuracy of visual low obstacles and can detect low obstacles across the road boundary.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for identifying an obstacle is provided, comprising:

[0007] determining a background region in a road image;

[0008] Obtain road boundary information, project the road boundary information onto the road image, and obtain the target road boundary area;

[0009] Obstacles are separated from the background area based on the target road boundary area.

[0010] Furthermore, determining the background area in the road image includes:

[0011] Use the pre-trained segmentation model to predict the background area in the road image;

[0012] The pre-trained segmentation model is obtained by the following steps:

[0013] Collect image segmentation data;

[0014] Determine a classification label based on the image segmentation data; wherein the classification label includes a background class and a non-background class;

[0015] Mapping pixels in the image segmentation data to corresponding classification category labels to generate image segmentation annotation data;

[0016] A training sample set is constructed based on the image segmentation annotation data, and a segmentation model is trained using the training sample set.

[0017] Furthermore, the road boundary information is obtained, including:

[0018] Get vector map information;

[0019] The vector map information is preprocessed, and road boundary information is extracted from the preprocessed vector map information.

[0020] Furthermore, the road boundary information is projected onto the road image to obtain the target road boundary area, including:

[0021] The road boundary information is coordinate-converted using a pre-configured coordinate conversion matrix to obtain a coordinate conversion result;

[0022] Based on the coordinate transformation results, the target road boundary area is drawn on the road image.

[0023] Furthermore, obstacles are separated from the background area based on the target road boundary area, including:

[0024] Configuring attribute features of obstacles; wherein the attribute features are background areas sandwiched between road boundary areas or located within road boundary areas;

[0025] Based on the attribute features, the background area is clipped and the obstacles are determined according to the clipping results.

[0026] Furthermore, after separating the obstacles from the background area, the method further includes:

[0027] Based on the inverse perspective mapping algorithm, the obstacle is mapped into the radar coordinate system to obtain the mapping result;

[0028] Perform point clustering on the mapping results to generate obstacle polygons;

[0029] Send the obstacle polygon as output to the target module.

[0030] Furthermore, the obstacle is mapped into the radar coordinate system based on the inverse perspective mapping algorithm, including:

[0031] The inverse perspective mapping algorithm is used to convert the pixel coordinates in the obstacle into three-dimensional point coordinates, and the three-dimensional point coordinates are converted into coordinates in the radar coordinate system.

[0032] According to a second aspect of an embodiment of the present invention, there is provided an obstacle recognition device, comprising:

[0033] A determination module, used for determining a background area in a road image;

[0034] A projection module is used to obtain road boundary information, project the road boundary information onto the road image, and obtain a target road boundary area;

[0035] The recognition module is used to separate obstacles from the background area based on the target road boundary area.

[0036] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0037] one or more processors;

[0038] a memory for storing one or more programs,

[0039] When one or more programs are executed by one or more processors, the one or more processors are enabled to implement the method of any of the above embodiments.

[0040] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method of any of the above embodiments is implemented.

[0041] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, including a computer program, which implements the method of any of the above embodiments when the computer program is executed by a processor.

[0042] An embodiment of the above invention has the following advantages or beneficial effects: by determining the background area in the road image; obtaining road boundary information, projecting the road boundary information onto the road image, and obtaining a target road boundary area; separating obstacles from the background area based on the target road boundary area; this embodiment can improve the detection rate and accuracy of visual low obstacles, and can detect low obstacles across the road boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0044] Figure 1 is a schematic diagram of the main process of the obstacle identification method according to an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the main process of an obstacle identification method according to a reference embodiment of the present invention;

[0046] Figure 3 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention;

[0047] Figure 4 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention;

[0050] Figure 7 is a schematic diagram of main modules of an obstacle recognition device according to an embodiment of the present invention;

[0051] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0052] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0054] It should be noted that the acquisition, storage and application of personal information etc. involved in the embodiments of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0055] Research has found that the vision-based low obstacle detection algorithms in autonomous driving scenarios mainly include the following: Occupancy Network: Use three-dimensional reconstruction methods to determine whether each position in the three-dimensional space is occupied by a low obstacle; Image semantic segmentation algorithm: Label all low obstacles on the road with a general category for identification; Image segmentation anomaly detection algorithm: First, use an image segmentation algorithm to segment the road, and then train an anomaly detection algorithm for the road area.

[0056] However, due to the sparsity of supervisory signals, the occupancy network can only reconstruct common obstacles with larger volumes, but has a low detection rate for smaller low obstacles and long-tail obstacles. The semantic segmentation algorithm of the image has problems such as difficulty in data collection due to the long tail of the data, uncertainty in categories, and inconsistent features of low obstacles, resulting in a low model detection rate and a high false positive rate. The anomaly detection algorithm of image segmentation performs anomaly detection based on the road area, and can only detect obstacles within the road area, but cannot detect low obstacles across the road boundary. In addition, when training anomaly detection, a large number of samples with abnormal data are also required. Low obstacles have a long tail of data, and also face the problems of difficulty in data collection and poor generalization ability in the real world.

[0057] In view of this, according to one aspect of an embodiment of the present invention, a method for identifying an obstacle is provided. Figure 1 FIG. 1 is a schematic diagram of the main process of the obstacle identification method according to an embodiment of the present invention. Figure 1 As shown, the obstacle recognition method according to an embodiment of the present invention includes:

[0058] Step S101, determining a background area in a road image.

[0059] A road image is an image obtained or photographed containing a road scene, for example, an image of the road conditions on which vehicles or pedestrians are traveling obtained by means of a camera, a sensor of an unmanned vehicle, or aerial photography.

[0060] The background area refers to the part of the road image other than the road, common dynamic objects on the road (such as cars, trucks, buses, bicycles, electric vehicles, motorcycles, pedestrians, etc.), and common static obstacles on the road (such as barriers, cones, isolation poles, manhole covers, etc.). These background areas are unrelated to the surrounding environment or the target objects in the scene.

[0061] When determining the background area in a road image, you can first collect and prepare a road image dataset, and you can obtain road images of various scenes including roads, vehicles, pedestrians, etc. by collecting, downloading or other means; then, select a pre-trained segmentation model suitable for the road image segmentation task and load the pre-trained segmentation model; pre-process the above road image, such as resizing, normalization, etc., so that the road image can adapt to the input requirements of the model; input the pre-processed road image into the loaded segmentation model to predict the background area, and the model outputs the probability or score of each pixel belonging to the background category; post-process the segmentation model prediction results, including threshold processing, connected region analysis, etc., to obtain the final background area segmentation result; output the predicted background area segmentation result, for example, it can be in the form of a binary segmentation image, a segmentation mask, a bounding box, etc. The above steps are used to predict the background area in the road image, providing a data basis for subsequent road scene analysis, obstacle detection and other tasks.

[0062] Determining the background area in the road image can also be based on traditional image feature extraction technology, by using information such as color histogram, texture features and shape features to distinguish between background areas and non-background areas. For example, the color histogram of the road image can be calculated to capture the color features of the background, and then these features can be matched with the pixels in the image to determine which areas belong to the background.

[0063] Step S102, obtaining road boundary information, and projecting the road boundary information onto the road image to obtain a target road boundary area.

[0064] Road boundary information is the boundary or contour information of the road in geographic space, including the road's geometric shape, edge lines, road width, lane lines, intersections and other features. Road boundary information can be obtained through map data extraction, remote sensing image analysis, vehicle-mounted sensor measurement and other methods.

[0065] The above-mentioned road boundary information is projected onto the road image, that is, the road boundary information in the geographic space is mapped or projected into the pixel coordinate system of the road image. Specifically, it can be achieved by means of coordinate transformation matrix and geometric calculation, etc., and the road boundary information in the geographic coordinate system is converted into an area in the pixel coordinate system matching the road image, thereby obtaining the target road boundary area.

[0066] The target road boundary area refers to the area or line segment representing the road boundary obtained after projection in the road image; the target road boundary area can be the edge line, lane line, driving lane, curb, etc. of the road, which can be represented by different colors or marking methods in the image.

[0067] Step S103 : Separating obstacles from the background area based on the target road boundary area.

[0068] Specifically, obstacle detection is performed using the information of the target road boundary area and background area in combination with target detection or image segmentation technology; feature extraction is performed on the detected obstacle area, including color features, texture features, shape features, etc.; based on the extracted features or model prediction results, obstacles in the background area are segmented, for example, the segmentation model can be used to separate obstacles from the background to obtain an accurate obstacle area; the segmented obstacle area is post-processed and optimized, for example, noise removal, hole filling, boundary smoothing, etc., to improve the accuracy and completeness of the segmentation results; the separated obstacle area is verified and analyzed, and the results are evaluated through visualization, statistical analysis, etc., so that the segmentation results are consistent with the actual scene; the separated obstacle area is applied to specific scenes, such as for obstacle detection, obstacle avoidance planning, scene understanding and other applications.

[0069] In addition, when separating obstacles from the background area, the image can be segmented by thresholding, the pixel values ​​of the obstacles and the background can be distinguished, and appropriate thresholds can be set according to the grayscale value, color channel value or other features of the pixels to separate the obstacle area from the background area; morphological operations such as dilation, corrosion, opening operation, closing operation, etc. can also be used to process the image to separate the obstacle area, and small noise points and disconnected areas can be removed; a region growing algorithm can also be used. The region growing algorithm is based on the similarity between pixels, and similar pixels are merged into a region. Specifically, starting from the known obstacle seed points, the obstacle area is gradually grown; image segmentation technology, such as semantic segmentation based on deep learning, instance segmentation and other methods can also be used to directly separate obstacles from the background. The above methods can be used alone or in combination. According to the specific image characteristics and task requirements, a suitable segmentation method is selected to accurately separate obstacles from the background area. The embodiments of the present invention are not specifically limited.

[0070] The embodiment of the present invention determines the background area in the road image; obtains road boundary information, projects the road boundary information onto the road image, and obtains a target road boundary area; separates obstacles from the background area based on the target road boundary area; this embodiment can improve the detection rate and accuracy of visual low obstacles, and can detect low obstacles that cross the road boundary.

[0071] Optionally, determining the background area in the road image includes: predicting the background area in the road image using a pre-trained segmentation model.

[0072] Among them, the segmentation model is a machine learning model used for image segmentation tasks, which can segment the input image into different regions or pixel categories. In the image semantic segmentation task, the segmentation model can assign each pixel in the image to the corresponding semantic category, such as non-background area or background area. The non-background area can be roads, vehicles, pedestrians or barriers, and the background area can be low obstacles or all remaining pixels. Specifically, the segmentation model is a model based on deep learning technology, such as convolutional neural network (CNN), fully convolutional network (FCN), etc.

[0073] Optionally, Figure 2 FIG. 1 is a schematic diagram of the main process of an obstacle identification method according to a reference embodiment of the present invention. Figure 2 As shown in Figure 1, the pre-trained segmentation model is obtained by the following steps:

[0074] Step S201, collecting image segmentation data.

[0075] To collect image segmentation data, we first need to determine the data source, including cameras, open data sets, or images generated through simulation environments. In the scenario of an autonomous driving system, the collected image segmentation data includes various road environments, weather conditions, and traffic scenarios to ensure the generalization ability of the model. The data collection process can be carried out in real time or offline and then annotated and processed.

[0076] Step S202, determining a classification label based on the image segmentation data; wherein the classification label includes a background class and a non-background class.

[0077] When determining the classification labels, a more fine-grained classification can be set. For example, the non-background category can be further subdivided into clean roads, common dynamic objects on the road, common static obstacles on the road, etc.; among them, the common dynamic objects on the road are subdivided into cars, trucks, buses, bicycles, electric vehicles, motorcycles, pedestrians, etc., and the common static obstacles on the road are subdivided into barriers, cones, isolation poles, manhole covers, etc.

[0078] Step S203, mapping pixels in the image segmentation data to corresponding classification category labels to generate image segmentation annotation data.

[0079] Specifically, use image annotation tools or automated annotation algorithms to map pixels in the collected image data to corresponding classification category labels. First, determine the image annotation tool, such as LabelImg (Image Labeling Tool), CVAT (Computer Vision Annotation Tool), LabelMe (Label Me), etc., or write a script for annotation; load the image segmentation data to be annotated into the above-mentioned annotation tool, define the classification category label in the annotation tool, and use the drawing tool or interactive annotation method in the annotation tool to annotate the image at the pixel level, and map each pixel point to the corresponding category label; save the annotated image as an annotation data file, such as an annotation file containing pixel-level label information, such as XML (eXtensibleMarkup Language), JSON (JavaScript Object Notation), etc. format; finally, verify and correct the image segmentation annotation data so that the label and pixel mapping are accurate.

[0080] Step S204: construct a training sample set based on the image segmentation annotation data, and obtain a segmentation model by training the training sample set.

[0081] Use image segmentation and annotation data to pair image data with corresponding pixel-level label information to form a training sample set; preprocess the training sample set, including image resizing and normalization; use the preprocessed training sample set to train using the selected model architecture, and adjust model parameters through optimization algorithms and loss functions (such as cross entropy loss, etc.). During the training process, you can also use the validation set to evaluate and tune the model, monitor the performance indicators of the model (such as accuracy, recall, etc.), and tune the model according to the evaluation results, including adjusting hyperparameters, increasing training data, applying regularization, etc., to improve the performance and stability of the model. Save the trained segmentation model and use it in actual applications to perform segmentation prediction on new images to enable the autonomous driving system to recognize and analyze the road environment.

[0082] The embodiments of the present invention collect and annotate image segmentation data to provide accurate training data for the model, thereby training a model that can effectively segment background areas and non-background areas. In fields such as autonomous driving, the segmentation model can improve the system's understanding and recognition of the road environment, thereby achieving a safer and smarter driving experience.

[0083] Optionally, Figure 3 FIG. 1 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention. Figure 3 As shown, step S102 obtains road boundary information, including:

[0084] Step S301, obtaining vector map information.

[0085] Step S302: pre-process the vector map information, and extract road boundary information from the pre-processed vector map information.

[0086] When obtaining vector map information, first determine the vector map data source, such as a map provider, a government geographic information system, or a professional map data provider; obtain the vector map information through an API (Application Programming Interface) interface or by downloading. The vector map information includes road information, road attributes, and geographic coordinates.

[0087] Clean the vector map information to remove invalid or duplicate data. If the geographic coordinates in the data are not standard longitude and latitude coordinates, perform coordinate conversion. According to the road characteristics or attributes, use map data processing tools or custom scripts to extract road information from the preprocessed vector map data. Road information includes road centerline, road width, road type, etc. Based on the road information, extract road boundary information through algorithms or rules, such as the coordinate points of the left and right boundaries of the road, road contour lines, etc. Finally, store the extracted road boundary information in an appropriate data format, such as Shapefile.

[0088] The embodiment of the present invention improves data quality and accuracy by preprocessing vector map information, and improves the accuracy and efficiency of obstacle detection and anomaly detection by extracting road boundary information from the preprocessed vector map information.

[0089] Figure 4 FIG. 1 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention. Figure 4 As shown, step S102 projects the road boundary information onto the road image to obtain the target road boundary area, including:

[0090] Step S401, using a pre-configured coordinate transformation matrix to transform the road boundary information to obtain a coordinate transformation result;

[0091] Step S402: based on the coordinate conversion result, draw the target road boundary area on the road image.

[0092] In step S401, accurate road boundary information is first obtained, which may be road geometry information extracted from vector map data or other forms of road boundary data; a coordinate transformation matrix is ​​pre-configured according to the coordinate system of the map and the image, and the coordinate transformation matrix includes transformation parameters such as translation, rotation, and scaling; the pre-configured coordinate transformation matrix is ​​used to perform coordinate transformation on the road boundary information through mathematical operations, such as using matrix multiplication to convert map coordinates into image coordinates to obtain a coordinate transformation result.

[0093] In step S402, the road image is first opened. According to the coordinate conversion result in step S401, the position information of the road boundary on the image can be obtained. Using the position information, the target road boundary area is drawn on the image, which can be a simple boundary line or a filled area to highlight the position and shape of the road boundary. After the drawing is completed, the processed image is saved.

[0094] The embodiment of the present invention utilizes a preconfigured coordinate conversion matrix to convert road boundary information from a map coordinate system to an image coordinate system. Through the coordinate conversion result, the target road boundary area can be accurately drawn on the road image, providing accurate basic data for road obstacle identification and analysis, thereby improving the accuracy and visualization of the road boundary area.

[0095] Figure 5 FIG. 1 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention. Figure 5 As shown, step S103 separates obstacles from the background area based on the target road boundary area, including:

[0096] Step S501, configuring the attribute characteristics of the obstacle; wherein the attribute characteristics are the background area sandwiched between the road boundary areas or located within the road boundary areas;

[0097] Step S502: based on the attribute features, the background area is region-clipped, and obstacles are determined according to the region-clipped result.

[0098] Among them, configuring the attribute characteristics of obstacles refers to determining specific attributes used to identify obstacles, and the attribute characteristics can be used to distinguish obstacle areas sandwiched between road boundary areas or located within road boundary areas; obstacles sandwiched between road boundary areas are obstacles existing between the road boundaries on both sides; obstacles located within the road boundary area are obstacles existing inside the road boundary area, and the obstacles span the road boundary.

[0099] Specifically, attribute features include color, texture, shape, size, etc. For example, an obstacle may have a different color from the road background, or present a specific shape and texture in the image.

[0100] Based on the above-mentioned attribute features of the configuration, the background area is clipped, and the obstacle area that meets the specific attribute features is identified by using image processing technology, and it is separated from the background area. First, the attribute features of the configuration are extracted by using image processing algorithms, such as color distribution, texture features or shape information to distinguish between background and obstacles; based on the extracted attribute features, the obstacle area that meets the specific attributes is identified by using threshold segmentation, edge detection, regional growth and other technologies; the identified obstacle area can be clipped by image segmentation algorithms and morphology, such as dilation, corrosion, opening and closing operations, etc.; the location, shape and size of the obstacle and other attributes are determined according to the clipping results, such as by using contour detection, connected region analysis and other methods to determine the characteristics of the obstacle.

[0101] The embodiments of the present invention can effectively separate the obstacle area from the road image, improve the accuracy of obstacle detection, and especially improve the detection rate and accuracy of visual low obstacles.

[0102] Figure 6 FIG. 1 is a schematic diagram of the main process of an obstacle identification method according to another reference embodiment of the present invention. Figure 6 As shown, after separating the obstacles from the background area, the method further includes:

[0103] Step S601, mapping the obstacle into the radar coordinate system based on the inverse perspective mapping algorithm to obtain a mapping result;

[0104] Step S602, performing point clustering processing on the mapping result to generate obstacle polygons;

[0105] Step S603: Send the obstacle polygon as an output result to the target module.

[0106] Specifically, step S601 first performs perspective correction on the obstacle image to eliminate the perspective distortion caused by the camera viewing angle, so that the position of the obstacle in the radar coordinate system is accurate; through the predetermined geometric relationship between the camera and the radar, the mapping relationship between the obstacle in the image and the radar is calculated, and the pixel points in the image are mapped to the physical coordinates of the radar to obtain the mapping result.

[0107] Step S602 performs cluster analysis on the obstacle position information mapped to the radar coordinate system, aggregates points with close distances into a cluster to represent an obstacle; based on the clustering results, generates a polygon representing the shape of the obstacle, for example, the clustered points can be grouped into a polygonal area using a convex hull algorithm or a least squares method.

[0108] Step S603 sends the generated obstacle polygon as the output result to the target module, that is, passes the processed obstacle shape information to the downstream module, such as the target module for obstacle recognition, tracking, path planning and other tasks, to provide accurate obstacle data support for the autonomous driving system.

[0109] The embodiments of the present invention improve the automatic driving system's ability to perceive the surrounding environment, thereby enhancing the safety and driving efficiency of the vehicle.

[0110] Optionally, step S601 maps the obstacle into the radar coordinate system based on an inverse perspective mapping algorithm, including:

[0111] The inverse perspective mapping algorithm is used to convert the pixel coordinates in the obstacle into three-dimensional point coordinates, and the three-dimensional point coordinates are converted into coordinates in the radar coordinate system.

[0112] First, it is necessary to obtain the camera's intrinsic and extrinsic parameters, including the camera's focal length, principal point coordinates, distortion coefficient and other internal parameters, as well as the camera's position and orientation in the vehicle coordinate system and other external parameters; use the camera's intrinsic and extrinsic parameters to perform inverse perspective transformation on the obstacles in the image, and convert the pixel coordinates in the image into three-dimensional point coordinates in the camera coordinate system; determine the geometric relationship between the camera and the radar, such as the position and direction of the radar relative to the camera, as well as information such as the coordinate transformation matrix between the two, and convert the three-dimensional point coordinates in the camera coordinate system into coordinates in the radar coordinate system; output the obstacle coordinates in the radar coordinate system after conversion as the mapping result for use in subsequent processing steps, such as point cloud data processing, obstacle identification, and obstacle avoidance planning.

[0113] In addition to converting the pixel coordinates of the obstacle into three-dimensional point coordinates and then converting the three-dimensional point coordinates into coordinates in the radar coordinate system, it is also possible to extract key points or feature points in the image and then match them with the feature points in the radar point cloud to achieve obstacle mapping based on matching points into the radar coordinate system; it is also possible to combine other sensor information, such as lidar, to fuse the data of multiple sensors, such as cameras, lidars, and radars.

[0114] The embodiment of the present invention converts the pixel coordinates of the obstacle in the image into three-dimensional point coordinates through an inverse perspective mapping algorithm, and then converts these three-dimensional point coordinates into coordinates in the radar coordinate system, so that the autonomous driving system can more accurately perceive low obstacles on the road, improve the accuracy and robustness of low obstacle detection, thereby enhancing the safety and reliability of the driving system, achieving more accurate and efficient low obstacle perception and recognition, and improving the intelligence of autonomous driving technology.

[0115] According to a second aspect of the embodiment of the present invention, Figure 7As shown, an obstacle identification device 700 is provided, comprising:

[0116] A determination module 701 is used to determine a background area in a road image;

[0117] A projection module 702 is used to obtain road boundary information and project the road boundary information onto the road image to obtain a target road boundary area;

[0118] The recognition module 703 is used to separate obstacles from the background area based on the target road boundary area.

[0119] Optionally, the determination module 701 is further configured to predict the background area in the road image using a pre-trained segmentation model;

[0120] The obstacle identification device 700 further includes a training module, which is used to:

[0121] Collect image segmentation data;

[0122] Determine a classification label based on the image segmentation data; wherein the classification label includes a background class and a non-background class;

[0123] Mapping pixels in the image segmentation data to corresponding classification category labels to generate image segmentation annotation data;

[0124] A training sample set is constructed based on the image segmentation annotation data, and a segmentation model is trained using the training sample set.

[0125] Optionally, the projection module 702 is further configured to:

[0126] Get vector map information;

[0127] The vector map information is preprocessed, and road boundary information is extracted from the preprocessed vector map information.

[0128] Optionally, the projection module 702 is further configured to:

[0129] The road boundary information is coordinate-converted using a pre-configured coordinate conversion matrix to obtain a coordinate conversion result;

[0130] Based on the coordinate transformation results, the target road boundary area is drawn on the road image.

[0131] Optionally, the identification module 703 is further used for:

[0132] Configuring attribute features of obstacles; wherein the attribute features are background areas sandwiched between road boundary areas or located within road boundary areas;

[0133] Based on the attribute features, the background area is clipped and the obstacles are determined according to the clipping results.

[0134] Optionally, the obstacle identification device 700 further includes an output module, and the output module is used to:

[0135] Based on the inverse perspective mapping algorithm, the obstacle is mapped into the radar coordinate system to obtain the mapping result;

[0136] Perform point clustering on the mapping results to generate obstacle polygons;

[0137] Send the obstacle polygon as output to the target module.

[0138] Optionally, the output module is also used to:

[0139] The inverse perspective mapping algorithm is used to convert the pixel coordinates in the obstacle into three-dimensional point coordinates, and the three-dimensional point coordinates are converted into coordinates in the radar coordinate system.

[0140] It should be noted that the specific implementation content of the obstacle identification device of the present invention has been described in detail in the above obstacle identification method, so the repeated content will not be described again here.

[0141] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for data analysis, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention.

[0142] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0143] According to a fifth aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0144] Figure 8 An exemplary system architecture 800 is shown to which the obstacle recognition method or obstacle recognition device according to the embodiment of the present invention can be applied.

[0145] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0146] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0147] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0148] The server 805 may be a server that provides various services, such as a background management server (only for example) that provides support for shopping websites browsed by users using the terminal devices 801, 802, and 803. The background management server may analyze and process the received obstacle recognition request and other data, and feed back the processing result (target obstacle - only for example) to the terminal device.

[0149] It should be noted that the obstacle recognition method provided in the embodiment of the present invention is generally executed by the server 805, and accordingly, the obstacle recognition device is generally set in the server 805. The obstacle recognition method provided in the embodiment of the present invention can also be executed by the terminal devices 801, 802, and 803, and accordingly, the obstacle recognition device can be set in the terminal devices 801, 802, and 803.

[0150] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0151] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0152] like Fig. 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0153] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0154] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0155] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0156] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a determination module, a projection module and a recognition module, wherein the names of these modules do not constitute a limitation on the modules themselves under certain circumstances, for example, the determination module may also be described as a "module for determining a background area in a road image".

[0158] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device implements the following method: determining the background area in the road image; obtaining road boundary information, projecting the road boundary information onto the road image, and obtaining the target road boundary area; and separating obstacles from the background area based on the target road boundary area.

[0159] The computer program product provided by the embodiment of the present invention includes a computer program. When the computer program is executed by a processor, the obstacle recognition method in the embodiment of the present invention is implemented.

[0160] According to the technical solution of the embodiment of the present invention, the obstacle recognition method of the embodiment of the present invention determines the background area in the road image; obtains road boundary information, and projects the road boundary information onto the road image to obtain the target road boundary area; separates the obstacle from the background area based on the target road boundary area; this embodiment can improve the detection rate and accuracy of visual low obstacles, and can detect low obstacles across the road boundary.

[0161] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0162] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

Claims

1. An obstacle recognition method, It is characterized in that include: determining a background region in a road image; Acquiring road boundary information, and projecting the road boundary information onto the road image to obtain a target road boundary area; Obstacles are separated from the background area based on the target road boundary area.

2. The method according to claim 1, It is characterized in that Determining the background area in the road image includes: Predicting a background area in the road image using a pre-trained segmentation model; The pre-trained segmentation model is obtained by the following steps: Collect image segmentation data; Determine a division category label based on the image segmentation data; wherein the division category label includes a background category and a non-background category; Mapping pixels in the image segmentation data to corresponding classification category labels to generate image segmentation annotation data; A training sample set is constructed based on the image segmentation annotation data, and the segmentation model is obtained by training using the training sample set.

3. The method according to claim 1, It is characterized in that The obtaining of road boundary information includes: Get vector map information; The vector map information is preprocessed, and road boundary information is extracted from the preprocessed vector map information.

4. The method according to claim 1, It is characterized in that Projecting the road boundary information onto the road image to obtain a target road boundary area includes: Using a pre-configured coordinate transformation matrix to perform coordinate transformation on the road boundary information to obtain a coordinate transformation result; Based on the coordinate transformation result, a target road boundary area is drawn on the road image.

5. The method according to claim 1, It is characterized in that Separating obstacles from the background area based on the target road boundary area includes: Configuring attribute features of obstacles; wherein the attribute features are background areas sandwiched between road boundary areas or located within road boundary areas; Based on the attribute features, the background area is region-clipped, and obstacles are determined according to the region-clipped result.

6. The method according to claim 1, It is characterized in that After separating the obstacles from the background area, the method further includes: Mapping the obstacle into the radar coordinate system based on an inverse perspective mapping algorithm to obtain a mapping result; Performing point clustering processing on the mapping results to generate obstacle polygons; The obstacle polygon is sent as an output result to the target module.

7. The method according to claim 6, It is characterized in that Mapping the obstacle into the radar coordinate system based on an inverse perspective mapping algorithm includes: The pixel coordinates in the obstacle are converted into three-dimensional point coordinates by using an inverse perspective mapping algorithm, and the three-dimensional point coordinates are converted into coordinates in a radar coordinate system.

8. An obstacle recognition device, It is characterized in that include: A determination module, used for determining a background area in a road image; A projection module, used to obtain road boundary information, and project the road boundary information onto the road image to obtain a target road boundary area; A recognition module is used to separate obstacles from the background area based on the target road boundary area.

9. An electronic device, It is characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images