A barrier identification method, device, equipment and readable storage medium

Abstract

This application discloses an obstacle recognition method, apparatus, device, and readable storage medium. It acquires an original noisy point cloud model and a semantic map of the target's travel area generated by a LiDAR scanner, and determines the vehicle's passage area in the original noisy point cloud model based on the annotation information of the semantic map. Using a LiDAR point cloud matching mechanism, the noisy point cloud within the vehicle's passage area is coarsely filtered to generate a preliminary noisy point cloud model. Images of the target's travel area captured by a fisheye camera are acquired, and the location of drainage equipment in the images is used to determine the drainage equipment area. Using the results of cross-referencing the overlapping models between the vehicle's passage area and the drainage equipment area, the preliminary noisy point cloud model is finely filtered to generate a final noisy point cloud model, and the obstacle recognition result is determined. This application, through two rounds of filtering of the point cloud model, effectively improves the thoroughness and accuracy of noise point cloud filtering and reduces interference from drainage equipment reflections.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to an obstacle recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] Unlike other autonomous vehicles, autonomous sanitation vehicles require low-speed, edge-sweeping in practical applications to effectively remove litter from roadside areas and ensure cleaning quality. During edge-sweeping, they encounter numerous manhole covers with high refractive indexes, such as water and water mist. When the autonomous sanitation vehicle's lidar passes near these covers, reflected noise clouds accumulate, easily causing false detections. If these false detections indicate an obstacle in the area the vehicle is about to pass, it will frequently stall and request remote manual verification to confirm the false detection and determine whether to continue.

[0003] According to incomplete statistics, there are 150-200 grid-shaped manhole covers in an average 5km cleaning route that may cause false detection or blockage. If each manhole cover requires remote manual intervention for confirmation, it will seriously affect cleaning efficiency. At the same time, frequent requests will seriously consume remote manual resources, thus delaying requests that truly require remote intervention.

[0004] Furthermore, the reflections from drainage equipment such as sewers and manhole covers can cause vehicles to make large lateral detours during cleaning. This unreasonable detours can lead to the loss of a lot of garbage and leaves, reducing the quality of cleaning.

[0005] Based on the above situation, there is an urgent need for an obstacle recognition solution to solve the problem that when an autonomous sweeper is cleaning at low speed along the edge, it may cause false detections due to the noise point cloud of reflected LiDAR when passing through drainage equipment such as sewers and manhole covers, resulting in the vehicle getting stuck or taking unreasonable detours. Summary of the Invention

[0006] In view of this, this application provides an obstacle recognition method, apparatus, device, and readable storage medium, which combines semantic map and lidar point cloud matching mechanism to perform coarse filtering on noisy point cloud model, and combines fisheye camera image to perform fine filtering on noisy point cloud model, thereby reducing the interference of drainage equipment reflection, thereby reducing the problem of false detection and jamming caused by it, and reducing the frequency of manual remote intervention.

[0007] An obstacle recognition method, comprising:

[0008] Obtain the original noisy point cloud model generated by the lidar scanning the target's travel area;

[0009] Obtain a semantic map of the target travel area, and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map;

[0010] Using the lidar point cloud matching mechanism, the noise point cloud in the aircraft passage area of ​​the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model.

[0011] Acquire images of the target's travel area captured by a fisheye camera, and determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images;

[0012] Using the results of cross-comparison of the model overlap between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate the final noise point cloud model.

[0013] Obstacle recognition is performed based on the noise point cloud model obtained from the results, and the obstacle recognition result is determined.

[0014] Preferably, the semantic map of the target travel area is obtained, and the body travel area in the original noisy point cloud model is determined based on the annotation information of the semantic map, including:

[0015] Obtain a semantic map of the target's travel area;

[0016] Based on the white solid line annotations and roadbed annotations in the semantic map, determine the lateral distance of the aircraft passage area in the original noisy point cloud model;

[0017] Based on the road surface and roadbed markings in the semantic map, determine the longitudinal distance of the body passage area in the original noisy point cloud model;

[0018] Based on the horizontal distance and the vertical distance, the body passage area in the original noisy point cloud model is determined.

[0019] Preferably, the lateral distance of the aircraft's passage area in the original noisy point cloud model is determined based on the white solid line annotations and roadbed annotations in the semantic map, including:

[0020] Determine the first three-dimensional coordinate position of the white solid line annotation and the second three-dimensional coordinate position of the roadbed annotation in the semantic map;

[0021] The difference in lateral coordinates between the first three-dimensional coordinate position and the second three-dimensional coordinate position is determined as the lateral distance of the body passage area in the original noisy point cloud model.

[0022] Preferably, the longitudinal distance of the vehicle's passage area in the original noisy point cloud model is determined based on the road surface and roadbed markings in the semantic map, including:

[0023] Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map;

[0024] The difference in longitudinal coordinates between the second and third three-dimensional coordinate positions is determined as the longitudinal distance of the body passage area in the original noisy point cloud model.

[0025] Preferably, the longitudinal distance of the vehicle's passage area in the original noisy point cloud model is determined based on the road surface and roadbed markings in the semantic map, including:

[0026] Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map;

[0027] Based on the longitudinal coordinate difference between the second and third three-dimensional coordinate positions, and combined with a preset floating error value, the longitudinal distance of the body passage area in the original noisy point cloud model is determined.

[0028] Preferably, the initial screening noise point cloud model is finely filtered using the results of cross-comparison of the model overlap between the machine passage area and the drainage equipment area to generate a final noise point cloud model, including:

[0029] The model overlap and cross-comparison of the machine passage area and the drainage equipment area are performed to determine the overlapping noise point cloud;

[0030] The overlapping noise points in the initial screening noise point cloud model are removed to generate the final noise point cloud model.

[0031] Preferably, acquiring images of the target's travel area captured by a fisheye camera, and determining the drainage equipment region in the original noisy point cloud model based on the location of the drainage equipment in the captured images, includes:

[0032] Acquire images of the target's travel area captured by a fisheye camera;

[0033] Identify drainage equipment in the captured image, and determine the location of the drainage equipment by performing image segmentation on the captured image;

[0034] The area corresponding to the location of the drainage equipment in the original noisy point cloud model is determined as the drainage equipment area.

[0035] An obstacle recognition device, comprising:

[0036] The lidar unit is used to acquire the original noisy point cloud model generated by the lidar scanning the target's travel area;

[0037] A semantic map unit is used to acquire a semantic map of the target's travel area and determine the body's passage area in the original noisy point cloud model based on the annotation information of the semantic map.

[0038] The point cloud screening unit is used to coarsely filter the noise point cloud in the body passage area of ​​the original noise point cloud model using the lidar point cloud matching mechanism, and generate a preliminary screening noise point cloud model.

[0039] The fisheye camera unit is used to acquire images of the target's travel area captured by the fisheye camera, and to determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images.

[0040] The cross-comparison unit is used to perform fine filtering on the initial screening noise point cloud model by utilizing the cross-comparison results of the model overlap between the machine passage area and the drainage equipment area, and to generate the result noise point cloud model.

[0041] The recognition result unit is used to identify obstacles based on the result noise point cloud model and determine the obstacle recognition result.

[0042] An obstacle recognition device includes a memory and a processor;

[0043] The memory is used to store programs;

[0044] The processor is configured to execute the program to implement the various steps of the obstacle recognition method as described in any of the preceding claims.

[0045] A readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the obstacle recognition method as described in any of the preceding claims.

[0046] As can be seen from the above technical solutions, the obstacle recognition method, apparatus, device, and readable storage medium provided in this application acquire an original noise point cloud model generated by a LiDAR scanner scanning a target's travel area, as well as a semantic map of the target's travel area. Based on the annotation information of the semantic map, the machine's passage area in the original noise point cloud model is determined; this machine's passage area represents the area where most of the drainage equipment in the target's travel area is likely located. Then, using a LiDAR point cloud matching mechanism, the noise point cloud within the machine's passage area in the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model, which can reduce the interference from drainage equipment reflections to a certain extent and lower the probability of false detection. Finally, by acquiring images of the target's travel area captured by a fisheye camera, the drainage equipment area in the original noise point cloud model is determined based on the location of the drainage equipment in the captured images; this drainage equipment area is the area where the drainage equipment is actually located, as determined by the captured images. By utilizing the results of cross-referencing the overlapping models between the machine's passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate a final noise point cloud model. This further reduces interference from drainage equipment reflections and improves the thoroughness and accuracy of noise point cloud filtering. Finally, obstacle identification is performed based on the final noise point cloud model to determine the obstacle identification result.

[0047] This application performs two filtering steps on the original noisy point cloud model: a coarse filtering step combining semantic map and lidar point cloud matching mechanism, and a fine filtering step combining images captured by fisheye camera. This effectively improves the thoroughness and accuracy of noisy point cloud filtering, reduces interference from drainage equipment reflections, thereby reducing false detection and jamming problems and lowering the frequency of manual remote intervention. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 This is a flowchart of an obstacle recognition method disclosed in this application;

[0050] Figure 2 This is a schematic diagram of the horizontal and vertical distances disclosed in this application;

[0051] Figure 3 This is a structural block diagram of an obstacle recognition device disclosed in this application;

[0052] Figure 4 This is a hardware structure block diagram of an obstacle recognition device disclosed in this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This application can be used in a wide variety of general-purpose or special-purpose computing environments or configurations. For example: personal computers, server computers, multiprocessor devices, distributed computing environments including any of the above devices or equipment, etc.

[0055] This application provides an obstacle recognition method, which can be applied to the control system of an autonomous sanitation vehicle, and can also be applied to various computer terminals or smart terminals. The executing entity can be the processor or server of the computer terminal or smart terminal.

[0056] Figure 1 This is a flowchart of an obstacle recognition method disclosed in an embodiment of this application, such as... Figure 1 As shown, the method may include:

[0057] Step S1: Obtain the original noisy point cloud model generated by the LiDAR scanning the target's travel area.

[0058] Specifically, the autonomous sanitation vehicle of this application needs to be equipped with LiDAR and a fisheye camera, and it also needs to be able to acquire semantic maps. During the edge-sweeping process, the autonomous sanitation vehicle uses LiDAR to scan the target travel area in real time, generating an original noise point cloud model. In this original noise point cloud model, if there are drainage devices such as manhole covers or sewer outlets in the target travel area, the presence of water and water mist (high refractive index media) inside these devices will result in a large number of interfering noise point clouds in the original noise point cloud model.

[0059] Step S2: Obtain the semantic map of the target travel area, and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map.

[0060] Specifically, a semantic map is a high-level, high-precision map based on a geometric map. When an autonomous vehicle is on the road, in addition to relying on its own sensors for pathfinding and obstacle avoidance, it also needs to use a semantic map to determine its position and plan its route. The semantic map also contains a lot of labeled information, such as curves, straight sections, intersections, and information about street equipment. Therefore, in this application, the autonomous sanitation vehicle can obtain a semantic map of the target travel area and determine the vehicle's passage area in the original noisy point cloud model based on the labeled information of the semantic map. This vehicle passage area represents the area where most of the drainage equipment in the target travel area is likely located.

[0061] Step S3: Using the lidar point cloud matching mechanism, coarsely filter the noise point cloud in the machine passage area of ​​the original noise point cloud model to generate a preliminary noise point cloud model.

[0062] Specifically, the point cloud matching mechanism of the blind zone lidar is used to initially screen the noisy point cloud in the area where the aircraft passes, and to perform preliminary coarse filtering on the clustered noisy point cloud in the area where the aircraft passes. Since most of the drainage equipment may be located in the target's travel area, coarse filtering can reduce the reflection interference of the drainage equipment to a certain extent and reduce the probability of false detection.

[0063] Step S4: Obtain images of the target's travel area captured by a fisheye camera, and determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images.

[0064] Specifically, considering the impact of water spray and dust generated by the rotating brushes during actual use of autonomous sanitation vehicles in China, the accuracy and precision of the noise point cloud filtering model are not high enough, leaving considerable residual noise point clouds. Therefore, to further improve the accuracy of noise point cloud filtering, a combination of LiDAR and a fisheye camera is used to identify and segment drainage equipment in the target area to be cleaned. After identifying the area where the drainage equipment is located on the normal road surface from the fisheye camera image, the drainage equipment area in the original noise point cloud model can be determined based on the location of the drainage equipment in the captured image. This drainage equipment area is the area where the drainage equipment is located, determined through actual shooting, and may specifically include:

[0065] ①Acquire images of the target's travel area captured by a fisheye camera;

[0066] ② Identify the drainage equipment in the captured image, and determine the location of the drainage equipment by performing image segmentation on the captured image;

[0067] ③The area corresponding to the location of the drainage equipment in the original noise point cloud model is determined as the drainage equipment area.

[0068] Step S5: Using the results of the model overlap and cross-comparison between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate the final noise point cloud model.

[0069] Specifically, the overlapping models of the machine passage area and the drainage equipment area are cross-compared based on their lateral and longitudinal distances. This allows for a further screening of the remaining noise point clouds after the initial coarse filtering by noise point cloud model matching. Using the results of the model overlap cross-comparison, the initially screened noise point cloud model is then finely filtered. This involves removing overlapping noise point clouds from the machine passage area and drainage equipment area in the initial noise point cloud model, thereby improving the thoroughness and accuracy of noise point cloud filtering. Specifically, this can include:

[0070] ① Perform model overlap and cross-comparison between the machine body passage area and the drainage equipment area to determine the overlapping noise point cloud;

[0071] ②The overlapping noise point clouds in the initial screening noise point cloud model are removed to generate the result noise point cloud model.

[0072] Step S6: Based on the resulting noise point cloud model, perform obstacle recognition and determine the obstacle recognition result.

[0073] As can be seen from the above technical solutions, the obstacle recognition method, apparatus, device, and readable storage medium provided in this application acquire an original noise point cloud model generated by a LiDAR scanner scanning a target's travel area, as well as a semantic map of the target's travel area. Based on the annotation information of the semantic map, the machine's passage area in the original noise point cloud model is determined; this machine's passage area represents the area where most of the drainage equipment in the target's travel area is likely located. Then, using a LiDAR point cloud matching mechanism, the noise point cloud within the machine's passage area in the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model, which can reduce the interference from drainage equipment reflections to a certain extent and lower the probability of false detection. Finally, by acquiring images of the target's travel area captured by a fisheye camera, the drainage equipment area in the original noise point cloud model is determined based on the location of the drainage equipment in the captured images; this drainage equipment area is the area where the drainage equipment is actually located, as determined by the captured images. By utilizing the results of cross-referencing the overlapping models between the machine's passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate a final noise point cloud model. This further reduces interference from drainage equipment reflections and improves the thoroughness and accuracy of noise point cloud filtering. Finally, obstacle identification is performed based on the final noise point cloud model to determine the obstacle identification result.

[0074] This application performs two filtering steps on the original noisy point cloud model: a coarse filtering step combining semantic map and lidar point cloud matching mechanism, and a fine filtering step combining images captured by fisheye camera. This effectively improves the thoroughness and accuracy of noisy point cloud filtering, reduces interference from drainage equipment reflections, thereby reducing false detection and jamming problems and lowering the frequency of manual remote intervention.

[0075] In some embodiments of this application, the process of step S2, obtaining a semantic map of the target travel area, and determining the body passage area in the original noisy point cloud model based on the annotation information of the semantic map, is described, and may specifically include:

[0076] Step S21: Obtain the semantic map of the target travel area.

[0077] Step S22: Determine the lateral distance of the aircraft passage area in the original noisy point cloud model based on the white solid line annotations and roadbed annotations in the semantic map.

[0078] Specifically, this application uses the coverage area of ​​the blind spot lidar and the coverage area of ​​the sweeping trajectory of the autonomous sanitation vehicle as references. It requires segmenting the area marked by the solid white lines and the roadbed on the semantic map, and using this segmentation as the basis for initially screening the lateral distance of the noise point cloud area. This is because the construction locations of most road maintenance facilities such as manhole covers, sewers, and drainage outlets are chosen within the area between the solid white lines and the roadbed. The normal distance between the solid white lines and the roadbed is 30cm-50cm, although this may vary slightly depending on the location.

[0079] Based on the white solid line annotations and roadbed annotations in the semantic map, the lateral distance of the aircraft's passage area in the original noisy point cloud model can be determined, which may specifically include the following steps:

[0080] ① Determine the first three-dimensional coordinate position of the white solid line annotation and the second three-dimensional coordinate position of the roadbed annotation in the semantic map;

[0081] ②The difference in lateral coordinates between the first three-dimensional coordinate position and the second three-dimensional coordinate position is determined as the lateral distance of the body passage area in the original noise point cloud model.

[0082] like Figure 2 As shown, the drainage equipment is located between the solid white line on the road surface and the roadbed. Based on the solid white line and roadbed labels in the semantic map, the first three-dimensional coordinate position of the solid white line label and the second three-dimensional coordinate position of the roadbed label can be determined. The lateral coordinate difference a1 between the first three-dimensional coordinate position and the second three-dimensional coordinate position can be calculated, and a1 can be determined as the lateral distance of the body passage area in the original noisy point cloud model.

[0083] Step S23: Determine the longitudinal distance of the vehicle passage area in the original noisy point cloud model based on the road surface and roadbed markings in the semantic map.

[0084] Specifically, by using blind zone lidar combined with semantic map annotations, the three-dimensional coordinates of the road surface and roadbed can be obtained. The height difference between the highest and lowest points can be used to determine the true height of the roadbed from the ground. Due to possible detection errors of lidar and height errors in roadbed manufacturing, a floating error value can be set, such as 1cm. The value of the true height plus or minus 1cm can be used as the basis for initially screening the longitudinal distance of the noise point cloud area.

[0085] This application provides two optional methods for determining the longitudinal distance of the aircraft's passage area in the original noisy point cloud model based on the road surface and roadbed annotations in the semantic map:

[0086] The first type

[0087] ① Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map;

[0088] ②The difference between the longitudinal coordinates of the second three-dimensional coordinate position and the third three-dimensional coordinate position is determined as the longitudinal distance of the body passage area in the original noisy point cloud model.

[0089] like Figure 2 As shown, the drainage equipment is located between the solid white line on the road surface and the roadbed, and its height is between the top of the roadbed and the ground. Based on the road surface and roadbed labels in the semantic map, the second three-dimensional coordinate position of the roadbed label and the third three-dimensional coordinate position of the road surface label can be determined. The longitudinal coordinate difference a2 between the second three-dimensional coordinate position and the third three-dimensional coordinate position can be calculated, and a2 can be determined as the longitudinal distance of the body passage area in the original noisy point cloud model.

[0090] The second type

[0091] ① Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map;

[0092] ②Based on the difference in longitudinal coordinates between the second and third three-dimensional coordinate positions, and combined with a preset floating error value, the longitudinal distance of the body passage area in the original noisy point cloud model is determined.

[0093] exist Figure 2In the example, due to the possible detection error of the lidar and the height difference in the roadbed production, a floating error value of 1cm can be set. Then, the longitudinal coordinate difference a2 between the second and third three-dimensional coordinate positions can be calculated, and a2+1cm can be determined as the longitudinal distance of the body passage area in the original noisy point cloud model.

[0094] Step S24: Based on the horizontal distance and the vertical distance, determine the body passage area in the original noisy point cloud model.

[0095] Specifically, after calculating the lateral and longitudinal distances according to the above method, the three-dimensional solid region constructed by the lateral and longitudinal distances in the original noise point cloud model can be identified as the body passage area in the original noise point cloud model.

[0096] The obstacle recognition device provided in the embodiments of this application is described below. The obstacle recognition device described below and the obstacle recognition method described above can be referred to in correspondence.

[0097] See Figure 3 , Figure 3 This is a structural block diagram of an obstacle recognition device disclosed in an embodiment of this application.

[0098] like Figure 3 As shown, the obstacle recognition device may include:

[0099] The lidar unit 110 is used to acquire the original noisy point cloud model generated by the lidar scanning the target's travel area;

[0100] Semantic map unit 120 is used to acquire a semantic map of the target travel area and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map.

[0101] The point cloud screening unit 130 is used to use the lidar point cloud matching mechanism to coarsely filter the noise point cloud in the body passage area of ​​the original noise point cloud model and generate a preliminary screening noise point cloud model.

[0102] The fisheye camera unit 140 is used to acquire images of the target's travel area captured by the fisheye camera, and to determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images.

[0103] Cross-comparison unit 150 is used to perform fine filtering on the initial screening noise point cloud model by utilizing the cross-comparison results of the model overlap between the machine passage area and the drainage equipment area, and generate the result noise point cloud model.

[0104] The recognition result unit 160 is used to perform obstacle recognition based on the result noise point cloud model and determine the obstacle recognition result.

[0105] As can be seen from the above technical solutions, the obstacle recognition method, apparatus, device, and readable storage medium provided in this application acquire an original noise point cloud model generated by a LiDAR scanner scanning a target's travel area, as well as a semantic map of the target's travel area. Based on the annotation information of the semantic map, the machine's passage area in the original noise point cloud model is determined; this machine's passage area represents the area where most of the drainage equipment in the target's travel area is likely located. Then, using a LiDAR point cloud matching mechanism, the noise point cloud within the machine's passage area in the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model, which can reduce the interference from drainage equipment reflections to a certain extent and lower the probability of false detection. Finally, by acquiring images of the target's travel area captured by a fisheye camera, the drainage equipment area in the original noise point cloud model is determined based on the location of the drainage equipment in the captured images; this drainage equipment area is the area where the drainage equipment is actually located, as determined by the captured images. By utilizing the results of cross-referencing the overlapping models between the machine's passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate a final noise point cloud model. This further reduces interference from drainage equipment reflections and improves the thoroughness and accuracy of noise point cloud filtering. Finally, obstacle identification is performed based on the final noise point cloud model to determine the obstacle identification result.

[0106] This application performs two filtering steps on the original noisy point cloud model: a coarse filtering step combining semantic map and lidar point cloud matching mechanism, and a fine filtering step combining images captured by fisheye camera. This effectively improves the thoroughness and accuracy of noisy point cloud filtering, reduces interference from drainage equipment reflections, thereby reducing false detection and jamming problems and lowering the frequency of manual remote intervention.

[0107] Optionally, the semantic map unit may include:

[0108] The map acquisition unit is used to acquire a semantic map of the target's travel area;

[0109] The lateral distance unit is used to determine the lateral distance of the aircraft passage area in the original noisy point cloud model based on the white solid line annotations and roadbed annotations in the semantic map.

[0110] The longitudinal distance unit is used to determine the longitudinal distance of the vehicle passage area in the original noisy point cloud model based on the road surface and roadbed markings in the semantic map.

[0111] The passage area unit is used to determine the body passage area in the original noisy point cloud model based on the horizontal distance and the vertical distance.

[0112] Optionally, the lateral distance unit may include:

[0113] A horizontal coordinate unit is used to determine the first three-dimensional coordinate position of the white solid line annotation and the second three-dimensional coordinate position of the roadbed annotation in the semantic map;

[0114] A lateral calculation unit is used to determine the lateral coordinate difference between the first three-dimensional coordinate position and the second three-dimensional coordinate position as the lateral distance of the body passage area in the original noise point cloud model.

[0115] Optionally, the longitudinal distance unit may include:

[0116] The vertical coordinate unit is used to determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map;

[0117] The longitudinal calculation unit is used to determine the longitudinal coordinate difference between the second three-dimensional coordinate position and the third three-dimensional coordinate position as the longitudinal distance of the body passage area in the original noisy point cloud model.

[0118] Optionally, the longitudinal calculation unit can also be used to determine the longitudinal distance of the body passage area in the original noisy point cloud model based on the longitudinal coordinate difference between the second three-dimensional coordinate position and the third three-dimensional coordinate position, combined with a preset floating error value.

[0119] Optionally, the cross-comparison unit may include:

[0120] The overlap comparison unit is used to perform model overlap cross-comparison between the machine body passage area and the drainage equipment area to determine the overlapping noise point cloud;

[0121] The point cloud removal unit is used to remove the overlapping noise point clouds in the initial screening noise point cloud model to generate the result noise point cloud model.

[0122] Optionally, the fisheye camera unit may include:

[0123] The image capturing unit is used to acquire images of the target's travel area captured by the fisheye camera.

[0124] The device identification unit is used to identify the drainage device in the captured image and determine the location of the drainage device by performing image segmentation on the captured image.

[0125] The drainage area unit is used to determine the area corresponding to the location of the drainage equipment in the original noisy point cloud model as the drainage equipment area.

[0126] The obstacle recognition device provided in this application embodiment can be applied to obstacle recognition equipment. Figure 4 The hardware structure block diagram of the obstacle recognition device is shown below. Figure 4 The hardware structure of an obstacle recognition device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0127] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4.

[0128] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0129] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0130] The memory stores a program, which the processor can call. The program is used for:

[0131] Obtain the original noisy point cloud model generated by the lidar scanning the target's travel area;

[0132] Obtain a semantic map of the target travel area, and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map;

[0133] Using the lidar point cloud matching mechanism, the noise point cloud in the aircraft passage area of ​​the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model.

[0134] Acquire images of the target's travel area captured by a fisheye camera, and determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images;

[0135] Using the results of cross-comparison of the model overlap between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate the final noise point cloud model.

[0136] Obstacle recognition is performed based on the noise point cloud model obtained from the results, and the obstacle recognition result is determined.

[0137] Optionally, the refined and extended functions of the program can be referred to the above description.

[0138] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0139] Obtain the original noisy point cloud model generated by the lidar scanning the target's travel area;

[0140] Obtain a semantic map of the target travel area, and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map;

[0141] Using the lidar point cloud matching mechanism, the noise point cloud in the aircraft passage area of ​​the original noise point cloud model is coarsely filtered to generate a preliminary noise point cloud model.

[0142] Acquire images of the target's travel area captured by a fisheye camera, and determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images;

[0143] Using the results of cross-comparison of the model overlap between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate the final noise point cloud model.

[0144] Obstacle recognition is performed based on the noise point cloud model obtained from the results, and the obstacle recognition result is determined.

[0145] Optionally, the refined and extended functions of the program can be referred to the above description.

[0146] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0147] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0148] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An obstacle recognition method characterized by, include: Obtain the original noisy point cloud model generated by the lidar scanning the target's travel area; Obtain a semantic map of the target travel area, and determine the body travel area in the original noisy point cloud model based on the annotation information of the semantic map; Using the lidar point cloud matching mechanism, the noise point cloud in the aircraft passage area of ​​the original noise point cloud model is coarsely filtered to generate a preliminary screening noise point cloud model. Acquire images of the target's travel area captured by a fisheye camera, and determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images; Using the results of cross-comparison of the model overlap between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate the final noise point cloud model. Based on the resulting noisy point cloud model, obstacle recognition is performed to determine the obstacle recognition result; Obtain a semantic map of the target travel area, and determine the aircraft's passage area in the original noisy point cloud model based on the annotation information of the semantic map, including: Obtain a semantic map of the target's travel area; Based on the white solid line annotations and roadbed annotations in the semantic map, determine the lateral distance of the aircraft passage area in the original noisy point cloud model; Based on the road surface and roadbed markings in the semantic map, determine the longitudinal distance of the body passage area in the original noisy point cloud model; Based on the horizontal distance and the vertical distance, the body passage area in the original noisy point cloud model is determined.

2. The method of claim 1, wherein, Based on the white solid line annotations and roadbed annotations in the semantic map, determine the lateral distance of the aircraft's passage area in the original noisy point cloud model, including: Determine the first three-dimensional coordinate position of the white solid line annotation and the second three-dimensional coordinate position of the roadbed annotation in the semantic map; The difference in lateral coordinates between the first three-dimensional coordinate position and the second three-dimensional coordinate position is determined as the lateral distance of the body passage area in the original noisy point cloud model.

3. The method of claim 1, wherein, Based on the road surface and roadbed annotations in the semantic map, the longitudinal distance of the vehicle's passage area in the original noisy point cloud model is determined, including: Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map; The difference in longitudinal coordinates between the second and third three-dimensional coordinate positions is determined as the longitudinal distance of the body passage area in the original noisy point cloud model.

4. The method of claim 1, wherein, Based on the road surface and roadbed annotations in the semantic map, the longitudinal distance of the vehicle's passage area in the original noisy point cloud model is determined, including: Determine the second three-dimensional coordinate position of the roadbed annotation and the third three-dimensional coordinate position of the road surface annotation in the semantic map; Based on the longitudinal coordinate difference between the second and third three-dimensional coordinate positions, and combined with a preset floating error value, the longitudinal distance of the body passage area in the original noisy point cloud model is determined.

5. The method according to claim 1, characterized in that, Based on the results of cross-referencing the model overlap between the machine passage area and the drainage equipment area, the initial screening noise point cloud model is finely filtered to generate a final noise point cloud model, including: The model overlap and cross-comparison of the machine passage area and the drainage equipment area are performed to determine the overlapping noise point cloud; The overlapping noise points in the initial screening noise point cloud model are removed to generate the final noise point cloud model.

6. The method according to claim 1, characterized in that, Acquire images of the target's travel area captured by a fisheye camera, and determine the drainage equipment region in the original noisy point cloud model based on the location of the drainage equipment in the captured images, including: Acquire images of the target's travel area captured by a fisheye camera; Identify drainage equipment in the captured image, and determine the location of the drainage equipment by performing image segmentation on the captured image; The area corresponding to the location of the drainage equipment in the original noisy point cloud model is determined as the drainage equipment area.

7. An obstacle recognition device, characterized in that, include: The lidar unit is used to acquire the original noisy point cloud model generated by the lidar scanning the target's travel area; A semantic map unit is used to acquire a semantic map of the target's travel area and determine the body's passage area in the original noisy point cloud model based on the annotation information of the semantic map. The point cloud screening unit is used to coarsely filter the noise point cloud in the body passage area of ​​the original noise point cloud model using the lidar point cloud matching mechanism, and generate a preliminary screening noise point cloud model. The fisheye camera unit is used to acquire images of the target's travel area captured by the fisheye camera, and to determine the drainage equipment area in the original noise point cloud model based on the location of the drainage equipment in the captured images. The cross-comparison unit is used to perform fine filtering on the initial screening noise point cloud model by utilizing the cross-comparison results of the model overlap between the machine passage area and the drainage equipment area, and to generate the result noise point cloud model. The recognition result unit is used to perform obstacle recognition based on the result noise point cloud model and determine the obstacle recognition result; Obtain a semantic map of the target travel area, and determine the aircraft's passage area in the original noisy point cloud model based on the annotation information of the semantic map, including: Obtain a semantic map of the target's travel area; Based on the white solid line annotations and roadbed annotations in the semantic map, determine the lateral distance of the aircraft passage area in the original noisy point cloud model; Based on the road surface and roadbed markings in the semantic map, determine the longitudinal distance of the body passage area in the original noisy point cloud model; Based on the horizontal distance and the vertical distance, the body passage area in the original noisy point cloud model is determined.

8. An obstacle recognition device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the obstacle recognition method as described in any one of claims 1-6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the various steps of the obstacle recognition method as described in any one of claims 1-6.

Images