Negative obstacle recognition method and related equipment

Abstract

The invention provides a negative obstacle recognition method and related equipment. The method comprises the following steps: acquiring a road image of a lane when a vehicle runs on the lane; performing first image identification on the road image to obtain a first area which at least comprises one negative obstacle; performing second image identification on the road image to obtain the lane pixel width of the lane line; a second area is obtained according to the first area and the lane pixel width, and the size of the second area is the actual size of the negative obstacle area; first point cloud data of the negative obstacle is acquired, a first distance is determined according to the first point cloud data, and the first distance is used for representing the distance from the spatial position of the negative obstacle to the road reference plane; and determining the type of the negative obstacle based on the second area and the first distance. The negative obstacle can be detected in real time, and the accuracy of recognizing the type of the negative obstacle is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a negative obstacle recognition method and related equipment. Background Art

[0002] With the continuous increase in the number of motor vehicles in my country, considerable pressure has been placed on the management and maintenance of roads. It is particularly important to accurately identify common road obstacles such as potholes, fracture surfaces, and collapses.

[0003] In the prior art, road pothole identification is usually based on sensors. Usually, the information output of the three-axis acceleration sensor is used to detect whether the road has potholes, and the location of the potholes is obtained through the satellite positioning system. In this way, for the scene of pothole section information, the vehicle cannot make real-time judgments. For example, the information collected by the first vehicle can only be transmitted to other vehicles after the first vehicle passes the pothole section. Since the vehicle cannot identify the negative obstacles in the current driving scene in real time, the vehicle cannot issue an early warning for the current negative obstacles. At the same time, in the prior art, the vehicle can only collect smaller potholes on the road, and cannot collect large-scale road cracks and collapses.

[0004] Therefore, the existing methods for detecting negative obstacles on the road by vehicles still have great limitations, and it is particularly important to develop a method for detecting negative obstacles in real time. Summary of the invention

[0005] In view of this, the present application provides a negative obstacle identification method and related equipment, which can detect negative obstacles in real time and improve the accuracy of identifying the types of negative obstacles.

[0006] A first aspect of an embodiment of the present application provides a negative obstacle identification method, comprising: collecting a road image of a lane when a vehicle is traveling on the lane; performing a first image recognition on the road image to obtain a first area, the first area including at least one negative obstacle; performing a second image recognition on the road image to obtain a lane pixel width of a lane line; obtaining a second area based on the first area and the lane pixel width, the size of the second area being the actual size of the negative obstacle area; obtaining first point cloud data of the negative obstacle, determining a first distance based on the first point cloud data, the first distance being used to represent the distance from the spatial position of the negative obstacle to a road reference plane; and determining the type of the negative obstacle based on the second area and the first distance.

[0007] Compared with the related art, the embodiments of the present application have at least the following advantages: This application can obtain the recognition results of negative obstacles in real time by collecting road images and point cloud data in real time during vehicle driving and analyzing the road images and point cloud data. At the same time, this application can obtain the correspondence between lane line images and lane lines by identifying the negative obstacle area and lane line images through road images, and determine the actual area of ​​the negative obstacle based on the correspondence; obtain the depth information of the negative obstacle through point cloud data, identify the two-dimensional features of the negative obstacle through images, and identify the three-dimensional features of the negative obstacle through point cloud data, thereby improving the accuracy of obtaining the types of negative obstacles.

[0008] Optionally, first point cloud data of the negative obstacle is obtained, and a first distance is determined based on the first point cloud data, including: preprocessing the first point cloud data to determine point cloud features of the first point cloud data; spatially clustering the first point cloud data based on the point cloud features to obtain spatial feature points; and obtaining the distance from the spatial feature points to a road reference plane to obtain a first distance.

[0009] Optionally, determining the type of the negative obstacle based on the second area and the first distance includes: traversing the boundary points of the second area to find the longest distance between the boundary points of the second area; and determining the type of the negative obstacle according to the longest distance and the first distance.

[0010] Optionally, the negative obstacles include: first category negative obstacles, second category negative obstacles and third category negative obstacles, the first category negative obstacles include potholes, the second category negative obstacles include: extra-wide road potholes and road subsidence, and the third category negative obstacles include: road collapse; according to the longest distance and the first distance, the type of negative obstacle is determined, including: when the first distance is less than the first preset distance and the longest distance is less than half the lane width, the negative obstacle is the first category negative obstacle; when the first distance is less than the first preset distance and the longest distance is not less than half the lane width, the negative obstacle is the second category negative obstacle; when the first distance is not less than the first preset distance; or, obtain and judge the road condition, and when the road condition meets the preset road condition conditions, the negative obstacle is the third category negative obstacle.

[0011] Optionally, judging the road condition includes: obtaining a lane line structure and a lane line category based on a road image; judging whether the lane is a continuous lane based on the lane line structure and the lane line category; if the judgment result is a non-continuous lane, the road condition meets a preset road condition condition.

[0012] Optionally, after determining the type of the negative obstacle, the method further includes: determining an avoidance attribute of the negative obstacle, where the avoidance attribute includes an avoidable attribute and an unavoidable attribute.

[0013] Optionally, determining the avoidance attribute of the negative obstacle includes: determining the avoidance space of the vehicle based on the road image and the first area; obtaining the safe driving distance of the vehicle, and when the avoidance space is greater than the safe driving distance of the vehicle, the avoidance attribute is an avoidable attribute; when the avoidance space is not greater than the safe driving distance of the vehicle, the avoidance attribute is an unavoidable attribute.

[0014] In a second aspect, the present application provides a negative obstacle recognition device, comprising: A collection unit, used for collecting a road image of a lane when a vehicle travels in the lane; A first recognition unit, configured to perform first image recognition on the road image to obtain a first area, wherein the first area includes at least one negative obstacle; A second recognition unit is used to perform second image recognition on the road image to obtain a lane pixel width of the lane line; A first determining unit is used to obtain a second area according to the first area and the lane pixel width, where the size of the second area is the actual size of the negative obstacle area; A second determination unit is used to obtain first point cloud data of the negative obstacle, and determine a first distance according to the first point cloud data, where the first distance is used to represent the distance from the spatial position of the negative obstacle to the road reference plane; The result unit is used to determine the type of the negative obstacle based on the second area and the first distance.

[0015] In a third aspect, an embodiment of the present application further provides a vehicle, the vehicle comprising a processor and a memory, the memory being used to store instructions, and the processor being used to call the instructions in the memory so that the vehicle executes the negative obstacle recognition method as in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a vehicle, the vehicle executes the negative obstacle identification method as in the first aspect.

[0017] The technical effects obtained by the above-mentioned second, third and fourth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of the steps of a negative obstacle identification method provided in one embodiment of the present application.

[0019] Figure 2 A flowchart of the steps of obtaining a second area provided in one embodiment of the present application.

[0020] Figure 3 A flowchart of the steps of a negative obstacle identification method provided in yet another embodiment of the present application.

[0021] Figure 4 A flowchart of the steps of a negative obstacle identification method provided in yet another embodiment of the present application.

[0022] Figure 5 A functional module diagram of a negative obstacle recognition device provided in one embodiment of the present application.

[0023] Figure 6 A schematic diagram of the structure of a vehicle provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the implementation methods of the present application and the features in the implementation methods can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The described implementations are only part of the implementations of the present application, rather than all the implementations.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0027] It should be further noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0028] In this application, "at least one" means one or more, and "more" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0029] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0030] To facilitate understanding, some illustrations of concepts related to the embodiments of the present application are given by way of example for reference.

[0031] Road potholes or road potholes refer to the phenomenon of potholes caused by improper design, construction, maintenance, inadequate control, and the influence of natural factors such as climate, environment, geology, and hydrology, as well as vehicle operation and vehicle overloading. Road potholes are usually small, deep, and irregular in shape, and are often found on the surface or joints of concrete slabs.

[0032] Extra-wide road potholes are large-area potholes caused by differential settlement of the new and old roadbeds or construction problems during the process of widening the road. These potholes are usually large and deep, which may affect driving safety and road capacity.

[0033] Subsidence, also known as settlement, is a very common road disease. The road surface sinks due to compression of the foundation soil or other reasons. Settlement usually manifests as partial or overall sinking of the road surface, which may be accompanied by cracks, affecting the flatness and comfort of the road.

[0034] Road collapse refers to the collapse of a road in whole or in part due to geological disasters (such as landslides, earthquakes) or serious construction quality problems. Collapse usually refers to large-scale road damage, which may make the road completely impassable and difficult to repair.

[0035] With the continuous increase in the number of motor vehicles in my country, considerable pressure has been placed on the management and maintenance of roads. It is particularly important to accurately identify common road obstacles such as potholes, fracture surfaces, and collapses.

[0036] In the prior art, road pothole identification is usually based on sensors. Usually, the information output of the three-axis acceleration sensor is used to detect whether the road has potholes, and the location of the potholes is obtained through the satellite positioning system. In this way, for the scene of pothole section information, the vehicle cannot make real-time judgments. For example, the information collected by the first vehicle can only be transmitted to other vehicles after the first vehicle passes the pothole section. Since the vehicle cannot identify the negative obstacles in the current driving scene in real time, the vehicle cannot issue an early warning for the current negative obstacles. At the same time, in the prior art, the vehicle can only collect smaller potholes on the road, and cannot collect large-scale road cracks and collapses.

[0037] Therefore, the existing methods for detecting negative obstacles on the road by vehicles still have great limitations, and it is particularly important to develop a method for detecting negative obstacles in real time.

[0038] The embodiments of the present application provide a negative obstacle identification method and related devices, which can detect negative obstacles in real time and improve the accuracy of identifying the types of negative obstacles.

[0039] The specific process of this embodiment is as follows Figure 1 As shown, the following steps are included: Step 110 , collecting a road image of the lane when the vehicle is traveling in the lane.

[0040] It is understandable that a vehicle may include multiple cameras, one of which is used to collect road images. The collected road images are real-time road images. Negative obstacles refer to obstacles below the road surface such as potholes, potholes, fracture surfaces, and collapses.

[0041] Different negative space obstacles have different image features in the image. Potholes usually appear as irregular black or dark areas with clear edges and may be surrounded by cracks or gravel. Ultra-wide road potholes occupy a large area in the image, with irregular shapes and deep depths, which may affect multiple lanes. Subsidence appears in the image as a drop in the entire or part of the road surface, which may be accompanied by cracks or ripples. The color change is not obvious, but there will be obvious shadow effects. Collapse appears in the image as a large area of ​​missing road surface, with irregular edges, and may be accompanied by a large amount of time and soil, with strong color contrast.

[0042] Step 120 , performing first image recognition on the road image to obtain a first area, where the first area includes at least one negative obstacle.

[0043] The first image recognition can use the RetinaNet algorithm. The RetinaNet algorithm of the present application consists of a main network and two sub-networks. Among them, the Feature Pyramid Network (FPN) is used as the main network. The main network is responsible for extracting multi-scale features from the input image. Each layer of the main network is horizontally connected to the ResNet convolutional network from top to bottom. The ResNet convolutional network is used to calculate convolutional features of different scales. The two sub-networks include a first sub-network and a second sub-network. The first sub-network is a class sub-network (class subnet), and the second sub-network is a bounding box regression sub-network (box subnet). The first sub-network is used to perform convolution target classification on the output of the main network, and the second sub-network uses a bounding box regression algorithm to accurately locate the bounding box surrounding the target object. In this embodiment, the obstacle area of ​​the obstacle image can be determined according to the bounding box.

[0044] Step 130 , performing a second image recognition on the road image to obtain the lane pixel width of the lane line.

[0045] In this embodiment, lane line pixels and background in a road image can be classified into two categories through a binarization segmentation network (lane segmentation net) and an instance segmentation network (lane embedding net), thereby identifying lane line pixels in the road image and obtaining a lane line image.

[0046] Step 140, obtaining a second area according to the first area and the lane pixel width, wherein the size of the second area is the actual size of the negative obstacle area.

[0047] In this embodiment, the lane line image of the road image is identified, the lane width is obtained, and the image conversion relationship is determined according to the lane line image and the lane width. The image conversion relationship is used to convert the pixel length in the image coordinate system into the actual length in the world coordinate system. Figure 2 As shown, Figure 2 To obtain a scene diagram of the second area, Figure 2 There is a second area 200 on the road in , and there is a negative obstacle in the second area. Optionally, the second area is a bounding box of the negative obstacle. The left lane line of the road is H1, the right lane line of the road is H2, the lane width L2 is the vertical distance between H1 and H2, and the obstacle width L1 is the maximum width obtained in the direction parallel to L2.

[0048] In the road image phase, the lane pixel width w-ane corresponding to L1 is obtained, and the obstacle pixel width w-hole corresponding to L2 is obtained. The specific conversion relationship between L1 and L2 is shown in formula (1): Formula (1); From formula (1), we can know that the lane width of L2 is the actual width of the road on which the vehicle is traveling. The value of L2 can be actually measured by the vehicle sensor or obtained through road construction standards. When L2 and the lane pixel width w-ane are known, the proportional relationship between the world pixel coordinates and the image coordinates can be calculated. The actual obstacle width can be calculated based on the proportional relationship between the world pixel coordinates and the image coordinates and the obstacle pixel width whole in the image.

[0049] Step 150: Acquire first point cloud data of the negative obstacle, and determine a first distance according to the first point cloud data. The first distance is used to represent the distance from the spatial position of the negative obstacle to the road reference plane.

[0050] The road point cloud data is collected by a lidar sensor. The road point cloud data includes the depth information of negative obstacles. Different negative obstacles correspond to different depth information.

[0051] The road reference plane is the horizontal plane of the road measured by the vehicle. During the driving process of the vehicle, the vehicle can determine the road reference plane through laser radar, radar, camera or inertial measurement unit. Laser radar generates a high-precision three-dimensional road map by emitting laser beams and measuring the time of reflection, so as to determine the shape and horizontal plane of the road from the three-dimensional road map. Radar detects the distance and speed of objects by emitting electromagnetic waves and receiving reflected signals, which helps to identify the ups and downs of the road. The camera collects road images, and the vehicle identifies road markings and boundaries through image processing technology to determine the horizontal plane of the road. The inertial measurement unit monitors the acceleration and angular velocity of the vehicle through accelerometers and gyroscopes, helps the vehicle understand its motion state and posture, and determines the horizontal plane of the road based on the vehicle's motion posture.

[0052] Step 160, determining the type of negative obstacle based on the second area and the first distance. Determining the type of negative obstacle based on the second area and the first distance includes: traversing the boundary points of the second area, finding the longest distance between the boundary points of the second area; and determining the type of negative obstacle based on the longest distance and the first distance.

[0053] Among them, negative obstacles include: potholes, extra-wide road potholes, subsidence and road collapse. The preliminary features of negative obstacles are judged according to the negative obstacle area and lane line pixels, and the preliminary features are further judged according to the depth information to obtain the type of negative obstacles.

[0054] like Figure 3 As shown, Figure 3 A flowchart of a negative obstacle identification method provided in another embodiment of the present application, step 150 acquires first point cloud data of a negative obstacle, and determines a first distance according to the first point cloud data, including: Step 1510: pre-process the first point cloud data to determine point cloud features of the first point cloud data.

[0055] The point cloud data of different negative obstacles present different characteristics, among which: potholes are manifested as local height reduction, low point cloud density, and high edge point cloud density. Ultra-wide road potholes are manifested as reduced point cloud height, deep depth, and obvious changes in point cloud density, which may affect the point cloud data of multiple lanes. Subsidence is manifested in point cloud data as a gradual decrease in height over a large area, uniform point cloud density, but low overall height, which may be accompanied by slight fluctuations. Collapse is manifested in point cloud data as a large area of ​​missing point clouds, high edge point cloud density, and may be accompanied by a large amount of gravel and soil point clouds.

[0056] Step 1520: spatially cluster the first point cloud data based on the point cloud features to obtain spatial feature points.

[0057] It should be noted that due to the characteristics of laser scanning of laser radar to generate point clouds, point clouds are structured data. Taking 16-line laser radar as an example, the information of each point is saved with its coordinate value, the line beam it is in, the index in the current line beam and reflectivity, so the index of the adjacent points of each point can be obtained, that is, by traversing each point, the attribute characteristics of its adjacent points can be obtained. Based on these characteristics, it is possible to effectively judge whether the point is a negative obstacle.

[0058] Step 1530, obtaining the distance from the spatial feature point to the road reference plane to obtain a first distance.

[0059] The spatial feature point is the deepest point of the negative obstacle in the first point cloud data. The deepest distance of the negative obstacle from the road plane can be measured according to the first distance of the deepest point.

[0060] In one embodiment, determining the type of negative obstacle based on the second area and the first distance includes: traversing the boundary points of the second area to find the longest distance between the boundary points of the second area; and determining the type of negative obstacle according to the longest distance and the first distance.

[0061] This embodiment is used to find the longest boundary of the second area. According to the longest boundary of the second area, the longest width of the negative obstacle can be measured.

[0062] In one embodiment, the negative obstacles include: first type negative obstacles, second type negative obstacles and third type negative obstacles, the first type negative obstacles include potholes, the second type negative obstacles include: extra-wide road potholes and road subsidence, and the third type negative obstacles include: road collapse.

[0063] In one embodiment, the type of the negative obstacle is determined based on the longest distance and the first distance, including: when the first distance is less than the first preset distance and the longest distance is less than half the lane width, the negative obstacle is a first type of negative obstacle; when the first distance is less than the first preset distance and the longest distance is not less than half the lane width, the negative obstacle is a second type of negative obstacle; when the first distance is not less than the first preset distance; or, judging the road condition, when the road condition meets the preset road condition condition, the negative obstacle is a third type of negative obstacle.

[0064] Specifically, the first distance is 12-20cm, taking 15cm as an example. According to the reflection information of the road image and the lidar point cloud, if the width of the negative obstacle on the road is less than 1 / 2 of the width of the lane where the current vehicle is located and the distance between the deepest point of the negative obstacle and the horizontal plane of the road is less than 15cm, the negative obstacle is defined as a first-class negative obstacle. If the width of the negative obstacle on the road is greater than 1 / 2 of the width of the lane where the current vehicle is located and the distance between the negative obstacle and the horizontal plane of the road is less than 15cm, the negative obstacle is defined as a second-class negative obstacle. If the distance between the lane and the horizontal plane of the road is greater than 15cm or the road conditions are met, the negative obstacle is defined as a third-class negative obstacle.

[0065] In one embodiment, determining a road condition includes: obtaining a lane line structure and a lane line category based on a road image; determining whether the lane is a continuous lane based on the lane line structure and the lane line category; and when the determination result is a non-continuous lane, the road condition satisfies a preset road condition condition.

[0066] This embodiment clusters the lane line pixels of the road images of adjacent frames, and obtains the occurrence of all lane lines from left to right based on the clustering results. According to the occurrence of all lane lines, lane line attributes are assigned to the lane line pixels in each road image. For example, if the distance result includes four lanes, the lane lines are divided into the first left lane line, the second left lane line, the first right lane line and the second right lane line, and the classification results are stored in the lane line attributes in the form of labels. It should be noted that in the clustering process of the road images of adjacent frames of this embodiment, the next road image is matched with the clustering result of the previous frame until the road image stops being collected. This embodiment. By assigning lane line pixels to different lane instances, the lane lines of instance segmentation are finally obtained through clustering.

[0067] In the disclosed embodiment, the road image can be segmented using an image parsing model, a high-resolution neural network model (HRNet), a fully convolutional neural network model (FCN), etc. to obtain a plurality of lane line pixels. The lane line pixels of the road image of consecutive frames are tracked, and the lane line pixels of the same type are clustered to obtain the lane line types.

[0068] In this embodiment, the situation of non-continuous roads includes: the radar point cloud does not reflect depth information or the continuous road elements and road structures on the ground suddenly disappear (for example, continuous lane lines, continuous road edge lines, continuous curbs disappear, etc.).

[0069] In one embodiment, after determining the type of the negative obstacle, the method further includes: determining an avoidance attribute of the negative obstacle, where the avoidance attribute includes an avoidable attribute and an unavoidable attribute.

[0070] In one embodiment, Figure 4 A flowchart of a negative obstacle identification method provided in another embodiment of the present application is shown in FIG. Figure 4 As shown, the avoidance properties of negative obstacles are determined, including: Step 410: Determine the avoidance space of the vehicle based on the lane line image and the first area.

[0071] Step 420: Obtain the safe driving distance of the vehicle. When the avoidance space is greater than the safe driving distance of the vehicle, the avoidance attribute is an avoidable attribute.

[0072] Step 430: When the avoidance space is not greater than the safe driving distance of the vehicle, the avoidance attribute is an unavoidable attribute.

[0073] The safe driving distance of the vehicle can be determined based on the braking distance of the vehicle or based on the vehicle body distance. Determining the avoidance space includes: obtaining the shortest distance from the first area to the lane boundary line; calculating the actual shortest distance of the shortest distance based on the shortest distance and the lane pixel distance, and the actual shortest distance is the avoidance space of the lane.

[0074] Compared with the related art, the embodiments of the present application have at least the following advantages: This application can obtain the recognition results of negative obstacles in real time by collecting road images and point cloud data in real time during vehicle driving and analyzing the road images and point cloud data. At the same time, this application can obtain the correspondence between lane line images and lane lines by identifying the negative obstacle area and lane line images through road images, and determine the actual area of ​​the negative obstacle based on the correspondence; obtain the depth information of the negative obstacle through point cloud data, identify the two-dimensional features of the negative obstacle through images, and identify the three-dimensional features of the negative obstacle through point cloud data, thereby improving the accuracy of obtaining the types of negative obstacles.

[0075] Second, as Figure 5 As shown, the present application also provides a negative obstacle recognition device, including: a collection unit 510, a first recognition unit 520, a second recognition unit 530, a first determination unit 540, a second determination unit 550, and a result unit 560. Specifically, it includes: The acquisition unit 510 is used to acquire a road image of a lane when a vehicle is traveling in the lane.

[0076] The first recognition unit 520 is used to perform first image recognition on the road image to obtain a first area, where the first area includes at least one negative obstacle.

[0077] The second recognition unit 530 is used to perform second image recognition on the road image to obtain the lane pixel width of the lane line.

[0078] The first determination unit 540 is used to obtain a second area according to the first area and the lane pixel width, and the size of the second area is the actual size of the negative obstacle area. A second determining unit 550 is used to obtain first point cloud data of the negative obstacle, and determine a first distance according to the first point cloud data, where the first distance is used to represent the distance from the spatial position of the negative obstacle to the road reference plane; The result unit 560 is used to determine the type of the negative obstacle based on the second area and the first distance.

[0079] In one embodiment, first point cloud data of a negative obstacle is obtained, and a first distance is determined based on the first point cloud data, including: preprocessing the first point cloud data to determine point cloud features of the first point cloud data; spatially clustering the first point cloud data based on the point cloud features to obtain spatial feature points; and obtaining the distance from the spatial feature points to a road reference plane to obtain the first distance.

[0080] In one embodiment, determining the type of negative obstacle based on the second area and the first distance includes: traversing the boundary points of the second area to find the longest distance between the boundary points of the second area; and determining the type of negative obstacle according to the longest distance and the first distance.

[0081] In one embodiment, negative obstacles include: first type of negative obstacles, second type of negative obstacles and third type of negative obstacles, the first type of negative obstacles include potholes, the second type of negative obstacles include: extra-wide road potholes and road subsidence, and the third type of negative obstacles include: road collapse; according to the road condition, the longest distance and the first distance, the type of negative obstacle is determined, including: when the first distance is less than the first preset distance and the longest distance is less than half the lane width, the negative obstacle is the first type of negative obstacle; when the first distance is less than the first preset distance and the longest distance is not less than half the lane width, the negative obstacle is the second type of negative obstacle; when the first distance is not less than the first preset distance; or, the road condition is obtained and judged, and when the road condition meets the preset road condition condition, the negative obstacle is the third type of negative obstacle.

[0082] In one embodiment, determining a road condition includes: obtaining a lane line structure and a lane line category based on a road image; determining whether the lane is a continuous lane based on the lane line structure and the lane line category; and when the determination result is a non-continuous lane, the road condition satisfies a preset road condition condition.

[0083] In one embodiment, after determining the type of the negative obstacle, the method further includes: determining an avoidance attribute of the negative obstacle, where the avoidance attribute includes an avoidable attribute and an unavoidable attribute.

[0084] In one embodiment, determining the avoidance attribute of the negative obstacle includes: determining the avoidance space of the vehicle based on the lane line image and the first area; obtaining the safe driving distance of the vehicle, when the avoidance space is greater than the safe driving distance of the vehicle, the avoidance attribute is an avoidable attribute; when the avoidance space is not greater than the safe driving distance of the vehicle, the avoidance attribute is an unavoidable attribute.

[0085] Please refer to Figure 6 , Figure 6 It is a schematic diagram of an embodiment of a vehicle of the present application.

[0086] The vehicle 100 includes a memory 20, a processor 30, and a computer program 40 stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program 40, the steps in the above-mentioned negative obstacle identification method and related device embodiments are implemented, such as Figure 2 Steps 110 to 160 shown, or, Figure 3 Steps 1510 to 1530 shown, or, Figure 4 Steps 410 to 430 are shown.

[0087] Exemplarily, the computer program 40 can also be divided into one or more modules / units, one or more modules / units are stored in the memory 20 and executed by the processor 30. One or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 40 in the vehicle 100. For example, it can be divided into the collection unit 510, the first recognition unit 520, the second recognition unit 530, the first determination unit 540, the second determination unit 550, and the result unit 560 shown.

[0088] Those skilled in the art will appreciate that the schematic diagram is merely an example of the vehicle 100 and does not constitute a limitation on the vehicle 100 , which may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the vehicle 100 may also include input and output devices, network access devices, buses, etc.

[0089] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, a single-chip microcomputer, or the processor 30 may also be any conventional processor, etc.

[0090] The memory 20 can be used to store the computer program 40 and / or the module / unit. The processor 30 realizes various functions of the vehicle 100 by running or executing the computer program and / or the module / unit stored in the memory 20 and calling the data stored in the memory 20. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the vehicle 100, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0091] If the module / unit integrated in the vehicle 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

Claims

1. A negative obstacle recognition method, characterized in that: include: Collecting a road image of the lane when the vehicle is traveling in the lane; Performing first image recognition on the road image to obtain a first area, wherein the first area includes at least one negative obstacle; Performing a second image recognition on the road image to obtain a lane pixel width of a lane line; A second area is obtained according to the first area and the lane pixel width, wherein the size of the second area is the actual size of the negative obstacle area; Acquire first point cloud data of the negative obstacle, and determine a first distance according to the first point cloud data, where the first distance is used to represent the distance from the spatial position of the negative obstacle to a road reference plane; Based on the second area and the first distance, a type of the negative obstacle is determined.

2. The negative obstacle identification method according to claim 1, characterized in that: The acquiring first point cloud data of the negative obstacle and determining the first distance according to the first point cloud data comprises: Preprocessing the first point cloud data to determine point cloud features of the first point cloud data; Performing spatial clustering on the first point cloud data based on the point cloud features to obtain spatial feature points; The distance from the spatial feature point to the road reference plane is acquired to obtain the first distance.

3. The negative obstacle identification method according to claim 1, characterized in that: The determining the type of the negative obstacle based on the second area and the first distance includes: Traversing the boundary points of the second area, and finding the longest distance between the boundary points of the second area; The type of the negative obstacle is determined according to the longest distance and the first distance.

4. The negative obstacle identification method according to claim 3, characterized in that: The negative obstacles include: first-class negative obstacles, second-class negative obstacles and third-class negative obstacles. The first-class negative obstacles include potholes, the second-class negative obstacles include: over-wide road potholes and road settlement, and the third-class negative obstacles include: road collapse; The determining the type of the negative obstacle according to the longest distance and the first distance includes: When the first distance is less than the first preset distance and the longest distance is less than half of the lane width, the negative obstacle is the first type of negative obstacle; When the first distance is less than the first preset distance and the longest distance is not less than half the lane width, the negative obstacle is the second type of negative obstacle; When the first distance is not less than the first preset distance; or, The road condition is determined, and when the road condition satisfies a preset road condition, the negative obstacle is the third type of negative obstacle.

5. The negative obstacle identification method according to claim 4, characterized in that: The determining of road conditions includes: Acquire a lane line structure and a lane line category according to the road image; Determining whether the lane is a continuous lane according to the lane line structure and the lane line category; In the case that the lane is a non-continuous lane, it is determined that the road condition meets the preset road condition.

6. The negative obstacle identification method according to claim 1, characterized in that: After determining the type of the negative obstacle, the method further includes: determining an avoidance attribute of the negative obstacle, wherein the avoidance attribute includes an avoidable attribute and an unavoidable attribute.

7. The negative obstacle identification method according to claim 6, characterized in that: The determining the avoidance attribute of the negative obstacle includes: determining an avoidance space for the vehicle according to the road image and the first area; Acquire a safe driving distance of the vehicle, and when the avoidance space is greater than the safe driving distance of the vehicle, determine that the avoidance attribute is the avoidable attribute; When the avoidance space is not greater than the safe driving distance of the vehicle, the avoidance attribute is determined to be the unavoidable attribute.

8. A negative obstacle recognition device, characterized in that: include: A collection unit, used for collecting a road image of a lane when a vehicle travels in the lane; A first recognition unit, configured to perform first image recognition on the road image to obtain a first area, wherein the first area includes at least one negative obstacle; A second recognition unit is used to perform second image recognition on the road image to obtain a lane pixel width of the lane line; A first determining unit, configured to obtain the second area according to the first area and the lane pixel width, wherein the size of the second area is an actual size of the negative obstacle area; A second determining unit, configured to obtain first point cloud data of the negative obstacle, and determine a first distance according to the first point cloud data, wherein the first distance is used to represent a distance from a spatial position of the negative obstacle to a road reference plane; A result unit is used to determine the type of the negative obstacle based on the second area and the first distance.

9. A vehicle, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the vehicle executes the negative obstacle recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a vehicle, the vehicle is caused to perform the negative obstacle identification method according to any one of claims 1 to 7.

Images