A high-precision map-based long-distance cone bucket detection and ranging correction method

Abstract

The application relates to a long-distance cone cylinder detection and distance measurement correction method based on a high-precision map. First, the position, width, height, and inverted ground state of the cone cylinder and the position information of lane lines on an image are obtained. The lower edge, width, and height information of the cone cylinder and the position information of the lane lines on the image are used to obtain the distance measurement information of the cone cylinder 200 meters away and the position relationship relative to the lane lines. The relative position relationship between the cone cylinder and the lane and the real position information of the lane lines on the map are used to correct the lateral distance of the cone cylinder, so that more accurate lateral position information of the cone cylinder is obtained. The application is based on visual deep learning target detection and realizes cone cylinder distance measurement correction in combination with a high-precision map, can be applied to the development of a high-speed automatic driving system, promotes safe and reliable lane changing and detouring of an intelligent truck in a high-speed construction area, and improves operation efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent driving perception, and relates to a target detection method for intelligent truck high-speed driving, in particular to a long-distance cone bucket detection and distance measurement correction method based on a high-precision map. BACKGROUND

[0002] With the development of automatic driving technology, the demand for automatic lane changing and bypassing in high-speed scenarios is increasingly urgent. The construction area under the highway is a typical scenario for automatic driving. Due to the high speed of the ego vehicle, the cone bucket target is very small, and the position of the cone bucket in the construction area needs to be identified in advance to meet the safe lane changing distance.

[0003] There are mainly two methods for detecting cone buckets in construction areas: one is to provide target detection and passable space results through a laser radar. This kind of technology can provide accurate and reliable position information, but is limited by the sparseness of point cloud density at a long distance, and it is difficult to cover a detection distance of more than 100 meters. The second is to realize target detection and passable space detection through a visual camera and traditional distance measurement. This kind of technology can provide a long-distance detection range, but cannot meet the long-distance distance measurement accuracy requirement.

[0004] The lateral distance measurement error of the single distance measurement scheme of the visual detection cone bucket cannot meet the requirement. For example, if the yaw angle deviation of the external parameter calibration is 1°, a lateral error of 1.75 m will be generated at 100 m. If the lateral distance is 5 m at 100 m, the deviation of one BBOX pixel is about 0.21 m. If height distance measurement is used, the distance measurement error caused by the non-standard cone bucket not conforming to the preset width and height will be generated. In addition, the pitch angle error + ground height error will also cause certain error when the lower edge is measured.

[0005] With the continuous development of deep learning and high-precision map technology in recent years, the perception scheme based on deep learning target detection technology combined with high-precision map is increasingly valued, but so far no scheme has been found to apply deep learning and high-precision map technology to small target detection and distance measurement optimization of cone buckets and the like in high-speed scenarios in automatic driving technology. SUMMARY

[0006] The purpose of the present application is to solve the problems existing in the prior art, and to provide a long-distance cone bucket detection and distance measurement correction method based on a high-precision map. This method reconstructs the real environment, uses part of the prior information of the high-precision map, realizes long-distance detection and accurate distance measurement of the cone bucket in the high-speed scenario, and can provide relatively accurate positions of the cone bucket beyond 200 meters, effectively solving the problem of long-distance distance measurement accuracy in perception.

[0007] In order to achieve the above purpose, the present application provides a long-distance cone bucket detection and distance measurement correction method based on a high-precision map, comprising the following steps:

[0008] Step 1, the front cone barrel target and lane line are detected simultaneously by using visual 30-degree narrow-angle and 120-degree wide-angle cameras to meet the identification of the cone barrel target and lane line 200 meters away, and then the position, width, height, and ground state of the cone barrel and the position information of the lane line on the image are obtained;

[0009] Step 2, the cone barrel ranging information 200 meters away and the position relationship relative to the lane line are obtained by using a plurality of ranging methods for fusion according to the lower edge, width, height information of the cone barrel and the position information of the lane line on the image;

[0010] Step 3, the lateral distance of the cone barrel is corrected by using the relative position relationship between the cone barrel and the lane and the real position information of the lane line on the map, and more accurate lateral position information of the cone barrel is obtained.

[0011] The application is based on visual deep learning target detection, and realizes cone barrel ranging correction combined with a high-precision map, and can be applied to the development of a high-speed automatic driving system, promotes safe and reliable lane changing of an intelligent truck in a high-speed construction area, and improves operation efficiency.

[0012] The application further adopts the following technical scheme:

[0013] In step 1, the visual 30-degree narrow-angle and 120-degree wide-angle cameras collect the front image and input the cone barrel / lane line detection model of the deep learning network into the network, learn the detection result of the cone barrel / lane line on the image, and obtain stable continuous frame target information through tracking post-processing;

[0014] The cone barrel detection model selects Faster-RCNN as a basic framework to ensure the recall rate and accuracy of small targets and difficult targets, and outputs the box, width, height, and ground state information of the cone barrel, and the Track-By-Detection method is used for cone barrel tracking, which takes into account the tracking accuracy and operation efficiency;

[0015] The lane line detection model is based on an Unet network structure diagram, and outputs the segmentation diagram, label, and instance diagram of the image, and then performs coordinate conversion, lane line clustering, and denoising point processing, and performs lane line filtering, lane line tracking, and lane ID matching processing on the lane line, and then fits the denoising point processing result and the lane ID matching processing result to output the stable lane line position information in the image / body coordinate system.

[0016] In step 2, the cone barrel and lane line results obtained in step 1 are used for multi-ranging fusion, and the specific steps include:

[0017] a. Ground point ranging - a short distance range less than 30 meters is used, and the coordinates of the cone barrel in the ego vehicle system can be calculated through the ground model, camera internal and external parameters, and the midpoint of the bottom edge of the cone barrel (ground point); this method has small error within 30m, and can better adapt to the ranging of cone barrels and inverted cone barrels;

[0018] b. Prior information-based ranging (width and height - medium and long distances greater than 30m are used, according to the test scene cone barrel size standard of 0.4*0.4*0.8m, the distance is calculated using prior information and box width and height and camera internal and external parameters, and the inverted state of the cone barrel is needed to adapt to the ranging of cone barrels and inverted cone barrels;

[0019] c. Lane line assisted ranging - when the lane line is effective, lane line assisted ranging is increased, the depth information (longitudinal distance) of the cone barrel is calculated first through the ground point, the pixel distance on the image of the cone barrel and the lane line point with the same depth is used, and the lane line width can be calculated according to the lane line and is assumed to be constant within a certain distance, and finally the lateral offset of the cone barrel and the lane line is calculated, so as to output the distance of the cone barrel and the relative position relationship of the lane line.

[0020] In step 3, since L4 automatic driving usually has the characteristics of high-precision map and real-time positioning; the depth information (longitudinal distance) of the cone barrel is calculated first through the ground point ranging, the lane line position and width information with the same longitudinal distance in the high-precision map are obtained through the MAP-SDK interface according to the ego vehicle positioning information, and then the lateral offset is obtained by matching the image lane line and the map lane line, and the final distance of the cone barrel is output by superimposing the lateral offset on the map lane line. This ranging scheme helps to eliminate the jitter error and poor accuracy of lane line perception; at the same time, a variety of ranging fusion schemes are adopted to avoid the ranging deviation of a single ranging scheme.

[0021] The application discloses a cone barrel ranging correction method based on visual depth learning target detection and combined with a high-precision map, first, long-distance detection and tracking beyond 200 meters is realized according to multi-view sensor detection and fusion technology; then, a multi-ranging method fusion scheme based on cone barrel box, width, height, inverted state information and lane line perception position is used to realize relatively stable ranging accuracy of the cone barrel; finally, the cone barrel lateral distance is corrected based on the high-precision map lane line information, and the ranging accuracy is further improved, meeting the lane changing and detouring needs of planning control in high-speed construction areas. BRIEF DESCRIPTION OF DRAWINGS

[0022] The application will be further described below with reference to the drawings.

[0023] Figure 1 The figure is the architecture of the cone barrel target detection network in the application.

[0024] Figure 2 Flow chart for cone barrel target tracking in the present application.

[0025] Figure 3 Schematic diagram of the visual ranging scheme in the present application.

[0026] Figure 4 Architectural diagram of the lane line segmentation network in the present application.

[0027] Figure 5 Schematic diagram of the lane line post-processing output in the present application.

[0028] Figure 6 Flow chart for cone barrel correction based on multi-ranging scheme fusion and high-precision map lane line in the present application.

[0029] Figure 7 Schematic diagram of the detection result in the present application.

[0030] Figure 8 Schematic diagram of the cone barrel ranging test data in the present application. Embodiment

[0031] Embodiment

[0032] As shown in Figure 6 , a long-distance cone cylinder detection and ranging correction method based on a high-precision map mainly includes target detection, cone barrel ranging fusion, and high-precision correction of the lateral position of the cone barrel, and the specific steps are as follows:

[0033] Step 1: A 30-degree narrow-angle and a 120-degree wide-angle camera are used to simultaneously detect the front cone barrel target and the lane line to meet the identification of the cone barrel target and the lane line 200 meters away, and then the position, width, height, and ground state of the cone barrel and the position information of the lane line on the image are obtained.

[0034] The 30-degree narrow-angle and 120-degree wide-angle camera collects the front image and inputs it into the cone barrel / lane line detection model of the deep learning network, learns the detection result of the cone barrel / lane line on the image, and obtains stable continuous frame target information through tracking post-processing.

[0035] As shown in Figure 1 and Figure 4 , the cone barrel detection module selects Faster-RCNN as the basic framework to ensure the recall rate and accuracy of small targets and difficult targets, and outputs the box, width, height, and ground state information of the cone barrel. The cone barrel tracking uses the Track-By-Detection method, which takes into account the tracking accuracy and computational efficiency.

[0036] The lane line detection module outputs a segmentation map, a label and an instance map of the image based on an Unet network structure diagram, and then performs coordinate conversion, lane line clustering, and noise point removal. Meanwhile, lane line filtering, lane line tracking, and lane ID matching processing are performed on the lane line. Then, the results after noise point removal and lane ID matching are fitted to output stable lane line position information in the image / body coordinate system. Both perspectives use multi-ROI image simultaneous inference, which can realize 8 pixel minimum image size detection to meet the identification of cone and lane line targets beyond 200 meters.

[0037] As Figure 2 For the cone detection target, the Track-By-Detection method is used for tracking to avoid the problem of target loss caused by unstable detection. The matching strategy of the tracking module includes the following strategy matching:

[0038] 1) Similar categories: only when the detection results of two frames are the same type of object, matching is performed;

[0039] 2) Object center point proximity principle: the center deviation of the detection box of the same object in two frames cannot be greater than the object size;

[0040] 3) Size proximity principle: the detection box of the same object in two frames should not have a large size difference;

[0041] 4) Detection box matching principle: the overlapping area of the detection box of the same object in two frames must be large enough;

[0042] As Figure 5 shown, the lane line tracking post-processing mainly includes the following steps:

[0043] 1) Coordinate conversion: coordinate conversion is performed on the segmentation pixels in the model result, and conditional filtering is performed;

[0044] 2) Lane clustering: clustering is performed on the instance in the model result;

[0045] 3) Noise point removal: the lane candidate points after clustering may have abnormal points, and noise filtering is performed according to the following figure;

[0046] 4) Fitting: 3rd order curve fitting is used, and the basic principle is the least squares method;

[0047] 5) Lane line id matching: lane line id matching uses a segmented method, which sets reference key positions along the longitudinal distance of the lane line region and sets the confidence of the corresponding key positions;

[0048] 6) Lane line tracking: Kalman filtering is used for lane tracking;

[0049] 7) Lane filtering: Lane filtering is mainly used for lane line selection based on prior knowledge of lane lines. For example, the width of two lane lines in the national standard is 3.3 meters to 3.75 meters, the minimum is 2.2 meters, and the maximum is not more than 5 meters.

[0050] Step 2: Using the lower edge, width, height information of the cone barrel and the position information of the lane line on the image, a variety of ranging methods are fused to obtain the ranging information of the cone barrel beyond 200 meters and the positional relationship relative to the lane line. According to the results of the cone barrel and the lane line, multiple ranging fusion is performed, and the specific steps include:

[0051] a. Ground point ranging - used in a short distance range less than 30 meters, the coordinates of the cone barrel in the ego vehicle system can be calculated through the ground model, camera internal and external parameters, and the midpoint of the bottom edge of the cone barrel (ground point). This method has small error within 30 meters and can better adapt to the ranging of cone barrels and inverted cone barrels;

[0052] b. Ranging based on prior information (width and height) - used in a medium and long distance range greater than 30 meters, according to the test scene cone barrel size standard of 0.4*0.4*0.8m, the distance is calculated using prior information, box width and height, and camera internal and external parameters. The inverted state of the cone barrel is needed to adapt to the ranging of cone barrels and inverted cone barrels;

[0053] c. Lane line assisted ranging - when the lane line is effective, lane line assisted ranging is increased. First, the depth information (longitudinal distance) of the cone barrel is calculated by the ground point, the pixel distance on the image of the cone barrel and the lane line point with the same depth is used, and the lane line width is calculated according to the lane line and assumed to be constant within a certain distance. Finally, the lateral offset of the cone barrel and the lane line is calculated to output the distance of the cone barrel and the relative positional relationship of the lane line.

[0054] As shown in Figure 3 , the visual ranging scheme mainly relies on geometric operations:

[0055] 1) 3D projection ranging (target width and length as prior knowledge): suitable for vehicle ranging, as the cone barrel target is a 2D-BBOX without obvious length and width boundaries, this ranging scheme is not used;

[0056] 2) Height ranging (target height as prior knowledge): by presetting the prior height information of the target, the distance of the target is calculated based on the height of the target box on the image; it is difficult to cover scenes where the prior height and the actual height of the target are biased;

[0057] 3) Width ranging (target width as prior knowledge): same as height ranging;

[0058] 4) Lower edge ranging: By assuming the ground level, the distance of the target is calculated by the bottom edge of the target box on the image; it is difficult to cover the priori ground that is not level and acceleration and deceleration scenarios;

[0059] 5) Lane line assisted ranging: First, the depth information (longitudinal distance) of the target is calculated by the touch point, the pixel distance on the image of the target and the same depth lane line point is used, and the lane line width can be calculated according to the lane line and is assumed to be constant within a certain distance. Finally, the lateral offset between the cone barrel and the lane line is calculated to output the distance of the cone barrel and the relative position relationship of the lane line.

[0060] Step 3, using the relative position relationship between the cone barrel and the lane, and the real position information of the lane line on the map, the lateral distance of the cone barrel is corrected to obtain more accurate lateral position information of the cone barrel. Since L4 autonomous driving usually has the characteristics of high-precision map and real-time positioning, the depth information (i.e. longitudinal distance) of the cone barrel is calculated by the touch point ranging, and the lateral coordinate x and the longitudinal coordinate y of the cone barrel in the vehicle coordinate system can be obtained from the touch point distance. The lateral coordinate x represents the lateral distance of the cone barrel, and the longitudinal coordinate y represents the longitudinal distance of the cone barrel. Then, the same longitudinal distance lane line is selected according to the longitudinal distance of the cone barrel, and the position and width information of the same longitudinal distance lane line is obtained. Then, the position and width information of the same longitudinal distance lane line on the high-precision map is obtained through the MAP-SDK interface according to the self-positioning information. Then, the lateral offset is obtained by matching the image lane line and the map lane line. Since the lateral distance is more sensitive in autonomous driving, the longitudinal accuracy requirement is not so high, so the final output of the cone barrel is obtained by superimposing the lateral offset on the map lane line. When the lateral distance of the cone barrel and the lane line on the image is determined, the relative lateral distance of the cone barrel relative to the lane line in the vehicle coordinate system can also be calculated. The map provides more accurate lane line information, and the relative lateral distance calculated under the condition of longitudinal distance can obtain more accurate lateral distance than the original one, which is output as the final lateral distance of the cone barrel. This ranging scheme helps to eliminate the jitter error of lane line perception and the problem of poor accuracy at a distance.

[0061] Figure 7 The intermediate process result of the overall debugging represents the detection and tracking results of the cone barrel and the lane line, as well as the relative position relationship between the cone barrel and the lane line. The right side of the image represents the final cone barrel ranging output result (the blue line represents the perceived lane line, the green line represents the center line of the high-precision map lane, the red circle represents the cone barrel that does not invade the lane, and the blue circle represents the cone barrel that does not invade the cone barrel); Figure 8 The distance cone barrel correction test data shows that the ranging correction method combined with high-precision map can effectively improve the lateral ranging accuracy of long-distance targets.

[0062] It should be noted that the execution sequence of the above steps is determined by its inherent logic and function, and the execution sequence can complete the desired results of the technical solutions of the patent disclosure as long as it can complete the desired results of the technical solutions of the patent disclosure, and should not cause any limitation and constraint to the implementation of the application and the embodiments. In addition to the above embodiments, the application can also have other implementation manners. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope of the application.

Claims

1. A high-precision map-based long-distance cone detection and ranging correction method, characterized in that, The method comprises the following steps: Step 1: Simultaneously detect the front cone barrel target and lane line by using a visual 30-degree narrow-angle camera and a 120-degree wide-angle camera to meet the identification of the cone barrel target and lane line 200 meters away, and then obtain the position, width, height, and ground state of the cone barrel and the position information of the lane line on the image; Step 2: Obtain the distance measurement information of the cone barrel 200 meters away and the position relationship relative to the lane line by using the lower edge, width, and height information of the cone barrel and the position information of the lane line on the image and fusing multiple distance measurement methods; The specific steps of the multi-distance fusion based on the cone barrel and lane line results obtained in step 1 include: a. Ground point distance measurement: used in a short distance range less than 30 meters, the coordinates of the cone barrel in the ego vehicle system can be calculated through the ground model, camera internal and external parameters, and the midpoint of the bottom edge of the cone barrel box; b. Distance measurement based on prior information: used in a medium and long distance range greater than 30 meters, the distance is calculated by using the prior information, the width and height of the cone barrel box, and the camera internal and external parameters, and the ground state of the cone barrel is needed to adapt the distance measurement of the cone barrel and the fallen cone barrel; c. Lane line assisted distance measurement: when the lane line is valid, the lane line assisted distance measurement is increased, the depth information of the cone barrel is calculated by the ground point, the pixel distance on the image of the cone barrel and the lane line point at the same depth is used, the lane line width is calculated according to the lane line, and it is assumed that the lane line width is constant within a certain distance, and finally the lateral offset of the cone barrel and the lane line is calculated to output the distance of the cone barrel and the relative position relationship of the lane line; Step 3: Correct the lateral distance of the cone barrel by using the relative position relationship between the cone barrel and the lane line and the real position information of the lane line in the map, and obtain more accurate lateral position information of the cone barrel; The depth information of the cone barrel is calculated by the ground point distance measurement, the position and width information of the lane line at the same longitudinal distance in the high-precision map are obtained by the MAP-SDK interface according to the ego vehicle positioning information, the lateral offset is obtained by matching the image lane line and the map lane line, the final distance of the cone barrel is output by superimposing the lateral offset on the map lane line.

2. The long-range cone detection and ranging correction method based on high-precision maps of claim 1, wherein, In step 1, the front image is collected by the visual 30-degree narrow-angle camera and the 120-degree wide-angle camera and input into the cone barrel / lane line detection model of the deep learning network, the detection results of the cone barrel / lane line on the image are learned, and stable continuous frame target information is obtained through tracking post-processing; The cone barrel detection model selects Faster-RCNN as the basic framework, and outputs the box, width, height, and ground state information of the cone barrel, and the Track-By-Detection method is used for cone barrel tracking, which considers the tracking accuracy and operation efficiency. The lane line detection model is based on an Unet network structure diagram, and respectively outputs a segmentation diagram, a label and an instance diagram of an image, and then after coordinate conversion, lane line clustering, and de-noise point processing, lane line filtering, lane line tracking, and lane ID matching processing are performed on the lane line, and then the de-noise point processing result and the lane ID matching processing result are fitted to output stable lane line position information in an image / vehicle coordinate system.

Images