An autonomous vehicle pedestrian perception and digital twin method

Abstract

The application relates to the technical field of unmanned driving and discloses a method for pedestrian sensing and digital twinning of an unmanned vehicle, which accurately obtains measurement data by correcting visual measurement results through laser radar; a motion model of pedestrians relative to the vehicle is established through a grey prediction theory; the detection results and grey model parameters are transmitted to a cloud digital twinning system to restore real-time data; the results of machine vision measurement are corrected through laser radar; the efficient feature recognition of machine vision and the accurate space measurement characteristics of laser radar are fully utilized; and the accuracy and real-time performance of position detection of pedestrians by the unmanned vehicle are improved. The uncontrollable time delay in the data transmission process is reduced through a mathematical method; the on-site measurement results can be restored in real time in the digital twinning system; and the method has high precision and reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned driving, in particular to a pedestrian perception and digital twin method for unmanned vehicle. BACKGROUND

[0002] Although the vehicle unmanned driving technology has made great progress in recent years, there is still a long way to go to large-scale practical road application. One of the main reasons is that the system safety needs to be improved. The prerequisite for ensuring safety is to have a precise and real-time environment target detection mechanism. The random and subjective motion of pedestrians in the environment will cause serious interference to unmanned driving.

[0003] At present, in the field of unmanned driving, the detection of roadside pedestrians mainly relies on laser radar or machine vision. The laser radar has relatively reliable precision in detecting pedestrians, but it has less data features, and it is difficult to accurately identify pedestrians, and it is easy to produce false identification. Through machine vision, more feature information can be obtained, but the position detection accuracy is low. The current conventional method is to detect the current position of the pedestrian, which cannot be effectively predicted. When the detection result is uploaded to the cloud digital twin system, due to the system transmission delay, the data produces a time lag of uncertainty. This also leads to the digital twin system unable to work effectively.

[0004] In order to improve the accuracy and real-time performance of the unmanned vehicle in detecting the position of the pedestrian, and to minimize the time lag problem of the digital twin system, it is urgent to provide a pedestrian perception and digital twin method for unmanned vehicle. SUMMARY

[0005] Technical problems to be solved

[0006] In view of the shortcomings of the prior art, the present application provides a pedestrian perception and digital twin method for unmanned vehicle, which has the advantages of improving the accuracy and real-time performance of the unmanned vehicle in detecting the position of the pedestrian, and minimizing the time lag problem of the digital twin system, and solves the problem of time lag in the digital twin system.

[0007] (II) Technical solutions

[0008] To achieve the above purpose, the present application provides the following technical solutions: a pedestrian perception and digital twin method for unmanned vehicle for correcting machine vision measurement results in unmanned driving, comprising the following steps:

[0009] S1: correcting the vision measurement result by laser radar to accurately obtain the measurement data;

[0010] S2: establishing a motion model of the pedestrian relative to the vehicle by gray prediction theory;

[0011] S3: the detection result and the gray model parameter are transmitted to a cloud digital twin system, and real-time data is restored;

[0012] Preferably, the specific steps of S1 are as follows:

[0013] S1.1: a stereo vision camera is horizontally installed at the middle position of the vehicle head, the center position is the origin O of a spatial polar coordinate system, the shooting direction is the front direction, according to the right-hand rule, the spatial polar coordinate system is established, the horizontal right direction is the positive direction of the polar axis x, and the equatorial plane counterclockwise direction is the positive direction of the polar angle;

[0014] S1.2: a laser radar is symmetrically horizontally installed at the left and right sides of the vehicle head at the same horizontal height as the stereo vision camera, and the centers of the laser radars are L and R respectively;

[0015] S1.3: the stereo camera collects the front image, a mode recognition algorithm is used to recognize the coordinates of the roadside pedestrian in the image coordinate system, and the centroid P0 is calculated;

[0016] S1.4: the image coordinates of P0 are converted into spatial polar coordinates by the stereo measurement function of the camera, and the equatorial plane is projected to obtain P0(r0, θ0), wherein r0 is the polar radius, representing the distance OP0 of the pedestrian to the camera, and θ0 is the polar angle α;

[0017] S1.5: according to the geometric relationship, the size of the polar angle β of the pedestrian in the left laser radar is

[0018]

[0019] In the formula, w is the distance between the geometric centers of the left and right laser radars;

[0020] S1.6: the distance data in the direction of angle β is collected from the left laser radar data, and the laser measurement value of LP0 is obtained;

[0021] S1.7: according to the geometric relationship, the size of the polar angle γ of the pedestrian in the right laser radar is

[0022] ;

[0023] S1.8: the distance data in the direction of angle γ is collected from the right laser radar data, and the laser measurement value of RP0 is obtained;

[0024] S1.9: the laser radar measurement values of the left and right sides are arithmetically averaged, and the measurement result P0(r0, θ0) of the stereo vision is corrected through the laser radar, that is,

[0025]

[0026]

[0027] wherein

[0028]

[0029] .

[0030] Preferably, the specific steps of S2 are as follows:

[0031] S2.1: successively obtain the positions of the pedestrian relative to the vehicle at different times P1(r1, θ1), P2(r2, θ2), P3(r3, θ3) … Pn(rn, θn) according to the method of S1.1-S1.9 n (r n , θ n ), wherein the sampling interval is ;

[0032] S2.2: respectively perform one accumulation on the polar radius and the polar angle to obtain the polar radius 1-AGO sequence Q (1) and the polar angle 1-AGO sequence A (1)

[0033]

[0034]

[0035] In the formula,

[0036]

[0037] ;

[0038] S2.3: according to the least square method, obtain the values of the GM(1, 1) first-order differential equation parameters a r , b r of the polar radius

[0039]

[0040] wherein,

[0041]

[0042] According to the least square method, obtain the values of the GM(1, 1) first-order differential equation parameters aθ, bθ of the polar angle

[0043]

[0044] wherein,

[0045] .

[0046] Preferably, the specific steps of S3 are as follows:

[0047] S3.1: Send the pedestrian position P1(r1, θ1), the polar radius first-order differential equation parameters a r , b r , the polar angle first-order differential equation parameters a θ , b θ , the time stamp, the sampling interval together to the cloud digital twin system in the modeling process;

[0048] S3.2: After the cloud digital twin system receives the data of S3.1 at n+1 time, the time difference is calculated by the cloud timestamp t, that is, the current time pedestrian position P n+1 (r n+1 , θ n+1 )

[0049] Among them

[0050]

[0051] .

[0052] Preferably, the angle resolution of the laser radar in S1.2 is 0.2°.

[0053] Preferably, the pattern recognition algorithm in S1.3 adopts YOLO algorithm.

[0054] Preferably, the sampling data amount in S2.1 is between 10-20, and the sampling interval is 20-30ms.

[0055] (Three) beneficial effects

[0056] Compared with the prior art, the present application provides an unmanned vehicle pedestrian perception and digital twin method, which has the following beneficial effects:

[0057] The unmanned vehicle pedestrian perception and digital twin method corrects the results of machine vision measurement by laser radar, fully utilizes the efficient feature recognition of machine vision and the accurate space measurement characteristics of laser radar, and improves the accuracy and real-time performance of the unmanned vehicle in detecting the position of pedestrians. The mathematical method reduces the uncontrollable time delay in the data transmission process, which can restore the on-site measurement results in real time in the digital twin system, and has high precision and reliability. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is the system working principle diagram of the present application; DETAILED DESCRIPTION

[0059] ​With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0060] Embodiments

[0061] Reference Figure 1 A method for pedestrian perception and digital twin of an unmanned vehicle, comprising the following steps:

[0062] (I) Correct the visual measurement result by laser radar to accurately obtain the measurement data:

[0063] (1) A stereo vision camera is horizontally installed at the middle position of the vehicle head, with the center position as the origin O of a spatial polar coordinate system and the shooting direction as the front direction. According to the right-hand rule, a spatial polar coordinate system is established, with the horizontal right direction as the positive direction of the polar axis x and the equatorial plane counterclockwise direction as the positive direction of the polar angle. Since only the projection is considered, the vertical direction of the spatial polar coordinate system can not be considered for simplifying the calculation.

[0064] (2) Laser radars are symmetrically and horizontally installed at the left and right sides of the vehicle head at the same horizontal height as the stereo vision camera, with the centers of the laser radars being L and R, respectively. The number of laser lines of the laser radars is not required, but the angle resolution should be high, preferably about 0.2°.

[0065] (3) The stereo vision camera collects the front image, identifies the coordinates of the roadside pedestrian in the image coordinate system through a pattern recognition algorithm, and calculates the centroid P0. The pattern recognition algorithm only needs to be able to identify pedestrians, and should have rapidity, which can be met by the common YOLO algorithm.

[0066] (4) The image coordinates of P0 are converted into spatial polar coordinates by the stereo measurement function of the camera, and projected on the equatorial plane to obtain P0(r0, θ0), where r0 is the polar radius, representing the distance OP0 from the pedestrian to the camera, and θ0 is the polar angle α.

[0067] (5) According to the geometric relationship, the polar angle β of the pedestrian in the left laser radar can be obtained as

[0068]

[0069] where w is the distance between the geometric centers of the left and right laser radars.

[0070] (6) From the left laser radar data, the distance data in the direction of angle β is collected, and the laser measurement value of LP0 is obtained. In the dynamic case, the accuracy of laser radar measurement is millimeter level, which is much higher than the spatial measurement result of the camera.

[0071] (7) According to the geometric relationship, the size of the polar angle γ of the pedestrian in the right laser radar is

[0072]

[0073] (8) From the right laser radar data, the distance data in the direction of angle γ is collected, and the laser measurement value of RP0 is obtained.

[0074] (9) The laser radar measurement values on the left and right sides are arithmetically averaged, and the measurement result P0(r0, θ0) of the stereo vision is corrected through the laser radar, that is,

[0075]

[0076]

[0077] Among them,

[0078]

[0079]

[0080] Through the superposition of the data of the two sides of the laser radar, the measurement result has high accuracy. This part of the operation, that is, fully utilizes the image recognition function of the stereo vision camera, and fully plays the accurate measurement function of the laser radar, realizes the complementary function of the two.

[0081] (II) A motion model of the pedestrian relative to the vehicle is established by the gray prediction theory:

[0082] (10) According to the method of steps 1) ~ 9), the positions of the pedestrian relative to the vehicle at different times P1(r1, θ1), P2(r2, θ2), P3(r3, θ3) … P n n n ) are obtained successively, wherein the sampling interval is It is generally recommended that the sampling data amount is between 10-20, and the sampling interval is 20-30 ms. Too much or too little sampling data will have adverse effects on the gray prediction result.

[0083] (11) In order to reduce the influence of interference data on effective data, the polar radius and polar angle are accumulated once respectively, and the polar radius 1-AGO sequence is Q (1) , and the polar angle 1-AGO sequence is A (1) ​​

[0084]

[0085]

[0086] wherein

[0087]

[0088]

[0089] (12) The values of the GM (1, 1) first-order differential equation parameters a r , b r of the polar radius are obtained according to the least square method

[0090]

[0091] wherein,

[0092]

[0093] The values of the GM (1, 1) first-order differential equation parameters a θ , b θ of the polar angle are obtained according to the least square method

[0094]

[0095] wherein,

[0096]

[0097] This part of the operation involves the operation of matrix conversion and matrix inversion. Since high real-time performance is required, computer programming is recommended for solving.

[0098] (Three) Send the detection results and the gray model parameters to the cloud digital twin system to restore real-time data:

[0099] (13) Send the pedestrian position P1(r1, θ1) in the modeling process, the polar radius first-order differential equation parameters a r , b r , the polar angle first-order differential equation parameters a θ , b θ , the time stamp, and the sampling interval to the cloud digital twin system.

[0100] (14) After receiving the data in step 13) at time n+1, the cloud digital twin system calculates the time difference as t through the cloud timestamp, and can calculate the current pedestrian position P n+1 (r n+1 , θn+1 )

[0101] wherein

[0102]

[0103]

[0104] Wherein, in the implementation process, the cloud time system must be synchronized with the time system of the unmanned vehicle, otherwise the time stamp loses its meaning and cannot effectively calculate the current time position.

[0105] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely divergences of the principles and spirit of the application and that numerous modifications, changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. An autonomous vehicle pedestrian perception and digital twin method for autonomous in- repair machine vision measurement correction, characterized in that, It comprises the following steps: S1: correcting the visual measurement result by laser radar to accurately obtain measurement data; S2: establishing a motion model of pedestrians relative to vehicles by gray prediction theory; S3: restoring real-time data by uploading the detection result and gray model parameters to a cloud digital twin system; The specific steps of S1 are as follows: S1.1: horizontally installing a stereo vision camera at the middle position of the vehicle head, with the center position as the origin O of a spatial polar coordinate system and the shooting direction being the front direction; according to the right-hand rule, a spatial polar coordinate system is established, with the horizontal right direction being the positive direction of the polar axis x and the equatorial plane counterclockwise direction being the positive direction of the polar angle; S1.2: horizontally installing laser radars at the same height as the stereo vision camera on the left and right sides of the vehicle head respectively, with the centers of the laser radars being L and R respectively; S1.3: collecting the front image by the stereo camera, identifying the coordinates of the roadside pedestrians in the image coordinate system by a pattern recognition algorithm, and calculating the centroid P0; S1.4: converting the image coordinates of P0 into spatial polar coordinates by the stereo measurement function of the camera, and projecting on the equatorial plane to obtain P0(r0, θ0), wherein r0 is the polar radius, representing the distance OP0 from the pedestrian to the camera, and θ0 is the polar angle α; S1.5: according to the geometric relationship, the polar angle β of the pedestrian in the left laser radar can be obtained as wherein w is the distance between the geometric centers of the left and right laser radars; S1.6: collecting the distance data in the direction of angle β from the left laser radar data, and obtaining the laser measurement value of LP0; S1.7: according to the geometric relationship, the polar angle γ of the pedestrian in the right laser radar can be obtained as ； S1.8: collecting the distance data in the direction of angle γ from the right laser radar data, and obtaining the laser measurement value of RP0; S1.9: performing arithmetic averaging on the laser radar measurement values on the left and right sides, and correcting the measurement result P0(r0, θ0) of the stereo vision by the laser radar, i.e. wherein ； The specific steps of S2 are as follows: S2.1: successively obtain the positions of the pedestrian relative to the vehicle P1(r1, θ1), P2(r2, θ2), P3(r3, θ3)... Pn(rn, θn) at different times according to the methods of S1.1-S1.9 n (r n , θ n n), wherein the sampling interval is ; S2.2: One accumulation is carried out on the polar radius and polar angle respectively, and the polar radius 1-AGO sequence is Q (1) , and the polar angle 1-AGO sequence is A (1) wherein ； S2.3: Determine the parameter a of the first-order differential equation of the polar radius GM(1,1) using the least squares method. r b r value wherein, The values of the GM(1, 1) first-order differential equation parameters aθ and bθ of the polar angle are obtained according to the least squares method wherein, ； The specific steps of S3 are as follows: S3.1: Send the pedestrian position P1(r1, 0i), the polar radius first-order differential equation parameters a r , b r , the polar angle first-order differential equation parameters a θ , b θ , the timestamp, the sampling interval together to the cloud digital twin system; S3.2: After receiving the data of S3.1 at n+1 time, the cloud digital twin system calculates the time difference as t through the cloud timestamp, that is, the current pedestrian position P n+1 (r n+1 , θ n+1 ) can be calculated through the time domain discrete formula of gray prediction theory wherein 。 2. The method of claim 1, wherein: The angle resolution of the laser radar in S1.2 is 0.2°.

3. The method of claim 1, wherein: The pattern recognition algorithm in S1.3 adopts the YOLO algorithm.

4. The unmanned vehicle pedestrian perception and digital twin method of claim 1, wherein: The sampling data quantity in S2.1 is between 10 and 20, and the sampling interval is 20-30 ms.

Images