Roadside sensor calibration system and method for intelligent network connection road

Abstract

According to the method for calibrating the roadside sensor of the intelligent network connection road, the roadside sensor does not need to be disassembled or moved in the initial calibration and re-calibration processes, measurement can be carried out only by passing through the intelligent network connection road through a calibration vehicle, daily traffic is not affected, the calculation method is simple and rapid, the calibration efficiency is greatly improved, and the method is suitable for popularization and application. The method is especially suitable for roadside environments. Meanwhile, the invention also discloses a road side sensor calibration system of the intelligent network connection road.

Description

Technical Field

[0001] The present invention relates to the field of intelligent connected traffic technology, and in particular to a roadside sensor calibration system and method for an intelligent connected road. Background Art

[0002] With the development of vehicle-road-cloud integration, the construction of intelligent connected roads has become one of the important links. Roadside sensors are used to provide beyond-line-of-sight perception information to make up for the lack of perception capabilities of single-vehicle intelligent sensors. At the beginning of installation, the roadside sensors of intelligent connected roads need to be calibrated for internal and external parameters to ensure the accuracy of target detection. However, since roadside sensors need to be installed on roadside poles, the passing of large vehicles or bad weather such as strong winds will seriously affect the calibration effect of roadside sensors, resulting in a decrease in the output target detection accuracy. The current common method is to recalibrate roadside sensors regularly. The existing method of recalibrating sensors requires staff to make long-term on-site adjustments, and even requires the roadside sensors to be disassembled, and then the calibration device is used with complex algorithms to recalibrate the roadside sensors. After the recalibration is completed, they are reinstalled, resulting in the entire calibration process being very complicated and cumbersome. The calibration process affects daily traffic and is not suitable for roadside environments. Summary of the invention

[0003] In view of the shortcomings of the existing roadside sensor calibration method, that the calibration process is complicated and cumbersome, the calibration process affects daily traffic, and is not suitable for the roadside environment, the present invention provides a roadside sensor calibration system and method for intelligent connected roads, which has high calibration efficiency, does not affect daily traffic, and is suitable for the roadside environment.

[0004] A roadside sensor calibration system for an intelligent connected road, characterized in that it comprises: a calibration vehicle, a roadside sensor, a roadside edge computing unit and a roadside communication terminal that are interconnected; the calibration vehicle is an intelligent connected car, equipped with a centimeter-level precision positioning device and a vehicle-to-external information exchange unit V2X; the roadside sensor comprises an image sensor.

[0005] It is further characterized in that the image sensor includes one or more combinations of a camera and a lidar, and the roadside sensor also includes a millimeter wave radar.

[0006] A method for calibrating a roadside sensor of an intelligent connected road, characterized in that it is based on the roadside sensor calibration system of an intelligent connected road, and comprises the following steps:

[0007] S1: Recording the initial calibration value; the roadside edge calculation unit starts the initial calibration, and the initial calibration process is as follows:

[0008] a1: the roadside edge computing unit selects a vehicle for initial calibration and records the vehicle model;

[0009] a2: the vehicle for initial calibration passes through an intelligent connected road on which the roadside sensor is installed. When passing through an area covered by the roadside sensor, the roadside sensor detects image information of the vehicle for initial calibration;

[0010] a3: The initial calibration vehicle sends its real-time information to the roadside communication terminal via V2X, where the real-time information includes the real-time positioning information and actual size of the calibration vehicle;

[0011] a4: the roadside edge calculation unit calculates and outputs a detection result according to the image information, wherein the detection result includes pixel coordinates and calculated sizes in the image information of the vehicle used for initial calibration;

[0012] a5: the roadside edge computing unit records the detection result and the real-time information as the initial calibration value;

[0013] The recording format of the test result is [(u 1j ,v 1j ),(u 2j ,v 2j ),(u 3j ,v 3j ),(u 4j ,v 4j ),(l j ,h j ,w j )],

[0014] Among them, (u ij ,v ij ) is the coordinate of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed by the vehicle for the initial calibration in the j-th frame image information detected by the roadside sensor, i<=4;

[0015] (l j ,h j ,w j ) is the length, height and width of the vehicle for initial calibration calculated based on the coordinates of the pixels occupied by the vehicle for initial calibration in the j-th frame image information detected by the roadside sensor and the actual size;

[0016] The recording format of the real-time information is [(x j ,y j ,z j ),(L,H,W)];

[0017] Among them, (x j ,y j ,zj ) is the latitude and longitude coordinates of the vehicle used for initial calibration at the j-th frame image, (L, H, W) is the actual length, height and width of the vehicle used for initial calibration;

[0018] S2: record the recalibration value;

[0019] The roadside edge calculation unit starts recalibration according to a preset calibration start condition; the recalibration process is as follows:

[0020] B1: The roadside edge computing unit selects a smart connected vehicle of the same model as the vehicle used for initial calibration as a vehicle for recalibration;

[0021] B2: The recalibration vehicle passes through the intelligent connected road in the same driving direction as the initial calibration vehicle, and the recalibration vehicle sends its real-time information to the roadside communication terminal via V2X;

[0022] The recording format of the real-time information of the recalibration vehicle is [(x j ',y j ',z j '),(L,H,W)]; where (x j ',y j ',z j ') are the latitude and longitude coordinates of the recalibration vehicle, and (L, H, W) are the actual length, height and width of the recalibration vehicle;

[0023] B3: When x j '=x j ,y j '=y j ,z j '=z j At a certain moment, the roadside sensor detects the image information of the recalibration vehicle, and the roadside edge calculation unit calculates and records the recalibration value according to the image information; the recalibration value includes the pixel coordinates and the calculated size in the image information of the vehicle for initial calibration;

[0024] The recording format of the recalibration value is [(u 1j ',v 1j '),(u 2j ',v 2j '),(u 3j ',v 3j '),(u 4j ',v 4j '),(l j ',h j ',w j ')];

[0025] Among them, (u ij',v ij ') is the coordinate of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed by the recalibration vehicle in the j-th frame image information detected by the roadside sensor, i<=4; (l j ',h j ',w j ') is the length, height and width of the recalibrated vehicle obtained based on the coordinates of the occupied pixels in the image information of the recalibrated vehicle and the actual size;

[0026] S3: Calculate the overall deviation between the initial calibration value and the recalibration value to determine whether recalibration is required;

[0027] x j =x j ',y j =y j ',z j =z j 'The recalibrated value at the moment is compared with the initial calibration value to obtain the overall deviation Δ between the recalibrated value at the current moment and the initial calibration value; if the overall deviation Δ is greater than a threshold value D, the current roadside sensor needs to be recalibrated and enters S4; otherwise, recalibration is not required and the process ends;

[0028] S4: Calculate the calibration matrix;

[0029] S5: storing the calibration matrix in the roadside edge calculation unit to complete recalibration; and multiplying the detection results of the roadside sensor before the next calibration is completed by the calibration matrix.

[0030] It is further characterized by:

[0031] In S3, the overall deviation is calculated as follows:

[0032] Assume the maximum value of j is n, and calculate the imaging position deviation:

[0033]

[0034] Calculate the deviation of the detected size:

[0035]

[0036] Calculate the overall deviation:

[0037] Δ=m1α+m2β

[0038] Wherein, m1 is the preset weight coefficient of the imaging position deviation, and m2 is the preset weight coefficient of the detection size deviation;

[0039] In S4, the calibration matrix is ​​calculated as follows:

[0040] For the initial calibration value and the recalibration value of the j-th frame image, assume:

[0041] u j =(u 1j +u 2j +u 3j +u 4j ) / 4,v j =(v 1j +v 2j +v 3j +v 4j ) / 4;

[0042] u j '=(u 1j '+u 2j '+u 3j '+u 4j ') / 4,v j '=(v 1j '+v 2j '+v 3j '+v 4j ') / 4;

[0043] All the initial calibration values ​​are processed as above and recorded as the initial matrix [u j ,v j ], all the recalibrated values ​​are processed as above and recorded as the recalibration matrix [u j ',v j '], calculate the calibration matrix X according to the following formula:

[0044] [u j ',v j ']X=[u j ,v j ];

[0045] In S1, the initial calibration is performed in good weather, where the good weather is sunny and the wind force is less than level 3;

[0046] In S2, the start-up conditions include encountering bad weather or the next good weather after the regular inspection time, and the bad weather includes rainy and snowy days, or wind force greater than or equal to level three.

[0047] The images of the calibration vehicles selected multiple times during the initial calibration of the roadside sensor cover the area where the vehicles pass by on the intelligent connected road;

[0048] In S1 and S2, the calibration vehicle passes through the intelligent connected road once at a speed of 3-5 km / h, or passes through the intelligent connected road repeatedly at a speed of 5-40 km / h.

[0049] After adopting the present invention, the initial calibration and recalibration process does not require disassembly or movement of the roadside sensor. It only needs to pass the calibration vehicle through the intelligent network road to perform measurement, which does not affect daily traffic. The calculation method is simple and fast, which greatly improves the calibration efficiency and is particularly suitable for the roadside environment. During calibration, it is only necessary to record the real-time positioning information, actual size, pixel coordinates and calculated size in the image information of the calibration vehicle, and the amount of recorded data is small, so that the recording process has good real-time performance and high recording efficiency. During the calculation process, to determine whether the roadside sensor needs to be recalibrated, it is only necessary to calculate the square of the difference between the recalibration value and the initial calibration value data, multiply it by the weight coefficient and add it, calculate the overall deviation of the roadside sensor, and thus determine; when calculating the calibration matrix, the initial calibration value and the recalibration value are processed and recorded as the initial matrix and the recalibration matrix. According to the formula, only one matrix equation needs to be solved to calculate the calibration matrix, and the calibration matrix is ​​stored in the roadside sensor. The subsequent measurement results are multiplied by the calibration matrix to complete the recalibration. The algorithm is simple and fast, which further improves the real-time performance and calculation efficiency, and is more suitable for the roadside environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a method for calibrating a roadside sensor on an intelligent connected road according to the present invention;

[0051] Figure 2 This is a scenario example diagram of the present invention. DETAILED DESCRIPTION

[0052] See Figure 2 , a roadside sensor 1 calibration system for an intelligent networked road 3, comprising: a calibration vehicle 2, a roadside sensor 1, a roadside edge computing unit and a roadside communication terminal connected to each other. When implemented, the roadside edge computing unit and the roadside communication terminal are installed in a device box with the roadside sensor 1; the calibration vehicle 2 is an intelligent networked car, equipped with a centimeter-level precision positioning device and a vehicle-to-external information exchange unit V2X; the roadside sensor 1 includes an image sensor. The image sensor includes one or more combinations of a camera and a laser radar; the image sensor of the roadside sensor 1 can also be combined with a millimeter-wave radar.

[0053] Figure 2 In FIG. 8 , the angle α represents the detection range angle of the roadside sensor 1 .

[0054] A method for calibrating a roadside sensor 1 of an intelligent connected road 3 is based on a roadside sensor 1 calibration system of an intelligent connected road 3, and comprises the following steps:

[0055] S1: Record the initial calibration value; the roadside edge calculation unit starts the initial calibration, and the initial calibration process is as follows:

[0056] a1: The roadside edge computing unit selects a vehicle for initial calibration and records the vehicle model;

[0057] a2: The vehicle for initial calibration passes through the intelligent connected road 3 on which the roadside sensor 1 is installed. When passing through the coverage area of ​​the roadside sensor 1, the roadside sensor 1 detects the image information of the vehicle for initial calibration;

[0058] The roadside sensor 1 evenly selects 10 frames of images per second. The images of the calibration vehicle 2 selected multiple times during the initial calibration of the roadside sensor 1 cover the area where cars pass through the intelligent connected road 3; the speed of the calibration vehicle 2 passing through the intelligent connected road 3 is 3-5km / h. The calibration vehicle 2 passes through the intelligent connected road 3 once at a speed of 3-5km / h, or passes through the intelligent connected road 3 repeatedly at a speed of 5-40km / h, to ensure that the roadside sensor 1 can collect enough calibration images to cover the area where cars pass through the intelligent connected road 3, and at the same time, the image will not be blurred due to excessive speed.

[0059] a3: The vehicle used for initial calibration sends its real-time information to the roadside communication terminal via V2X. The real-time information includes the real-time positioning information and actual size of the vehicle used for calibration;

[0060] a4: The roadside edge calculation unit calculates and outputs a detection result based on the image information. The detection result includes the pixel coordinates and calculated size in the image information of the vehicle used for the initial calibration;

[0061] a5: The roadside edge computing unit records the detection results and real-time information as the initial calibration value;

[0062] The actual size of the calibration vehicle 2 is preset by the calibration party, and is usually the standard design size of the vehicle manufacturer.

[0063] The record format of the test result is [(u 1j ,v 1j ),(u 2j ,v 2j ),(u 3j ,v 3j ),(u 4j ,v 4j ),(l j ,h j ,w j )]; In each frame of the image, a plane coordinate system is constructed, with the upper left corner as the 0 point, the pixel with the largest x and y values ​​in the area where the vehicle is located for the initial calibration as the upper left corner, and the pixel with the smallest x and y values ​​as the lower right corner, to construct a rectangular frame;

[0064] Among them, (u ij ,v ij ) is the first calibration of the vehicle detected by the roadside sensor j The coordinates of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed in the frame image information, i<=4;

[0065] (l j ,h j ,w j ) is the first time the vehicle is detected by the roadside sensor according to the initial calibration j The length, height and width of the vehicle used for initial calibration are calculated from the coordinates of the pixels occupied in the frame image information and the actual size; the method for calculating the size is prior art and will not be described in detail here;

[0066] The recording format of real-time information is [(x j ,y j ,z j ),(L,H,W)];

[0067] Among them, (x j ,y j ,z j ) is the latitude and longitude coordinates of the vehicle used for initial calibration at the j-th frame image, (L, H, W) is the actual length, height and width of the vehicle used for initial calibration;

[0068] S2: record the recalibration value;

[0069] The roadside edge computing unit starts recalibration according to the preset calibration start conditions; the start conditions include encountering bad weather or the next good weather after the regular inspection time. Bad weather includes rain and snow, or wind speed is greater than or equal to level three.

[0070] The recalibration process is as follows:

[0071] B1: The roadside edge computing unit selects an intelligent connected vehicle of the same model as the vehicle used for the initial calibration as the vehicle for recalibration;

[0072] B2: The recalibration vehicle travels through the intelligent connected road 3 in the same driving direction as the initial calibration vehicle, and the recalibration vehicle sends its real-time information to the roadside communication terminal via V2X;

[0073] The recording format of the real-time information of the recalibration vehicle is [(x j ',y j ',z j '),(L,H,W)]; where (x j ',y j ',z j ') is the jThe latitude and longitude coordinates of the recalibration vehicle when the frame image is taken, (L, H, W) are the actual length, height and width of the recalibration vehicle;

[0074] B3: When x j '=x j ,y j '=y j ,z j '=z j At the moment, the roadside sensor 1 detects the image information of the recalibration vehicle, and the roadside edge calculation unit calculates and records the recalibration value according to the image information; the recalibration value includes the pixel coordinates and the calculated size in the image information of the vehicle used for the initial calibration;

[0075] The recording format of the recalibration value is [(u 1j ',v 1j '),(u 2j ',v 2j '),(u 3j ',v 3j '),(u 4j ',v 4j '),(l j ',h j ',w j ')];

[0076] Among them, (u ij ',v ij ') is the coordinate of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed by the vehicle in the j-th frame image information detected by the roadside sensor 1, i<=4; (l j ',h j ',w j ') is the length, height and width of the recalibrated vehicle obtained based on the coordinates of the occupied pixels in the image information of the recalibrated vehicle and the actual size;

[0077] S3: Calculate the overall deviation between the initial calibration value and the re-test value to determine whether re-calibration is needed;

[0078] x j =x j ',y j =y j ',z j =z j 'The re-test value at the moment is compared with the initial calibration value to obtain the overall deviation between the re-test value at the current moment and the initial calibration value. The specific calculation method is as follows:

[0079] Assume the maximum value of j is n, and calculate the imaging position deviation:

[0080]

[0081] Calculate the deviation of the detected size:

[0082]

[0083] Calculate the overall deviation:

[0084] Δ=m1α+m2β;

[0085] Wherein, m1 is a preset weight coefficient of imaging position deviation, and m2 is a preset weight coefficient of detection size deviation. The weight coefficient of imaging position deviation and the weight coefficient of detection size deviation can be adjusted according to the actual application scenario and sensor combination.

[0086] If Δ>D, the current roadside sensor 1 needs to be recalibrated and the process goes to S4; otherwise, recalibration is not required and the process ends;

[0087] Wherein, D is the threshold of the overall deviation, and the specific data of D can be set according to the actual application scenario and sensor combination;

[0088] S4: Calculate the calibration matrix;

[0089] The calculation method is as follows:

[0090] For the initial calibration value and re-detection value of the j-th frame image, let:

[0091] u j =(u 1j +u 2j +u 3j +u 4j ) / 4,v j =(v 1j +v 2j +v 3j +v 4j ) / 4;

[0092] u j '=(u 1j '+u 2j '+u 3j '+u 4j ') / 4,v j '=(v 1j '+v 2j '+v 3j '+v 4j ') / 4;

[0093] All the initial calibration values ​​are processed as above and recorded as the initial matrix [uj,vj], and all the re-test values ​​are processed as above and recorded as the re-calibration matrix [u j ',v j'], calculate the calibration matrix X according to the following formula:

[0094] [u j ',v j ']X=[u j ,v j ];

[0095] S5: The calibration matrix is ​​stored in the roadside edge computing unit to complete the recalibration; the detection results of the roadside sensor 1 before the next calibration is completed are multiplied by the calibration matrix.

Claims

1. A roadside sensor calibration system for an intelligent connected road, characterized in that: It includes: a calibration vehicle, a roadside sensor, a roadside edge computing unit and a roadside communication terminal that are interconnected; the calibration vehicle is an intelligent connected car, equipped with a centimeter-level precision positioning device and a vehicle-to-external information exchange unit V2X; the roadside sensor includes an image sensor.

2. A roadside sensor calibration system for an intelligent connected road according to claim 1, characterized in that: The image sensor includes one or more combinations of a camera and a lidar, and the roadside sensor also includes a millimeter wave radar.

3. A method for calibrating the roadside sensor of an intelligent connected road, characterized in that: The method is based on a roadside sensor calibration system for an intelligent connected road according to claim 1, and comprises the following steps: S1: Recording the initial calibration value; the roadside edge calculation unit starts the initial calibration, and the initial calibration process is as follows: a1: the roadside edge computing unit selects a vehicle for initial calibration and records the vehicle model; a2: the vehicle for initial calibration passes through an intelligent connected road on which the roadside sensor is installed. When passing through an area covered by the roadside sensor, the roadside sensor detects image information of the vehicle for initial calibration; a3: The initial calibration vehicle sends its real-time information to the roadside communication terminal via V2X, where the real-time information includes the real-time positioning information and actual size of the calibration vehicle; a4: the roadside edge calculation unit calculates and outputs a detection result according to the image information, wherein the detection result includes pixel coordinates and calculated sizes in the image information of the vehicle used for initial calibration; a5: the roadside edge computing unit records the detection result and the real-time information as the initial calibration value; The recording format of the test result is [(u 1j ,v 1j ),(u 2j ,v 2j ),(u 3j ,v 3j ),(u 4j ,v 4j ),(l j ,h j ,w j )], Among them, (u ij ,v ij ) is the coordinate of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed by the vehicle for the initial calibration in the j-th frame image information detected by the roadside sensor, i<=4; (l j ,h j ,w j ) is the length, height and width of the vehicle for initial calibration calculated based on the coordinates of the pixels occupied by the vehicle for initial calibration in the j-th frame image information detected by the roadside sensor and the actual size; The recording format of the real-time information is [(x j ,y j ,z j ),(L,H,W)]; Among them, (x j ,y j ,z j ) is the latitude and longitude coordinates of the vehicle used for initial calibration at the j-th frame image, (L, H, W) is the actual length, height and width of the vehicle used for initial calibration; S2: record the recalibration value; The roadside edge calculation unit starts recalibration according to a preset calibration start condition; the recalibration process is as follows: B1: The roadside edge computing unit selects a smart connected vehicle of the same model as the vehicle used for initial calibration as a vehicle for recalibration; B2: The recalibration vehicle passes through the intelligent connected road in the same driving direction as the initial calibration vehicle, and the recalibration vehicle sends its real-time information to the roadside communication terminal via V2X; The recording format of the real-time information of the recalibration vehicle is [(x j ',y j ',z j '),(L,H,W)]; where (x j ',y j ',z j ') are the latitude and longitude coordinates of the recalibration vehicle, and (L, H, W) are the actual length, height and width of the recalibration vehicle; B3: When x j '=x j ,y j '=y j ,z j '=z j At a certain moment, the roadside sensor detects the image information of the recalibration vehicle, and the roadside edge calculation unit calculates and records the recalibration value according to the image information; the recalibration value includes the pixel coordinates and the calculated size in the image information of the vehicle for initial calibration; The recording format of the recalibration value is [(u 1j ',v 1j '),(u 2j ',v 2j '),(u 3j ',v 3j '),(u 4j ',v 4j '),(l j ',h j ',w j ')]; Among them, (u ij ',v ij ') is the coordinate of the i-th corner of the four corners of the pixels occupied by the rectangular frame constructed by the recalibration vehicle in the j-th frame image information detected by the roadside sensor, i<=4; (l j ',h j ',w j ') is the length, height and width of the recalibrated vehicle obtained based on the coordinates of the occupied pixels in the image information of the recalibrated vehicle and the actual size; S3: Calculate the overall deviation between the initial calibration value and the recalibration value to determine whether recalibration is required; x j =x j ',y j =y j ',z j =z j 'The recalibrated value at the moment is compared with the initial calibration value to obtain the overall deviation Δ between the recalibrated value at the current moment and the initial calibration value; if the overall deviation Δ is greater than the threshold value D, the current roadside sensor needs to be recalibrated and enters S4; otherwise, recalibration is not required and the process ends; S4: Calculate the calibration matrix; S5: storing the calibration matrix in the roadside edge calculation unit to complete recalibration; and multiplying the detection results of the roadside sensor before the next calibration is completed by the calibration matrix.

4. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: In S3, the overall deviation is calculated as follows: Assume the maximum value of j is n, and calculate the imaging position deviation: Calculate the deviation of the detected size: Calculate the overall deviation: Δ=m1α+m2β Among them, m1 is the preset weight coefficient of the imaging position deviation, and m2 is the preset weight coefficient of the detection size deviation.

5. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: In S4, the calibration matrix is ​​calculated as follows: For the initial calibration value and the recalibration value of the j-th frame image, assume: u j =(u 1j +u 2j +u 3j +u 4j ) / 4,v j =(v 1j +v 2j +v 3j +v 4j ) / 4; u j '=(u 1j '+u 2j '+u 3j '+u 4j ') / 4,v j '=(v 1j '+v 2j '+v 3j '+v 4j ') / 4; All the initial calibration values ​​are processed as above and recorded as the initial matrix [u j ,v j ], all the recalibrated values ​​are processed as above and recorded as the recalibration matrix [u j ',v j '], calculate the calibration matrix X according to the following formula: [u j ’,v j ’]X=[u j ,v j ]。 6. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: In S1, the initial calibration is performed in good weather, where the good weather is sunny and the wind force is less than level three.

7. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: In S2, the start-up conditions include encountering bad weather or the next good weather after the regular inspection time, and the bad weather includes rainy and snowy days, or wind force greater than or equal to level three.

8. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: The images of the calibration vehicles selected multiple times during the initial calibration of the roadside sensor cover the area where the car passes by on the intelligent connected road.

9. The method for calibrating the roadside sensor of an intelligent connected road according to claim 3, characterized in that: In S1 and S2, the calibration vehicle passes through the intelligent connected road once at a speed of 3-5 km / h, or passes through the intelligent connected road repeatedly at a speed of 5-40 km / h.

Images