A target center positioning compensation method

Abstract

The application relates to the technical field of intelligent traffic, and particularly discloses a target center positioning compensation method, which converts the pixel coordinates of a camera into GPS coordinates through calibration, and then respectively obtains the GPS coordinates of a vehicle, the heading angle beta2 of the vehicle and the north deviation angle beta1 of the camera. When the GPS coordinates of the vehicle are obtained, a time t is selected, vehicle detection is carried out in an image, a two-dimensional rectangular frame is used to frame the position of the vehicle, the center coordinates of the lower edge of the two-dimensional rectangular frame are obtained, the pixel coordinates are converted into first frame GPS coordinates through calibration, a vehicle model is established according to the heading angle beta2 and the north deviation angle beta1, the intersection of the diagonals of the vehicle model is obtained and center compensation is carried out, so as to obtain the position of the target center point. Through the above operation, the vehicle detection position can be compensated to the positive center of the vehicle, the positions detected by all the cameras are consistent, and multi-sensor fusion can be conveniently carried out.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent transportation, and particularly relates to a target center positioning compensation method. BACKGROUND

[0002] A camera is one of important sensors for target perception, and has been widely applied in intelligent transportation, industry, military and other fields. In the field of intelligent transportation, a single camera is usually used for vehicle detection and identification, and multiple cameras are used for holographic perception. When detecting a vehicle, a two-dimensional rectangular frame is first used to frame the pixel position of the vehicle, and then each pixel coordinate is converted to the same coordinate system for fusion through calibration. Because the posture of the vehicle relative to each camera is different, the positions of the vehicle detected by different cameras are different, such as some cameras detecting the head of the vehicle and some cameras detecting the tail of the vehicle, so that the position is abnormal when fusion is performed, resulting in that the vehicle detection position cannot be compensated to the center of the vehicle, which is not convenient for multi-sensor fusion. Therefore, we need to propose a target center positioning compensation method to solve the above problems. SUMMARY

[0003] The application aims to provide a target center positioning compensation method, which can compensate the vehicle detection position to the center of the vehicle, so that the positions detected by each camera are consistent, and multi-sensor fusion is facilitated, so as to solve the problems proposed in the background.

[0004] To achieve the above-mentioned purpose, the application adopts the following technical scheme:

[0005] A target center positioning compensation method comprises the following steps:

[0006] S1, calibrating the pixel coordinates of the camera to GPS coordinates;

[0007] S2, obtaining the GPS coordinates of the vehicle according to the calibration result;

[0008] S3, detecting the vehicle in the image and obtaining the heading angle β2 of the vehicle;

[0009] S4, obtaining the north deviation angle β1 of the camera when the tail of the vehicle is directly opposite to the camera;

[0010] S5, establishing a vehicle model according to the heading angle β2 and the north deviation angle β1;

[0011] S6, obtaining the intersection of the diagonals of the vehicle model and performing center compensation to obtain the position of the target center point.

[0012] Preferably, in step S1, the camera comprises a world coordinate system, a camera coordinate system, an image coordinate system and a pixel coordinate system when imaging, and the camera sequentially converts from the world coordinate system to the camera coordinate system, from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system when imaging, wherein the conversion from the world coordinate system to the camera coordinate system is referred to as rigid transformation, the conversion from the camera coordinate system to the image coordinate system is referred to as perspective projection, and the conversion from the image coordinate system to the pixel coordinate system is referred to as affine transformation, and the imaging formula of the camera is as follows:

[0013]

[0014] wherein (U, V, W) is the physical coordinates of a point in the world coordinate system, (u, v) is the pixel coordinates of the point in the pixel coordinate system, z is a scale factor, f is an image distance, dX and dY respectively represent the physical length of a pixel in the X and Y directions on the camera photosensitive plate, u0 and v0 respectively represent the coordinates of the center of the camera photosensitive plate in the pixel coordinate system, and θ represents the angle between the horizontal side and the vertical side of the photosensitive plate. which is referred to as the extrinsic matrix of the camera, and the extrinsic matrix depends on the relative position of the camera coordinate system and the world coordinate system, wherein R represents a rotation matrix and T represents a translation vector.

[0015] Preferably, in step S2, when acquiring the vehicle GPS coordinates, t time is selected, vehicle detection is performed in the image, a two-dimensional rectangular frame is used to frame the position of the vehicle, the center coordinates of the lower edge of the two-dimensional rectangular frame are acquired, and the pixel coordinates are converted into the first frame GPS coordinates through calibration.

[0016] Preferably, in step S3, when acquiring the heading angle β2 of the vehicle, t+1 time is selected, vehicle detection is performed in the image, the center point coordinates of the lower edge of the two-dimensional rectangular frame are acquired, the pixel coordinates are converted into the second frame GPS coordinates through calibration, and the heading angle of the vehicle is calculated according to the first frame GPS coordinates and the second frame GPS coordinates.

[0017] Preferably, when calculating the heading angle, the azimuth is solved according to the latitude and longitude, the azimuth is calculated according to the latitude and longitude of the first frame GPS coordinates and the second frame GPS coordinates, the latitude and longitude of the first frame GPS coordinates are (lon0, lat0), the latitude and longitude of the second frame GPS coordinates are (lon1, lat1), the azimuth is represented by bearing, and the calculation formula of the azimuth is as follows:

[0018] distance_North = R * (lat1-lat0)

[0019] distance_East = R * cos(lat0) * (lon1-lon0)

[0020] bearing=mod(atan2(distance_East,distance_North),2*pi).

[0021] Preferably, in step S4, when the north deviation angle β1 of the camera is obtained, the heading angle of the vehicle is calculated when the tail of the vehicle is facing the camera, that is, driving away from the camera.

[0022] Preferably, in step S5, the vehicle model is established using a matrix surrounded by four points A, B, C and D, and the length and width of the matrix are the length and width of the vehicle, wherein the length and width of the vehicle are obtained by detecting the vehicle model through the image, and the length and width of the vehicle are estimated according to the vehicle model.

[0023] Preferably, the length and width of the vehicle are obtained by replacing the detection of the vehicle model through the image with the following scheme: the length and width of the vehicle are detected by using a laser radar, and the target detected by the laser radar is associated with the target detected by the image, so as to assign the size of the target detected by the laser radar to the image target.

[0024] Preferably, in step S6, the intersection of the diagonal lines of the vehicle model is obtained, and the coordinate system of the vehicle model is rotated for the convenience of analyzing the data of the vehicle model, the coordinate system is corrected, and the included angle between the heading angle β2 and the north deviation angle β1 is α.

[0025] Preferably, when the center compensation of the vehicle model is performed, the target position obtained by the pixel coordinates is set as T, the center compensation is performed through the coordinates K converted after the calibration, that is, the coordinates of the center point position O of the target are obtained, the distance to be compensated is the distance L between K and O, and the calculation formula of the distance L is:

[0026] L=(length*|cos(α)|+width*|sin(α)|) / 2.

[0027] Compared with the prior art, the target center positioning compensation method has the following advantages:

[0028] The pixel coordinates of the camera are converted into GPS coordinates through calibration, then the GPS coordinates of the vehicle, the heading angle β2 of the vehicle and the north deviation angle β1 of the camera are obtained, the vehicle model is established according to the heading angle β2 and the north deviation angle β1, the intersection of the diagonal lines of the vehicle model is obtained and the center compensation is performed, so as to obtain the target center point position, so that the vehicle detection position can be compensated to the center of the vehicle, the positions detected by each camera are consistent, and the multi-sensor fusion is facilitated. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A flow chart of the present application;

[0030] Figure 2 A two-dimensional rectangular frame diagram of the present application;

[0031] Figure 3 A vehicle model top view of the present application;

[0032] Figure 4 A vehicle model top view of the present application after coordinate correction;

[0033] Figure 5 A vehicle model circumscribed rectangular frame EFGH diagram of the present application;

[0034] Figure 6 A center point compensation diagram of the vehicle model of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0036] The present application provides a target center positioning compensation method as shown in Figure 1 The method comprises the following steps:

[0037] S1, calibrate the conversion of the pixel coordinates of the camera into GPS coordinates;

[0038] The camera includes a world coordinate system, a camera coordinate system, an image coordinate system and a pixel coordinate system when imaging, and the camera sequentially converts from the world coordinate system to the camera coordinate system, from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system when imaging, wherein the conversion from the world coordinate system to the camera coordinate system is called rigid body transformation, the conversion from the camera coordinate system to the image coordinate system is called perspective projection, and the conversion from the image coordinate system to the pixel coordinate system is called affine transformation. The imaging formula of the camera is as follows:

[0039]

[0040] Wherein, (U, V, W) is the physical coordinates of a point in the world coordinate system, (u, v) is the pixel coordinates of the corresponding point in the pixel coordinate system, Z is the scale factor, f is the image distance, dX, dY respectively represent the physical length of a pixel on the camera photosensitive plate in X, Y direction, u0, v0 respectively represent the coordinates of the center of the camera photosensitive plate in the pixel coordinate system, and θ represents the angle between the horizontal side and the vertical side of the photosensitive plate, The external parameter matrix of the camera is called, and the external parameter matrix depends on the relative position of the camera coordinate system and the world coordinate system, R represents the rotation matrix, and T represents the translation vector.

[0041] S2, obtaining the GPS coordinates of the vehicle according to the calibration result;

[0042] When obtaining the GPS coordinates of the vehicle, t time is selected, vehicle detection is carried out in the image, a two-dimensional rectangular frame (p1, p2, p3, p4) is used, as shown in Figure 2 The position where the vehicle is located is framed, the center coordinates (U t ,V t ) of the lower edge of the two-dimensional rectangular frame are obtained, and the pixel coordinates are converted into the first frame GPS coordinates (lat t ,lon t ) through calibration.

[0043] S3, vehicle detection is carried out in the image, and the heading angle β2 of the vehicle is obtained;

[0044] When obtaining the heading angle β2 of the vehicle, t+1 time is selected, vehicle detection is carried out in the image, the center point coordinates (U t+1 ,V t+1 ) of the lower edge of the two-dimensional rectangular frame are obtained, and the pixel coordinates are converted into the second frame GPS coordinates (lat t+1 ,lon t+1 ) through calibration, and the vehicle heading angle is calculated according to the first frame GPS coordinates (lat t ,lon t ) and the second frame GPS coordinates (lat t+1 ,lon t+1 ).

[0045] When calculating the heading angle, the azimuth is solved through the latitude and longitude, the azimuth is calculated according to the latitude and longitude of the first frame GPS coordinate and the second frame GPS coordinate point, the latitude and longitude of the first frame GPS coordinate is (lon0, lat0), the latitude and longitude of the second frame GPS coordinate point is (lon1, lat1), the azimuth is represented by bearing, and the calculation formula of the azimuth is:

[0046] distance_North=R*(lat1-lat0)

[0047] distance_East=R*cos(lat0)*(lon1-lon0)

[0048] bearing=mod(atan2(distance_East,distance_North),2*pi).

[0049] S4, obtaining the north angle β1 of the camera when the rear end of the vehicle is facing the camera;

[0050] When obtaining the north angle β1 of the camera, when the rear end of the vehicle is facing the camera, that is, when it is traveling in a direction away from the camera, the heading angle of the vehicle is calculated. The heading angle at this time is the north angle of the camera.

[0051] S5. Establish a vehicle model according to the heading angle β2 and the northing angle β1;

[0052] like Figure 3 As shown in the figure, when the vehicle model is established, a matrix formed by four points A, B, C, and D is used. The length and width of the matrix are the length and width of the vehicle respectively. The length and width of the vehicle are obtained by detecting the vehicle model through the image, and the length and width of the vehicle are estimated according to the model.

[0053] The vehicle's length and width are estimated based on the vehicle model by replacing the model detected in the image with the following scheme: the length and width of the vehicle are obtained using lidar detection, and the target detected by the lidar is associated with the target detected by the image, thereby assigning the target size detected by the lidar to the image target.

[0054] S6. Obtain the intersection of the vehicle model diagonals and perform center compensation to obtain the target center point position.

[0055] When the intersection of the vehicle model diagonal lines is obtained, in order to facilitate the analysis of the vehicle model data, the coordinate system of the vehicle model is rotated and the coordinate system is corrected, such as Figure 4 As shown, the included angle between the heading angle β2 and the northing angle β1 is α.

[0056] Draw the vehicle's circumscribed rectangular frame EFGH along the north angle of the camera, as shown in Figure 5 As shown,

[0057] Since the two-dimensional rectangular box of the vehicle on the pixel is (p1, p2, p3, p4), the projection point of the EF center point K on the pixel is T, and the coordinates are (U t ,V t ), GPS coordinates are (lat t ,lon t ).

[0058] As shown in Figure 6 , the intersection of the diagonal of the vehicle model is O, the vehicle model sets the target position obtained by the pixel coordinates as T, and performs center compensation through the coordinates K converted after calibration, that is, the coordinates of the center point position point O of the target are obtained, and the distance L to be compensated is the distance between K and O. The calculation formula of the distance L is:

[0059] L = (length * |cos(a)| + width * |sin(a)|) / 2.

[0060] Rotate the coordinate system, and then compensate to the center to obtain the GPS coordinates of point O. Since the GPS coordinates of K are (lat t ,lon t ), the distance L between K and O is (lenght * |cos(a)| + width * |sin(a)|) / 2, and the angle of K and O is a, then the GPS coordinates of point O can be calculated.

[0061] In summary, by calibration, the pixel coordinates of the camera are converted into GPS coordinates, and then the GPS coordinates of the vehicle, the heading angle β2 of the vehicle and the north deviation angle β1 of the camera are obtained. According to the heading angle β2 and the north deviation angle β1, a vehicle model is established, the intersection of the diagonal of the vehicle model is obtained, and center compensation is performed to obtain the center point position of the target, so that the vehicle detection position can be compensated to the center of the vehicle, and the positions detected by each camera are consistent, which is convenient for multi-sensor fusion.

[0062] The application can also be applied to estimate the size of the vehicle:

[0063] When the size of the vehicle is unknown, two cameras can be used to estimate the size of the vehicle. The steps are as follows:

[0064] 1) Use two cameras with different installation angles to detect the vehicle, such as two cameras with an included angle of 90°;

[0065] 2) Set the length of the vehicle as length and the width as width. Two cameras use the center point compensation method to obtain the compensated positions O1 and O2 respectively. (O is the English letter O)

[0066] 3) Traverse different length and width to calculate the positions of O1 and O2

[0067] 4) When the positions of O1 and O2 are the same, the length and width at this time are the size of the target.

[0068] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A target center positioning compensation method, characterized by: The steps include: S1, calibration converts the camera's pixel coordinates into GPS coordinates; S2. Obtain the GPS coordinates of the vehicle according to the calibration results; S3, detecting the vehicle in the image and obtaining the vehicle's heading angle β2; When obtaining the vehicle's heading angle β2, select time t+1 to detect the vehicle in the image and obtain the coordinates of the center point of the lower edge of the two-dimensional rectangular frame (U t+1 ,V t+1 ), and through calibration, the pixel coordinates are converted into the second frame GPS coordinates (lat t+1 ,lon t+1 ), according to the first frame GPS coordinates (lat t ,lon t ) and the second frame GPS coordinates (lat t+1 ,lon t+1 ), calculate the vehicle heading angle; When calculating the heading angle, the azimuth is solved by the longitude and latitude. The azimuth is calculated based on the longitude and latitude of the first frame GPS coordinates and the second frame GPS coordinates. The longitude and latitude of the first frame GPS coordinates are (lon0, lat0), and the longitude and latitude of the second frame GPS coordinates are (lon1, lat1). The azimuth is expressed in bearing. The calculation formula for the azimuth is: distance_North=R*(lat1-lat0) distance_East=R*cos(lat0)*(lon1-lon0) bearing = mod(atan2(distance_East, distance_North), 2*pi), where R represents the camera's rotation matrix, distance_North is the longitudinal distance between the first and second GPS frames, and distance_East is the lateral distance between the first and second GPS frames. S4, obtaining the north angle β1 of the camera when the rear end of the vehicle is facing the camera; S5. Establish a vehicle model according to the heading angle β2 and the northing angle β1; S6. Obtain the intersection of the vehicle model diagonals and perform center compensation to obtain the target center point position; When the intersection of the vehicle model diagonal lines is obtained, in order to facilitate the analysis of the vehicle model data, the coordinate system of the vehicle model is rotated and the coordinate system is corrected. The angle between the heading angle β2 and the north angle β1 is α; along the north angle direction of the camera, the circumscribed rectangular frame EFGH of the vehicle is drawn; since the two-dimensional rectangular frame of the vehicle on the pixel is (p1, p2, p3, p4), the projection point of the EF center point K on the pixel is T, and the coordinates are (U t ,V t ), GPS coordinates are (lat t ,lon t ); The intersection of the diagonal lines of the vehicle model is O. When performing center compensation on the vehicle model, the target position obtained by pixel coordinates is set to T. The center compensation is performed using the coordinates K converted after calibration, that is, the coordinates of the target center point O are obtained. The distance to be compensated is the distance L between K and K. The calculation formula for distance L is: L=(length*|cos(α)|+width*|sin(α)|) / 2; Rotate the coordinate system and compensate to the center to obtain the GPS coordinates of point O. Since the GPS coordinates of K are (lat t ,lon t ), KO distance L = (length*|cos(α)+width*|sin(α)|) / 2, KO angle is α, vehicle length is length, vehicle width is width, then the GPS coordinates of point O can be calculated.

2. The target center positioning compensation method according to claim 1, characterized in that: In step S1, the camera includes a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system during imaging, and the camera sequentially converts from the world coordinate system to the camera coordinate system, the camera coordinate system to the image coordinate system, and the image coordinate system to the pixel coordinate system during imaging. The conversion from the world coordinate system to the camera coordinate system is called a rigid body transformation, the conversion from the camera coordinate system to the image coordinate system is called a perspective projection, and the conversion from the image coordinate system to the pixel coordinate system is called an affine transformation. The imaging formula of the camera is as follows: Among them, (U, V, W) is the physical coordinate of a point in the world coordinate system, (u, v) is the pixel coordinate of the point in the pixel coordinate system, z is the scale factor, f is the image distance, dX and dY are the physical lengths of a pixel on the camera plate in the X and Y directions respectively, u0 and v0 are the coordinates of the center of the camera plate in the pixel coordinate system respectively, and θ is the angle between the horizontal and vertical edges of the plate. It is called the extrinsic parameter matrix of the camera. The extrinsic parameter matrix depends on the relative position of the camera coordinate system and the world coordinate system. R represents the rotation matrix and T represents the stationary vector.

3. The target center positioning compensation method according to claim 1, characterized in that: In step S2, when obtaining the vehicle GPS coordinates, time t is selected to perform vehicle detection in the image, and the location of the vehicle is framed using a two-dimensional rectangular frame. The center coordinates of the lower edge of the two-dimensional rectangular frame are obtained, and the pixel coordinates are converted into the first frame GPS coordinates through calibration.

4. The target center positioning compensation method according to claim 3, characterized in that: In step S4, when obtaining the camera's north angle β1, when the rear end of the vehicle is facing the camera, that is, when it is moving away from the camera, the vehicle's heading angle is calculated. The heading angle at this time is the camera's north angle.

5. The target center positioning compensation method according to claim 1, characterized in that: In step S5, when establishing the vehicle model, a matrix formed by four points A, B, C, and D is used. The length and width of the matrix are the length and width of the vehicle, respectively. The length and width of the vehicle are acquired by detecting the vehicle model through image detection, and the length and width of the vehicle are estimated based on the model.

6. The target center positioning compensation method according to claim 5, characterized in that: When obtaining the length and width of the vehicle, the following scheme is used to replace the vehicle model detected by the image, and the length and width of the vehicle are estimated based on the model: the length and width of the vehicle are obtained by using lidar detection, and the target detected by the lidar is associated with the target detected by the image, so as to assign the target size detected by the lidar to the image target.

Images