A Method for Automatic Rectification and Target Location of Roadside Cameras

Abstract

The present invention discloses a method for automatic rectification and target positioning of roadside cameras, which relates to the technical field of intelligent traffic control and includes the following steps: collecting images of the camera, wherein, by comparing with the original camera images, calculating the projection matrix, and calculating the projection transformation matrix from the collected images to the original images; according to the projection transformation matrix, calculating the rectified coordinates of the key points of the collected images, and calculating the projection error with the coordinates of the key points of the original images; according to the projection error and a preset error threshold, determining whether the projection transformation matrix meets the error accuracy requirements; performing projection transformation on the pixel coordinates of the target center point detected in real time to obtain the rectified pixel coordinates of the center point; calculating the world coordinates of the center point. The present invention can still meet the positioning accuracy requirements when the camera screen is offset, and is applied to artificial intelligence, intelligent traffic, vehicle-road collaborative roadside perception and computer vision in computer technology to achieve the effect of improving the target positioning accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic control, and specifically to a method for automatic correction and target positioning of roadside cameras. Background Art

[0002] Roadside cameras are important basic components in vehicle-road collaborative systems. By detecting and positioning traffic participants on the road through roadside cameras, basic data can be provided for various vehicle-road collaborative applications. In computer vision applications, to determine the position of a certain point on the surface of a spatial object in the world coordinate system, it is necessary to calibrate the camera, so as to convert the target pixel coordinates into world coordinates. The external parameter matrix (R, t) of the camera consists of rotation and translation matrices, and is related to the pose of the camera during installation.

[0003] However, affected by various factors, the pose of the camera will slowly change over time. Usually, within a day, the maximum offset of the camera image can reach 10 pixels. This will result in low target positioning accuracy in actual applications if the conversion parameters from pixel coordinates to world coordinates obtained during calibration are directly used. The positioning accuracy of the target is an extremely important perception performance indicator. If target data with poor positioning accuracy is sent to an autonomous vehicle, it will cause the vehicle to make incorrect judgments. However, due to the installation method, the camera image generally changes slightly over time and there is an offset from the image during camera calibration. Using the calibration parameters directly without correction will reduce the positioning accuracy.

[0004] Therefore, the present invention proposes a method for automatic correction and target positioning of roadside cameras. Summary of the Invention

[0005] The present invention provides a method for automatic correction and target positioning of roadside cameras, which promotes the solution of the problems mentioned in the above background art.

[0006] The present application provides a method for automatic correction and target positioning of roadside cameras, and adopts the following technical solutions: A method for automatic correction and target positioning of roadside cameras includes the following steps:

[0007] S1. Collect images of the camera,

[0008] wherein, by comparing with the original camera image, a projection matrix is calculated, including:

[0009] Detect key points of the collected image and the original image;

[0010] S2. Calculate the projective transformation matrix from the collected image to the original image according to the key points of the collected image and the key points of the original image;

[0011] S3. Calculate the corrected coordinates of the key points of the collected image according to the projective transformation matrix, and calculate the projection error with the coordinates of the key points of the original image;

[0012] S4. Determine whether the projection transformation matrix meets the error accuracy requirement according to the projection error and a preset error threshold;

[0013] S5. Perform projection transformation on the pixel coordinates of the target center point detected in real time to obtain the corrected pixel coordinates of the center point;

[0014] S6. Calculate the world coordinates of the center point according to the pixel coordinates of the center point.

[0015] Through the above technical solution, the image collected in real time is feature-associated with the original image saved during calibration to obtain the projection transformation matrix from the collected image to the original image. After detecting the target pixel coordinates, first use the above matrix for correction, and then calculate the world coordinates corresponding to the corrected pixel coordinates.

[0016] Further, S2 includes the following steps:

[0017] S21. Use Scale-Invariant Feature Transform (SIFT) to detect the local feature descriptors of the image;

[0018] S22. According to the local feature descriptors of the collected image and the original image, use k-Nearest Neighbor (kNN) matching to obtain the pairwise correspondence;

[0019] S23. Calculate the homography matrix from the collected image to the original image according to the kNN matching result.

[0020] Further, S3 includes:

[0021] Detect all the tire contact points of the target;

[0022] According to all the detected tire contact points, calculate the average coordinates to obtain the center point;

[0023] Perform projection transformation on the pixel coordinates of the geometric center point according to the projection transformation matrix.

[0024] Through the above technical solution, when using the projection transformation matrix, its reprojection error can be combined. The offset of the camera screen can be ignored in a short period of time, and multiple groups of projection transformation matrices and reprojection errors will be calculated during this period. During actual correction, the projection transformation matrix with the smallest reprojection error can be selected to perform projection transformation on the target pixel coordinates. Generally, a projection transformation matrix with a reprojection error less than 1 pixel can be selected.

[0025] Further, S6 includes:

[0026] Based on the calibrated internal and external parameters of the camera, calculate the coordinates of the center point in the camera coordinate system;

[0027] Based on the orientation angle of the target, calculate the coordinates of the center point in the geodetic coordinate system;

[0028] According to the geodetic coordinate system coordinates of the center point, calculate the longitude and latitude coordinates of the center point.

[0029] Through the above technical solution, it is applied to artificial intelligence, intelligent transportation, vehicle-road collaborative roadside perception, and computer vision in computer technology to achieve the effect of improving the target positioning accuracy.

[0030] Further, S21 includes:

[0031] Search for image positions at all scales, and identify potential interest points that are invariant to scale, rotation, and brightness;

[0032] At each candidate position, determine the position and scale through a fitting refinement model, and the selection basis is the stability degree;

[0033] Based on the local gradient direction of the image, assign several directions to each key point position;

[0034] In the neighborhood around each key point, measure the local gradient of the image at the selected scale.

[0035] Further, S22 includes:

[0036] Initialization step: Load the local feature descriptor sets of two images, denoted as descriptor set A and descriptor set B respectively, where each descriptor represents a feature point in the image;

[0037] Configure the index: Use the fast nearest neighbor search library to configure an index according to the characteristics of descriptor set A or B;

[0038] Perform nearest neighbor matching: Utilize the index to find the K nearest neighbors of each descriptor in descriptor set A in descriptor set B, where K is an integer greater than or equal to 1;

[0039] Match result processing: According to the matching results, screen high-quality matches by comparing the distance ratio between the nearest neighbor and the second nearest neighbor.

[0040] Further, configuring the index includes selecting the index type and setting the index parameters, where:

[0041] In the step of configuring the index, the user selects a suitable index type according to the type and characteristics of the descriptor, and sets the corresponding index parameters to optimize the search performance and matching accuracy.

[0042] The present invention has the following beneficial effects:

[0043] 1. The present invention can still meet the positioning accuracy requirements when the camera image is offset, and is applied to artificial intelligence, intelligent transportation, vehicle-road collaborative roadside perception, and computer vision in computer technology to achieve the effect of improving the target positioning accuracy.

[0044] 2. The present invention associates the features of the real-time collected image with the original image saved during calibration to obtain the projection transformation matrix from the collected image to the original image. After detecting the target pixel coordinates, the above matrix is first used for rectification, and then the world coordinates corresponding to the rectified pixel coordinates are calculated. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic flow chart of the method of the present invention;

[0046] Figure 2 It is a schematic diagram of the key point matching result of the collected image and the original image, where the left figure is the original image and the right figure is the collected image;

[0047] Figure 3 It is a schematic diagram of the target ground contact point detection result. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] Embodiment 1

[0050] Refer to Figure 1 , a roadside camera automatic rectification and target positioning method, including the following steps:

[0051] Collect the image of the camera,

[0052] Among them, compared with the original camera image, calculate the projection matrix, including:

[0053] Detect the key points of the collected image and the original image;

[0054] According to the key points of the collected image and the key points of the original image, calculate the projection transformation matrix from the collected image to the original image;

[0055] According to the projection transformation matrix, calculate the rectified coordinates of the key points of the collected image, and calculate the projection error with the coordinates of the key points of the original image;

[0056] According to the projection error and the preset error threshold, judge whether the projection transformation matrix meets the error accuracy requirements;

[0057] Perform a projective transformation on the pixel coordinates of the center point of the target detected in real time to obtain the corrected pixel coordinates of the center point;

[0058] Calculate the world coordinates of the center point based on the pixel coordinates of the center point.

[0059] Calculating the projective transformation matrix from the acquired image to the original image includes the following steps:

[0060] Use Scale-Invariant Feature Transform (SIFT) to detect local feature descriptors of the image;

[0061] Based on the local feature descriptors of the acquired image and the original image, use k-Nearest Neighbor (kNN) matching to obtain pairwise correspondence;

[0062] Calculate the homography matrix from the acquired image to the original image according to the kNN matching result.

[0063] Performing a projective transformation on the pixel coordinates of the center point of the target detected in real time to obtain the corrected pixel coordinates of the center point specifically includes:

[0064] Detect all tire contact points of the target;

[0065] Calculate the average coordinates based on all detected tire contact points to obtain the center point;

[0066] Perform a projective transformation on the pixel coordinates of the geometric center point according to the projective transformation matrix.

[0067] Calculating the world coordinates of the center point based on the pixel coordinates of the center point includes:

[0068] Based on the calibrated internal and external camera parameters, calculate the coordinates of the center point in the camera coordinate system;

[0069] Based on the orientation angle of the target, calculate the coordinates of the center point in the earth coordinate system;

[0070] Calculate the longitude and latitude coordinates of the center point based on the coordinates of the center point in the earth coordinate system.

[0071] Using Scale-Invariant Feature Transform (SIFT) to detect local feature descriptors of the image includes:

[0072] Search for image positions at all scales to identify potential interest points that are invariant to scale, rotation, brightness, etc.;

[0073] At each candidate position, determine the position and scale by fitting a refined model, with the selection criterion being the stability;

[0074] Based on the local gradient direction of the image, assign several directions to each key point position;

[0075] In the neighborhood around each key point, measure the local gradient of the image at the selected scale.

[0076] Based on the local feature descriptors of the acquired image and the original image, use nearest neighbor matching to obtain pairwise correspondence relationships, including:

[0077] Initialization step: Load the sets of local feature descriptors of the two images, denoted as descriptor set A and descriptor set B respectively, where each descriptor represents a feature point in the image;

[0078] Configure the index: Use a fast nearest neighbor search library to configure an index according to the characteristics of descriptor set A or B;

[0079] Perform nearest neighbor matching: Utilize the index to find the K nearest neighbors of each descriptor in descriptor set A in descriptor set B, where K is an integer greater than or equal to 1;

[0080] Process the matching results: According to the matching results, screen for high-quality matches by comparing the distance ratio between the nearest neighbor and the second nearest neighbor.

[0081] The configuration of the index includes selecting the index type and setting the index parameters, where:

[0082] In the step of configuring the index, the user can select a suitable index type according to the type and characteristics of the descriptors, and set the corresponding index parameters to optimize the search performance and matching accuracy.

[0083] Embodiment 2

[0084] Perform feature association between the real-time acquired image and the original image saved during calibration to obtain the projection transformation matrix from the acquired image to the original image. After detecting the target pixel coordinates, first use the above matrix for rectification, and then calculate the world coordinates corresponding to the rectified pixel coordinates.

[0085] Based on the above inventive concept, the present application provides a roadside camera automatic rectification method and a target positioning method, which are applied to artificial intelligence, intelligent transportation, vehicle-road collaborative roadside perception, and computer vision in computer technology to achieve the effect of improving the target positioning accuracy.

[0086] Figure 1 is a schematic diagram according to the first embodiment of the present application, as Figure 1 shown, the method includes:

[0087] Step S11: Acquire a camera image, compare it with the original camera image, and calculate the projection transformation matrix;

[0088] Step S12: According to the projection transformation matrix, perform a projection transformation on the target pixel coordinates detected in real time to obtain the rectified pixel coordinates;

[0089] Step S13: Calculate the corresponding world coordinates based on the rectified pixel coordinates.

[0090] Exemplarily, the roadside camera in this embodiment can be a bullet camera or a fisheye camera installed at an intersection. When calibrating the camera, a frame of picture is saved as the original camera image. After deploying the target detection system, each frame of the collected camera image is used as the acquired camera image.

[0091] Exemplarily, the above step S11 of comparing the acquired camera image with the original camera image to calculate the projective transformation matrix includes:

[0092] Identify the key points of the acquired camera image and the original camera image;

[0093] Match the key points of the acquired camera image and the original camera image;

[0094] Calculate the homography matrix from the acquired camera image to the original camera image according to the pairwise matching relationship.

[0095] Exemplarily, the above method of identifying the key points of the acquired camera image and the original camera image can use the SIFT (Scale Invariant Feature Transform) method to extract the key points of the image. The SIFT method is a region detection algorithm that is less affected by factors such as illumination, partial target occlusion, and noise. By searching for image positions at all scales and using the Gaussian differential function to identify the extreme points that are invariant to size and rotation. Taking the extreme points as the center, calculate the gradient and direction distribution features of the pixels within their neighborhood windows. Based on the position, scale, and direction obtained for each key point, a descriptor is established for each key point, and a set of vectors is used to describe the key point, making it invariant to various changes, such as illumination changes and perspective changes.

[0096] It should be noted that the above embodiments are only used for illustrative purposes. The SIFT method for extracting the key points of the image can be adopted in this embodiment, rather than being construed as a limitation on the method for extracting the key points of the image.

[0097] Exemplarily, for the above-mentioned key point matching of the collected camera image and the original camera image, the kNN (K-Nearest Neighbors) nearest neighbor matching method can be adopted. When using the kNN method for matching, according to the descriptor of each key point, the distance between the remaining key points and this key point is calculated. Exemplarily, the Euclidean distance is generally selected. Select k = 2 key points that are the nearest neighbors to this key point. According to the matching result, a distance threshold can be used to filter out reliable matching pairs. Exemplarily, the Euclidean distance can be used as the calculation method for the distance threshold. The physical meaning of the Euclidean distance is the offset pixel distance between the frames of the collected camera image and the original camera image.

[0098] It should be noted that the above embodiments are only used for illustrative purposes. The kNN method for matching the key points of images can be adopted in this embodiment, rather than being construed as a limitation on the method for matching the key points of images.

[0099] Among them, the feature points and matching results of the collected camera image and the original camera image can be referred to Figure 2 .

[0100] Exemplarily, for the above-mentioned calculation of the homography matrix from the collected camera image to the original camera image based on the pairwise matching relationship, it is necessary to ensure that there are no less than 4 pairs of key point matches first. The projection transformation matrix is to solve a 3X3 matrix to unidirectionally transform a set of two-dimensional coordinate points into another set of two-dimensional coordinate points. The transformation matrix can be solved by establishing an equation system using the least squares method.

[0101] Exemplarily, for the above-mentioned step S12, according to the projection transformation matrix, performing a projection transformation on the target pixel coordinates detected in real time to obtain the corrected pixel coordinates includes:

[0102] Detecting all the grounding points of the target;

[0103] Calculating the average coordinates based on all the detected grounding points to obtain the center point;

[0104] Performing a projection transformation on the pixel coordinates of the geometric center point according to the projection transformation matrix.

[0105] Among them, the detection results of the target grounding points can be referred to Figure 3 .

[0106] It is worth noting that there is a deviation between the center point of the target box and the geometric center point of the actual target object during 2D detection. In order to obtain the accurate coordinates of the geometric center point of the actual target object in the picture, the method of identifying the target grounding points can be adopted.

[0107] Exemplarily, when using the projection transformation matrix, it can be used in combination with its reprojection error. The offset of the camera screen can be ignored in a short period of time, and multiple sets of projection transformation matrices and reprojection errors will be calculated during this period. During actual rectification, the projection transformation matrix with the smallest reprojection error can be selected to perform projection transformation on the target pixel coordinates. Generally, a projection transformation matrix with a reprojection error less than 1 pixel can be selected.

[0108] Exemplarily, in step S13 above, according to the rectified pixel coordinates, the corresponding world coordinates are calculated, including:

[0109] Based on the calibrated internal and external camera parameters, calculate the coordinates of the center point in the camera coordinate system;

[0110] Based on the orientation angle of the camera, calculate the coordinates of the center point in the geodetic coordinate system;

[0111] According to the geodetic coordinate system coordinates of the center point, calculate the longitude and latitude coordinates of the center point.

[0112] Exemplarily, the internal and external camera parameter matrices can combine the coordinate conversion principle between the pixel coordinate system and the world coordinate system in the related art to convert two-dimensional coordinates and three-dimensional coordinates, so as to obtain the internal and external camera parameters.

[0113] Exemplarily, the above geodetic coordinate system can be a Gauss projection coordinate system. Based on the coordinates of the center point in the camera coordinate system below and the orientation angle h of the camera, the center point can be rotated to the camera coordinate system parallel to the Gauss projection coordinate system , and the coordinates are obtained. The conversion formula is:

[0114] ;

[0115] Based on the coordinates and the Gauss coordinates of the camera origin, the coordinates of the target in the Gauss coordinate system can be calculated.

[0116] Exemplarily, for the target coordinates in the above Gauss coordinate system, in the related art, by determining the projection parameters, calculating the projection zone number, and using the inverse transformation formula of Gauss projection to solve the longitude and latitude coordinates.

[0117] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device.

[0118] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for automatic deviation correction and target positioning of a roadside camera, characterized in that: The following steps are involved: S1, collect the image of the camera, Among them, compared with the original camera image, the projection matrix is ​​calculated, including: Detect key points of the acquired image and the original image; S2. Calculate the projection transformation matrix from the collected image to the original image according to the key points of the collected image and the key points of the original image; S3, according to the projection transformation matrix, calculate the correction coordinates of the key points of the collected image, and calculate the projection error with the coordinates of the key points of the original image; S4. Judging whether the projection transformation matrix meets the error accuracy requirement according to the projection error and the preset error threshold; S5, performing projection transformation on the pixel coordinates of the target center point detected in real time to obtain the pixel coordinates of the center point after correction; S6. Calculate the world coordinates of the center point according to the pixel coordinates of the center point; S2 includes the following steps: S21, using size-invariant feature transformation to detect local feature descriptors of the image; S22, using kNN nearest neighbor matching to obtain a pairwise correspondence relationship based on the local feature descriptors of the collected image and the original image; S23, calculating a homography matrix from the collected image to the original image based on the kNN nearest neighbor matching result; S22 includes: an initialization step: loading local feature descriptor sets of two images, respectively denoted as descriptor set A and descriptor set B, where each descriptor represents a feature point in the image; configuring an index: using a fast nearest neighbor search library, configuring an index according to the characteristics of descriptor set A or B; performing nearest neighbor matching: using the index, for each descriptor in descriptor set A, searching for its K nearest neighbors in descriptor set B, where K is an integer greater than or equal to 1; matching result processing: based on the matching results, selecting high-quality matches by comparing the distance ratio between the nearest neighbor and the next nearest neighbor.

2. The method for automatic deviation correction and target positioning of a roadside camera according to claim 1, characterized in that S5 include: Detect all tire contact points of the target; According to all the tire contact points detected, the average coordinates are calculated to obtain the center point; According to the projection transformation matrix, the pixel coordinates of the geometric center point are projected and transformed.

3. The method for automatic deviation correction and target positioning of a roadside camera according to claim 2, characterized in that S6 include: Based on the calibrated camera internal and external parameters, calculate the coordinates of the center point in the camera coordinate system; Based on the orientation angle of the target, calculate the coordinates of the center point in the geodetic coordinate system; Calculate the longitude and latitude coordinates of the center point based on the geodetic coordinate system coordinates of the center point.

4. The method for automatic deviation correction and target positioning of a roadside camera according to claim 1, characterized in that: S21 includes: Search image locations at all scales to identify potential points of interest that are invariant to scale, rotation, and brightness; At each candidate location, a refined model is fitted to determine the location and scale, selected based on the degree of stability; Based on the local gradient direction of the image, several directions are assigned to each key point position; In a neighborhood around each keypoint, the local gradient of the image is measured at a chosen scale.

5. The method for automatic deviation correction and target positioning of a roadside camera according to claim 1, characterized in that: Configuring an index involves selecting an index type and setting index parameters, where: In the step of configuring the index, the user selects an appropriate index type according to the type and characteristics of the descriptor, and sets corresponding index parameters to optimize the search performance and matching accuracy.

Images