Roadside camera automatic deviation correction and target positioning method

Abstract

The invention discloses an automatic deviation correction and target positioning method for a roadside camera, and relates to the technical field of intelligent traffic control, and the method comprises the following steps: collecting an image of the camera, comparing the image with an original image of the camera, calculating a projection matrix, and calculating a projection transformation matrix from the collected image to the original image; according to the projection transformation matrix, calculating deviation correction coordinates of the key points of the acquired image, and calculating projection errors with coordinates of the key points of the original image; according to the projection error and a preset error threshold value, whether the projection transformation matrix meets an error precision requirement or not is judged; projection transformation is carried out on target center point pixel coordinates detected in real time, and center point pixel coordinates after deviation correction are obtained; and world coordinates of the central point are calculated. The method can still meet the positioning precision requirement when the camera picture deviates, and is applied to artificial intelligence, intelligent traffic, vehicle-road cooperation roadside perception and computer vision in the computer technology, so as to achieve the effect of improving the target positioning precision.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic control technology, and in particular to a method for automatic deviation correction and target positioning of a roadside camera. Background Art

[0002] Roadside cameras are a crucial foundational component of vehicle-infrastructure cooperative systems. By detecting and locating traffic participants on the road, they provide essential data for various vehicle-infrastructure cooperative applications. In computer vision applications, to determine the position of a point on the surface of a spatial object in the world coordinate system, camera calibration is required. This process converts the target pixel coordinates into world coordinates. The camera's extrinsic parameter matrix (R, t) consists of rotation and translation matrices, which are related to the camera's installation pose.

[0003] However, due to various factors, the camera's position changes slowly over time. Typically, the camera image can shift by up to 10 pixels over the course of a day. This results in low target positioning accuracy in practical applications if the pixel-to-world coordinate conversion parameters obtained during calibration are directly used. Target positioning accuracy is an extremely important perception performance metric. Sending target data with poor positioning accuracy to an autonomous vehicle can cause the vehicle to make incorrect judgments. However, due to common installation methods, the camera image can change slightly over time, resulting in an offset from the image obtained during camera calibration. Using the calibration parameters without correction will reduce positioning accuracy.

[0004] Therefore, the present invention proposes a method for automatic deviation correction and target positioning of a roadside camera. Summary of the Invention

[0005] The present invention provides a method for automatic deviation correction and target positioning of a roadside camera, which helps solve the problems mentioned in the above background technology.

[0006] The present application provides a method for automatic deviation correction and target positioning of a roadside camera, which adopts the following technical solution: A method for automatic deviation correction and target positioning of a roadside camera, comprising the following steps: S1, collect the image of the camera, Among them, compared with the original camera image, the projection matrix is ​​calculated, including: Detect key points of the captured image and the original image; S2. Calculate the projection transformation matrix from the captured image to the original image based on the key points of the captured image and the key points of the original image; S3. Calculate the correction coordinates of the key points of the captured image according to the projection transformation matrix, and calculate the projection error with the coordinates of the key points of the original image; S4. Judging whether the projection transformation matrix meets the error accuracy requirement based on the projection error and a preset error threshold; S5. Performing projection transformation on the pixel coordinates of the target center point detected in real time to obtain the pixel coordinates of the center point after correction; S6. Calculate the world coordinates of the center point according to the pixel coordinates of the center point.

[0007] Through the above technical solution, the real-time captured image is feature-associated with the original image saved during calibration to obtain the projection transformation matrix from the captured image to the original image. After the target pixel coordinates are detected, the above matrix is ​​first used to correct the deviation, and then the world coordinates corresponding to the corrected pixel coordinates are calculated.

[0008] Furthermore, S2 includes the following steps: S21, using size-invariant feature transformation to detect local feature descriptors of the image; S22, using kNN nearest neighbor matching to obtain pairwise correspondences based on local feature descriptors of the collected image and the original image; S23. Calculate the homography matrix from the captured image to the original image based on the kNN nearest neighbor matching result.

[0009] Furthermore, S3 includes: Detect all tire contact points of the target; Calculate the average coordinates of all tire contact points detected to obtain the center point; According to the projection transformation matrix, the pixel coordinates of the geometric center point are projected and transformed.

[0010] The above technical solution allows the use of projection transformation matrices in conjunction with their reprojection errors. The camera image offset can be neglected for a short period of time, during which multiple sets of projection transformation matrices and reprojection errors are calculated. During actual correction, the projection transformation matrix with the smallest reprojection error is selected for the target pixel coordinates. Generally, a projection transformation matrix with a reprojection error of less than one pixel is selected.

[0011] Furthermore, S6 includes: Based on the calibrated camera internal and external parameters, calculate the coordinates of the center point in the camera coordinate system; Based on the target's heading angle, calculate the coordinates of the center point in the geodetic coordinate system; Calculate the latitude and longitude coordinates of the center point based on the geodetic coordinate system coordinates of the center point.

[0012] Through the above technical solutions, artificial intelligence, intelligent transportation, vehicle-road cooperative roadside perception and computer vision in computer technology can be applied to achieve the effect of improving target positioning accuracy.

[0013] Furthermore, S21 includes: Search image locations at all scales to identify potential points of interest that are invariant to scale, rotation, and brightness; At each candidate location, a refined model is fitted to determine the location and scale, with the selection based on the degree of stability. Based on the local gradient direction of the image, several directions are assigned to each key point position; In a neighborhood around each keypoint, the local gradient of the image is measured at a selected scale.

[0014] Furthermore, S22 includes: Initialization step: Load the local feature descriptor sets of two images, denoted as descriptor set A and descriptor set B, where each descriptor represents a feature point in the image; Configure index: Use the fast nearest neighbor search library to configure an index based on the characteristics of the descriptor set A or B; Perform nearest neighbor matching: For each descriptor in descriptor set A, find its K nearest neighbors in descriptor set B using the index, where K is an integer greater than or equal to 1. Matching result processing: Based on the matching results, high-quality matches are screened by comparing the distance ratio between the nearest neighbor and the next nearest neighbor.

[0015] Furthermore, configuring an index involves selecting an index type and setting index parameters, where: In the index configuration step, the user selects an appropriate index type based on the type and characteristics of the descriptor and sets the corresponding index parameters to optimize search performance and matching accuracy.

[0016] The present invention has the following beneficial effects: 1. The present invention can still meet the positioning accuracy requirements when the camera image is offset, and is applied to artificial intelligence, intelligent transportation, vehicle-road cooperative roadside perception and computer vision in computer technology to achieve the effect of improving target positioning accuracy.

[0017] 2. The present invention performs feature association between the real-time captured image and the original image saved during calibration to obtain a projection transformation matrix from the captured image to the original image. After detecting the target pixel coordinates, the above matrix is ​​first used to perform deviation correction, and then the world coordinates corresponding to the pixel coordinates after correction are calculated. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the process of the present invention; Figure 2 Schematic diagram of the key point matching results of the collected image and the original image, where the left image is the original image and the right image is the collected image; Figure 3 Schematic diagram of the target grounding point detection results. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Example 1 Reference Figure 1 , a method for automatic deviation correction and target positioning of a roadside camera, comprising the following steps: Capture images from the camera, Among them, compared with the original camera image, the projection matrix is ​​calculated, including: Detect key points of the captured image and the original image; Calculate the projection transformation matrix from the captured image to the original image based on the key points of the captured image and the key points of the original image; According to the projection transformation matrix, the correction coordinates of the key points of the captured image are calculated, and the projection error is calculated with the coordinates of the key points of the original image; According to the projection error and the preset error threshold, determine whether the projection transformation matrix meets the error accuracy requirements; Perform projection transformation on the pixel coordinates of the target center point detected in real time to obtain the pixel coordinates of the center point after correction; The world coordinates of the center point are calculated according to the pixel coordinates of the center point.

[0021] Calculating the projection transformation matrix from the captured image to the original image includes the following steps: Use size-invariant feature transformation to detect local feature descriptors of the image; Based on the local feature descriptors of the collected image and the original image, kNN nearest neighbor matching is used to obtain the pairwise correspondence; According to the kNN nearest neighbor matching results, the homography matrix of the collected image to the original image is calculated.

[0022] The pixel coordinates of the target center point detected in real time are projected and transformed to obtain the corrected pixel coordinates of the center point. Specifically, the following are included: Detect all tire contact points of the target; Calculate the average coordinates of all tire contact points detected to obtain the center point; According to the projection transformation matrix, the pixel coordinates of the geometric center point are projected and transformed.

[0023] According to the pixel coordinates of the center point, the world coordinates of the center point are calculated, including: Based on the calibrated camera internal and external parameters, calculate the coordinates of the center point in the camera coordinate system; Based on the target's heading angle, calculate the coordinates of the center point in the geodetic coordinate system; Calculate the latitude and longitude coordinates of the center point based on the geodetic coordinate system coordinates of the center point.

[0024] Using invariant feature transformation, the local feature descriptors of the image are detected, including: Search image locations at all scales to identify potential points of interest that are invariant to scale, rotation, brightness, etc. At each candidate location, a refined model is fitted to determine the location and scale, with the selection based on the degree of stability. Based on the local gradient direction of the image, several directions are assigned to each key point position; In a neighborhood around each keypoint, the local gradient of the image is measured at a selected scale.

[0025] Based on the local feature descriptors of the captured image and the original image, the pairwise correspondence is obtained using nearest neighbor matching, including: Initialization step: Load the local feature descriptor sets of two images, denoted as descriptor set A and descriptor set B, where each descriptor represents a feature point in the image; Configure index: Use the fast nearest neighbor search library to configure an index based on the characteristics of the descriptor set A or B; Perform nearest neighbor matching: For each descriptor in descriptor set A, find its K nearest neighbors in descriptor set B using the index, where K is an integer greater than or equal to 1. Matching result processing: Based on the matching results, high-quality matches are screened by comparing the distance ratio between the nearest neighbor and the next nearest neighbor.

[0026] Index configuration includes selecting the index type and setting index parameters, including: In the index configuration step, users can select the appropriate index type based on the type and characteristics of the descriptor and set the corresponding index parameters to optimize search performance and matching accuracy.

[0027] Example 2 The real-time captured image is feature-correlated with the original image saved during calibration to obtain the projection transformation matrix from the captured image to the original image. After the target pixel coordinates are detected, the above matrix is ​​first used for correction, and then the world coordinates corresponding to the corrected pixel coordinates are calculated.

[0028] Based on the above-mentioned inventive concept, the present application provides a roadside camera automatic deviation correction method and a target positioning method, which are applied to artificial intelligence, intelligent transportation, vehicle-road collaborative roadside perception and computer vision in computer technology to achieve the effect of improving target positioning accuracy.

[0029] Figure 1 is a schematic diagram according to the first embodiment of the present application, such as Figure 1 As shown, the method includes: Step S11: Capture the camera image, compare it with the original camera image, and calculate the projection transformation matrix; Step S12: performing a projection transformation on the target pixel coordinates detected in real time according to the projection transformation matrix to obtain the pixel coordinates after correction; Step S13: Calculate the corresponding world coordinates based on the corrected pixel coordinates.

[0030] For example, the roadside camera in this embodiment can be a gun-mounted camera or fisheye camera installed at an intersection. During camera calibration, a single frame is saved as the original camera image. After the target detection system is deployed, each frame of the captured camera image is used as the captured camera image.

[0031] Exemplarily, the above step S11, comparing the captured camera image with the original camera image and calculating the projection transformation matrix, includes: Identify key points of the acquired camera image and the original camera image; Match key points of the acquired camera image and the original camera image; The homography matrix from the acquired camera image to the original camera image is calculated based on the pairwise matching relationship.

[0032] For example, the key points of the captured camera image and the original camera image can be extracted using the SIFT (Scale Invariant Feature Transform) method. The SIFT method is a region detection algorithm that is less affected by illumination, partial object occlusion, noise, and other factors. By searching image positions at all scales, the Gaussian differential function is used to identify extreme points that are invariant to size and rotation. The gradient and directional distribution characteristics of the pixels within the neighborhood window centered at the extreme point are calculated. Based on the position, scale, and orientation of each key point, a descriptor is established for each key point, describing the key point using a set of vectors to ensure that it remains stable despite various changes, such as changes in illumination or viewing angle.

[0033] It should be noted that the above embodiment is only used for exemplary explanation. The SIFT method for extracting key points of an image can be adopted in this embodiment, and it should not be understood as a limitation on the method for extracting key points of an image.

[0034] For example, the key point matching between the captured camera image and the original camera image can be performed using the kNN (K-Nearest Neighbors) nearest neighbor matching method. During matching, the kNN method calculates the distance between each key point and the remaining key points based on the descriptor of that key point. For example, the Euclidean distance is generally used. The k=2 key points that are the nearest neighbors to the key point are selected. Based on the matching results, a distance threshold can be used to filter out reliable matching pairs. For example, the Euclidean distance can be used as the distance threshold calculation method. The physical meaning of the Euclidean distance is the offset pixel distance between the captured camera image and the original camera image.

[0035] It should be noted that the above embodiment is only used for exemplary explanation. The kNN method for matching key points of an image can be adopted in this embodiment, and it should not be understood as a limitation on the method for matching key points of an image.

[0036] The feature points and matching results of the acquired camera image and the original camera image can be found in Figure 2 .

[0037] For example, calculating the homography matrix from the captured camera image to the original camera image based on pairwise matching requires ensuring that at least four keypoint pairs match. The projective transformation matrix involves solving a 3x3 matrix to unidirectionally transform one set of two-dimensional coordinate points to another set of two-dimensional coordinate points. This transformation matrix can be solved using a least squares method by establishing a system of equations.

[0038] Exemplarily, the above step S12, performing a projection transformation on the target pixel coordinates detected in real time according to the projection transformation matrix to obtain the corrected pixel coordinates, includes: Detect all grounding points of the target; Based on all the detected grounding points, calculate the average coordinates and obtain the center point; According to the projection transformation matrix, the pixel coordinates of the geometric center point are projected and transformed.

[0039] The target grounding point detection results can be found in Figure 3 .

[0040] It is worth noting that during 2D detection, there is a deviation between the center point of the target frame and the geometric center point of the target object. In order to obtain the precise coordinates of the geometric center point of the target object in the image, the target ground point can be identified.

[0041] For example, when using a projection transformation matrix, its reprojection error can be used in conjunction with its use. The offset of the camera image can be ignored for a short period of time, during which multiple sets of projection transformation matrices and reprojection errors are calculated. During actual correction, the projection transformation matrix with the smallest reprojection error can be selected to perform the projection transformation on the target pixel coordinates. Generally, a projection transformation matrix with a reprojection error of less than 1 pixel can be selected.

[0042] Exemplarily, the above step S13, calculating the corresponding world coordinates based on the corrected pixel coordinates, includes: Based on the calibrated camera internal and external parameters, calculate the coordinates of the center point in the camera coordinate system; Based on the camera's orientation angle, calculate the coordinates of the center point in the geodetic coordinate system; Calculate the latitude and longitude coordinates of the center point based on the geodetic coordinate system coordinates of the center point.

[0043] For example, the camera's intrinsic and extrinsic parameter matrix can be combined with the coordinate conversion principle between the pixel coordinate system and the world coordinate system in related technologies to convert the two-dimensional coordinates and the three-dimensional coordinates, thereby obtaining the camera's intrinsic and extrinsic parameters.

[0044] For example, the above-mentioned geodetic coordinate system can be a Gaussian projection coordinate system. The coordinates below and the camera's orientation angle h, the center point can be rotated to the camera coordinate system parallel to the Gaussian projection coordinate system , get the coordinates , the conversion formula is: ; Based on coordinates And the Gaussian coordinates of the camera origin, the coordinates of the target in the Gaussian coordinate system can be calculated .

[0045] For example, in the related art, the target coordinates in the Gaussian coordinate system are obtained by determining the projection parameters, calculating the projection band number, and using the inverse transformation formula of the Gaussian projection to solve the longitude and latitude coordinates.

[0046] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0047] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for automatic deviation correction and target positioning of a roadside camera, characterized in that: The following steps are involved: S1, collect the image of the camera, Among them, compared with the original camera image, the projection matrix is ​​calculated, including: Detect key points of the acquired image and the original image; S2. Calculate the projection transformation matrix from the collected image to the original image according to the key points of the collected image and the key points of the original image; S3, according to the projection transformation matrix, calculate the correction coordinates of the key points of the collected image, and calculate the projection error with the coordinates of the key points of the original image; S4. Judging whether the projection transformation matrix meets the error accuracy requirement according to the projection error and the preset error threshold; S5, performing projection transformation on the pixel coordinates of the target center point detected in real time to obtain the pixel coordinates of the center point after correction; S6. Calculate the world coordinates of the center point according to the pixel coordinates of the center point.

2. The method for automatic deviation correction and target positioning of a roadside camera according to claim 1, characterized in that: S2 includes the following steps: S21, using size-invariant feature transformation to detect local feature descriptors of the image; S22, according to the local feature descriptors of the collected image and the original image, using kNN nearest neighbor matching to obtain a pairwise correspondence relationship; S23. Calculate the homography matrix from the collected image to the original image according to the kNN nearest neighbor matching result.

3. The method for automatic deviation correction and target positioning of a roadside camera according to claim 1, characterized in that S5 include: Detect all tire contact points of the target; According to all the tire contact points detected, the average coordinates are calculated to obtain the center point; According to the projection transformation matrix, the pixel coordinates of the geometric center point are projected and transformed.

4. The method for automatic deviation correction and target positioning of a roadside camera according to claim 3, characterized in that S6 include: Based on the calibrated camera internal and external parameters, calculate the coordinates of the center point in the camera coordinate system; Based on the orientation angle of the target, calculate the coordinates of the center point in the geodetic coordinate system; Calculate the longitude and latitude coordinates of the center point based on the geodetic coordinate system coordinates of the center point.

5. The method for automatic deviation correction and target positioning of a roadside camera according to claim 2, characterized in that: S21 includes: Search image locations at all scales to identify potential points of interest that are invariant to scale, rotation, and brightness; At each candidate location, a refined model is fitted to determine the location and scale, selected based on the degree of stability; Based on the local gradient direction of the image, several directions are assigned to each key point position; In a neighborhood around each keypoint, the local gradient of the image is measured at a chosen scale.

6. The method for automatic deviation correction and target positioning of a roadside camera according to claim 2, characterized in that: S22 includes: Initialization step: load the local feature descriptor sets of the two images, denoted as descriptor set A and descriptor set B, where each descriptor represents a feature point in the image; Configure index: Use the fast nearest neighbor search library to configure an index based on the characteristics of the descriptor set A or B; Perform nearest neighbor matching: For each descriptor in descriptor set A, find its K nearest neighbors in descriptor set B using the index, where K is an integer greater than or equal to 1. Matching result processing: Based on the matching results, high-quality matches are screened by comparing the distance ratio between the nearest neighbor and the second nearest neighbor.

7. The method for automatic deviation correction and target positioning of a roadside camera according to claim 6, characterized in that: Configuring an index involves selecting an index type and setting index parameters, where: In the step of configuring the index, the user selects an appropriate index type according to the type and characteristics of the descriptor, and sets corresponding index parameters to optimize the search performance and matching accuracy.

Images