A road monitoring camera calibration method based on road surface information

Abstract

The present invention relates to the technical field of road monitoring camera calibration, and in particular to a road monitoring camera calibration method based on road surface information. A road monitoring camera calibration method based on road surface information includes: S1: obtaining road related information and constructing a road coordinate system; S2: obtaining the regional image taken by the monitoring camera and performing partition processing on the regional image; S3: generating a second calibration object sequence according to the partition processing result; S4: matching the position of the calibration object in the image with the actual position in the road; S5: calculating the parameters of the monitoring camera according to the matching result. The present invention generates a calibration object sequence by partitioning the image; when the number of extractable feature points is insufficient, the generated calibration object sequence is replaced to ensure the accuracy and reliability of the calibration; the coordinate conversion model is determined by matching the position of the calibration object in the two coordinate systems, the parameters of the camera are calculated, and the calibration process is completed.

Description

Technical Field

[0001] The present invention relates to the technical field of road monitoring camera calibration, and in particular to a road monitoring camera calibration method based on road surface information. Background Art

[0002] With the development of transportation systems, road surveillance cameras play an increasingly important role in traffic management, accident prevention, and traffic flow analysis. In order for surveillance cameras to accurately locate events on the road and provide reliable data, the calibration of surveillance cameras becomes crucial. Traditional camera calibration methods often fail to meet the requirements of practical applications when faced with complex road environments and diverse monitoring needs. For example, when the original calibration object is defective or blocked, and based on the external environment, the surveillance camera will offset or shake, and the number of feature points in the image captured by the surveillance camera will be insufficient, resulting in inaccurate calibration accuracy.

[0003] Therefore, there is an urgent need to develop a road surveillance camera calibration method based on road surface information. Summary of the invention

[0004] In order to overcome the shortcoming of poor calibration accuracy of surveillance cameras due to insufficient image feature points, the present invention provides a road surveillance camera calibration method based on road surface information.

[0005] The technical solution of the present invention is: a road monitoring camera calibration method based on road surface information, comprising the following steps:

[0006] S1: Obtain road related information and construct a road coordinate system;

[0007] S2: Acquire the regional image captured by the surveillance camera, and perform partition processing on the regional image;

[0008] S3: generating a second calibration object sequence according to the result of the partitioning process;

[0009] S4: matching the position of the calibration object in the image with the actual position on the road;

[0010] S5: Calculate the parameters of the surveillance camera according to the matching result.

[0011] Preferably, the step of obtaining road related information and constructing a road coordinate system in S1 includes:

[0012] According to the road-related information, the starting point of the road is selected, and any point on the starting point is used as the origin. The extension direction of the road is used as the x-axis, the direction perpendicular to the road is used as the y-axis, and the direction perpendicular to the ground is used as the z-axis to establish a road coordinate system in three-dimensional space, wherein the road-related information includes but is not limited to road geometry, lane layout, road surface markings, and road facilities.

[0013] Preferably, the step of acquiring the regional image captured by the surveillance camera and performing partition processing on the regional image in S2 includes:

[0014] The surveillance camera captures regional images under different time periods and lighting conditions, and performs pre-processing operations of denoising, enhancing and geometric correction on the captured regional images;

[0015] According to the detection task of the monitoring camera, the area image is divided into a primary detection area and a secondary detection area.

[0016] Preferably, generating a second calibration object sequence according to the result of the partitioning processing in S3 includes:

[0017] According to the results of the partitioning process and combined with the road related information, the calibration objects of the primary detection area and the secondary detection area are determined by the formula:

[0018] ;

[0019] Calculate the comprehensive characteristics of the calibration object and sort the calibration objects, where S is the comprehensive characteristics of the calibration object, A is the shape score of the calibration object, B is the size score of the calibration object, C is the reflectivity score of the calibration object, and D is the stability score of the calibration object. are their respective weights;

[0020] Selecting the calibration objects by using a genetic algorithm according to the areas of the primary detection area and the secondary detection area to obtain a first calibration object sequence, wherein the first calibration object sequence is composed of the calibration objects in the primary detection area and the secondary detection area;

[0021] The calibrants in the first calibrant sequence are analyzed to generate a second calibrant sequence.

[0022] Preferably, the step of analyzing the markers in the first marker sequence to generate the second marker sequence includes:

[0023] By using the calibration object analysis model, analyzing the extractable feature points of the calibration objects in the first calibration object sequence, and calculating the total number of extractable feature points of the first calibration object sequence;

[0024] If the total number of extractable feature points is greater than or equal to the preset number of feature points, the first calibration object sequence is used as the second calibration object sequence for output;

[0025] If the total number of extractable feature points is less than the preset number of feature points, the calibration object is replaced until the total number of extractable feature points is greater than or equal to the preset number of feature points, and a second calibration object sequence is generated.

[0026] Preferably, the step of replacing the calibration object comprises:

[0027] The calibrant in the first calibrant sequence and in the main detection area is selected as the first calibrant, and the calibrant not in the first calibrant sequence and in the main detection area is selected as the second calibrant, according to the formula:

[0028] ;

[0029] Calculate the feature similarity between all first calibration objects and all second calibration objects, where: , are the i-th eigenvalues ​​of the first and second calibration objects, respectively;

[0030] Analyze the second calibration object, and obtain the number of extractable feature points of the second calibration object through the calibration object analysis model;

[0031] Compare with the first calibration object, and replace the second calibration object whose number of extractable feature points is greater than that of the first calibration object and whose similarity is greater than a preset similarity threshold;

[0032] The total number of extractable feature points in the current first calibration object sequence is calculated. If the total number of extractable feature points is less than the preset number of feature points, the secondary area calibration object is replaced.

[0033] Preferably, the replacing of the secondary area marker includes:

[0034] The calibrant in the first calibrant sequence and in the secondary detection area is selected as the third calibrant, and the calibrant not in the first calibrant sequence and in the secondary detection area is selected as the fourth calibrant, according to the formula:

[0035] ;

[0036] Calculate the feature similarity between all third calibration objects and all fourth calibration objects, where: , are the i-th eigenvalues ​​of the third and fourth calibration objects respectively;

[0037] Analyze the fourth calibration object, and obtain the number of extractable feature points of the fourth calibration object through the calibration object analysis model;

[0038] Compare with the third calibration object, and replace the fourth calibration object whose number of extractable feature points is greater than that of the third calibration object and whose similarity is greater than a preset similarity threshold;

[0039] The total number of extractable feature points in the current first calibration object sequence is calculated. If the total number of extractable feature points is less than the preset number of feature points, the region is expanded.

[0040] Preferably, the performing of regional expansion includes:

[0041] The surveillance camera rotation prediction model is used to predict the rotation of the surveillance camera. According to the prediction result, the surveillance camera is rotated, and the image of the current detection area is taken again. The image fusion technology is used for image stitching, and the stitched image is analyzed again to generate a second calibration object sequence.

[0042] Preferably, the step of matching the position of the calibration object in the image with the actual position in the road model in S4 includes:

[0043] According to the captured regional image, an image coordinate system is established with the upper left corner of the image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis;

[0044] The feature point coordinates of the calibration objects in the second calibration object sequence in the captured area image and the road model are extracted by a feature extraction method based on a pyramid network for matching.

[0045] Preferably, the step of calculating the parameters of the surveillance camera according to the matching result in S5 includes:

[0046] Select n feature points in the image, and the corresponding coordinates of the feature points in the image coordinate system are (x1, y1), (x2, y2), ..., (x n ,y n ), the corresponding coordinates in the road coordinate system are (X1, Y1, Z1), (X2, Y2, Z2),…, (X n , Y n , Z n ), generate the conversion model:

[0047] ;

[0048] Use the least squares method to transform the model to a ij Solve and obtain the conversion relationship between the image coordinate system and the road coordinate system;

[0049] The parameters of the surveillance camera are calculated according to the feature matching and coordinate conversion results.

[0050] Beneficial effects: The present invention first constructs a road coordinate system through road-related information, then partitions the image and generates a calibration object sequence using a genetic algorithm; when the number of extractable feature points is insufficient, the calibration accuracy and reliability are guaranteed by replacing the calibration object and expanding the area; the coordinate transformation model is determined by matching the position of the calibration object in the image with the actual position in the road model, the camera parameters are calculated, and the calibration process is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The present invention is a flow chart of a road monitoring camera calibration method based on road surface information. DETAILED DESCRIPTION

[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention.

[0053] Embodiment 1: A road monitoring camera calibration method based on road surface information, such as Figure 1 As shown, the following steps are included:

[0054] S1: Obtain road related information and construct a road coordinate system;

[0055] According to the road-related information, the starting point of the road is selected, and any point on the starting point is used as the origin. The extension direction of the road is used as the x-axis, the direction perpendicular to the road is used as the y-axis, and the direction perpendicular to the ground is used as the z-axis to establish a road coordinate system in three-dimensional space, wherein the road-related information includes but is not limited to road geometry, lane layout, road surface markings, and road facilities.

[0056] S2: Acquire the regional image captured by the surveillance camera, and perform partition processing on the regional image;

[0057] The surveillance camera captures regional images under different time periods and lighting conditions, and performs pre-processing operations of denoising, enhancing and geometric correction on the captured regional images;

[0058] According to the detection task of the monitoring camera, the area image is divided into a primary detection area and a secondary detection area.

[0059] It should be noted that, according to the detection task of the surveillance camera, the regional image is divided into a primary detection area and a secondary detection area. The primary detection area is an area with heavy traffic, frequent accidents or important traffic facilities, such as road intersections, highway entrances and exits, etc.; the secondary detection area is relatively less important, such as a straight section of the road or an area with relatively stable traffic.

[0060] S3: generating a second calibration object sequence according to the result of the partitioning process;

[0061] According to the results of the partitioning process and combined with the road related information, the calibration objects of the primary detection area and the secondary detection area are determined by the formula:

[0062] ;

[0063] Calculate the comprehensive characteristics of the calibration object and sort the calibration objects, where S is the comprehensive characteristics of the calibration object, A is the shape score of the calibration object, B is the size score of the calibration object, C is the reflectivity score of the calibration object, and D is the stability score of the calibration object. are their respective weights;

[0064] Selecting the calibration objects by using a genetic algorithm according to the areas of the primary detection area and the secondary detection area to obtain a first calibration object sequence, wherein the first calibration object sequence is composed of the calibration objects in the primary detection area and the secondary detection area;

[0065] The calibrants in the first calibrant sequence are analyzed to generate a second calibrant sequence.

[0066] The analyzing the markers in the first marker sequence to generate a second marker sequence includes:

[0067] By using the calibration object analysis model, analyzing the extractable feature points of the calibration objects in the first calibration object sequence, and calculating the total number of extractable feature points of the first calibration object sequence;

[0068] If the total number of extractable feature points is greater than or equal to the preset number of feature points, the first calibration object sequence is used as the second calibration object sequence for output;

[0069] If the total number of extractable feature points is less than the preset number of feature points, the calibration object is replaced until the total number of extractable feature points is greater than or equal to the preset number of feature points, and a second calibration object sequence is generated.

[0070] It should be noted that although the calibration object is selected, the feature points that can be extracted from each calibration object are different. In order to make the calibration of the surveillance camera more accurate, more feature points should be extracted. Therefore, a judgment is made here. When the total number of extractable feature points is greater than or equal to the preset number of feature points, parameter calculation is performed. Otherwise, a replacement operation is performed, where the preset number of feature points is determined based on the shooting resolution of the surveillance camera.

[0071] The step of replacing the calibration object comprises:

[0072] The calibrant in the first calibrant sequence and in the main detection area is selected as the first calibrant, and the calibrant not in the first calibrant sequence and in the main detection area is selected as the second calibrant, according to the formula:

[0073] ;

[0074] Calculate the feature similarity between all first calibration objects and all second calibration objects, where: , are the i-th eigenvalues ​​of the first and second calibration objects, respectively;

[0075] Analyze the second calibration object, and obtain the number of extractable feature points of the second calibration object through the calibration object analysis model;

[0076] Compare with the first calibration object, and replace the second calibration object whose number of extractable feature points is greater than that of the first calibration object and whose similarity is greater than a preset similarity threshold;

[0077] The total number of extractable feature points in the current first calibration object sequence is calculated. If the total number of extractable feature points is less than the preset number of feature points, the secondary area calibration object is replaced.

[0078] It should be noted that here you can set a threshold for the number of feature points in the main detection area based on the importance of the main detection area. If the total number of extractable feature points in the main detection area is greater than or equal to the threshold for the number of feature points in the main detection area, choose to continue adding or execute the next step based on the importance of the secondary detection area.

[0079] The replacing of the secondary area marker comprises:

[0080] The calibrant in the first calibrant sequence and in the secondary detection area is selected as the third calibrant, and the calibrant not in the first calibrant sequence and in the secondary detection area is selected as the fourth calibrant, according to the formula:

[0081] ;

[0082] Calculate the feature similarity between all third calibration objects and all fourth calibration objects, where: , are the i-th eigenvalues ​​of the third and fourth calibration objects respectively;

[0083] Analyze the fourth calibration object, and obtain the number of extractable feature points of the fourth calibration object through the calibration object analysis model;

[0084] Compare with the third calibration object, and replace the fourth calibration object whose number of extractable feature points is greater than that of the third calibration object and whose similarity is greater than a preset similarity threshold;

[0085] The total number of extractable feature points in the current first calibration object sequence is calculated. If the total number of extractable feature points is less than the preset number of feature points, the region is expanded.

[0086] It should be noted that, first, in the primary / secondary detection area, the calibration objects are divided into two categories, the first / third calibration objects in the first calibration object sequence, and the second / fourth calibration objects not in the sequence. The feature similarity between the first / third calibration object and the second / fourth calibration object is calculated by a specific formula, and the number of extractable feature points of the second / fourth calibration object is obtained with the help of the calibration object analysis model. Then, the first / third calibration object is replaced by the second / fourth calibration object whose number of extractable feature points is more than that of the first / third calibration object and whose similarity exceeds the preset threshold. Then, the total number of extractable feature points in the current first calibration object sequence is calculated. If the total number is less than the preset number of feature points, the area is expanded.

[0087] It should be understood that if a second / fourth calibration object with equal similarity and equal feature points appears for the same first / third calibration object, the area of ​​the figure surrounded by the second / fourth calibration object and other calibration objects in the first calibration object in the image is calculated, and the one with the larger area is selected as the second / fourth calibration object for replacement.

[0088] The regional expansion includes:

[0089] The surveillance camera rotation prediction model is used to predict the rotation of the surveillance camera. According to the prediction result, the surveillance camera is rotated, and the image of the current detection area is taken again. The image fusion technology is used for image stitching, and the stitched image is analyzed again to generate a second calibration object sequence.

[0090] It should be noted that when a surveillance camera is placed in its current position for a long time, it will be affected by the external environment. For example, the surveillance camera may shift or shake due to loose connectors or ground vibrations caused by the passage of large vehicles. On this basis, the rotation of the surveillance camera is predicted to make the stitched image more in line with the actual situation.

[0091] S4: matching the position of the calibration object in the image with the actual position on the road;

[0092] According to the captured regional image, an image coordinate system is established with the upper left corner of the image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis;

[0093] The feature point coordinates of the calibration objects in the second calibration object sequence in the captured area image and the road model are extracted by a feature extraction method based on a pyramid network for matching.

[0094] It should be noted that the feature extraction method based on the pyramid network extracts features at multiple scales. Because features at different scales are crucial to understanding images, for example, small-scale features (such as edges and textures) help understand local details, while large-scale features (such as shapes and object parts) help identify overall structures. By using the feature extraction method based on the pyramid network, features can be quickly extracted at lower resolutions and then refined at high resolutions, thereby improving computational efficiency.

[0095] S5: Calculate the parameters of the surveillance camera according to the matching result.

[0096] Select n feature points in the image, and the corresponding coordinates of the feature points in the image coordinate system are (x1, y1), (x2, y2), ..., (x n ,y n ), the corresponding coordinates in the road coordinate system are (X1, Y1, Z1), (X2, Y2, Z2),…, (X n , Y n , Z n ), generate the conversion model:

[0097] ;

[0098] Use the least squares method to transform the model to a ij Solve and obtain the conversion relationship between the image coordinate system and the road coordinate system;

[0099] The parameters of the surveillance camera are calculated according to the feature matching and coordinate conversion results.

[0100] Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation shall fall within the protection scope of the present invention.

Claims

1. A road monitoring camera calibration method based on road surface information, characterized in that: The following steps are involved: S1: Obtain road related information and construct a road coordinate system; S2: Acquire the regional image captured by the surveillance camera, and perform partition processing on the regional image; S3: generating a second calibration object sequence according to the partition processing result; S4: matching the position of the calibration object in the image with the actual position on the road; S5: Calculating the parameters of the surveillance camera according to the matching result; The step of generating a second calibration object sequence according to the partition processing result in S3 includes: According to the partition processing results, combined with road related information, the calibration objects of the main detection area and the secondary detection area are determined by the formula: ； Calculate the comprehensive characteristics of the calibration object and sort the calibration objects, where S is the comprehensive characteristics of the calibration object, A is the shape score of the calibration object, B is the size score of the calibration object, C is the reflectivity score of the calibration object, and D is the stability score of the calibration object. are their respective weights; Selecting the calibration objects by using a genetic algorithm according to the areas of the primary detection area and the secondary detection area to obtain a first calibration object sequence, wherein the first calibration object sequence is composed of the calibration objects in the primary detection area and the secondary detection area; Analyzing the calibrants in the first calibrant sequence to generate a second calibrant sequence; The analyzing the markers in the first marker sequence to generate a second marker sequence includes: By using the calibration object analysis model, analyzing the extractable feature points of the calibration objects in the first calibration object sequence, and calculating the total number of extractable feature points of the first calibration object sequence; If the total number of extractable feature points is greater than or equal to the preset number of feature points, the first calibration object sequence is used as the second calibration object sequence for output; If the total number of extractable feature points is less than the preset number of feature points, the calibration object is replaced until the total number of extractable feature points is greater than or equal to the preset number of feature points, thereby generating a second calibration object sequence; The step of replacing the calibration object comprises: The calibrant in the first calibrant sequence and in the main detection area is selected as the first calibrant, and the calibrant not in the first calibrant sequence and in the main detection area is selected as the second calibrant, according to the formula: ； Calculate the feature similarity between all first calibration objects and all second calibration objects, where: , are the i-th eigenvalues ​​of the first and second calibration objects, respectively; Analyze the second calibration object, and obtain the number of extractable feature points of the second calibration object through the calibration object analysis model; Compare with the first calibration object, and replace the second calibration object whose number of extractable feature points is greater than that of the first calibration object and whose similarity is greater than a preset similarity threshold; Calculate the total number of extractable feature points in the current first calibration object sequence, and if the total number of extractable feature points is less than the preset number of feature points, replace the secondary area calibration object; The replacing of the secondary area marker comprises: The calibrant in the first calibrant sequence and in the secondary detection area is selected as the third calibrant, and the calibrant not in the first calibrant sequence and in the secondary detection area is selected as the fourth calibrant, according to the formula: ； Calculate the feature similarity between all third calibration objects and all fourth calibration objects, where: , are the i-th eigenvalues ​​of the third and fourth calibration objects respectively; Analyze the fourth calibration object, and obtain the number of extractable feature points of the fourth calibration object through the calibration object analysis model; Compare with the third calibration object, and replace the fourth calibration object whose number of extractable feature points is greater than that of the third calibration object and whose similarity is greater than a preset similarity threshold; Calculate the total number of extractable feature points in the current first calibration object sequence, and if the total number of extractable feature points is less than the preset number of feature points, perform region expansion; The regional expansion includes: The surveillance camera rotation prediction model is used to predict the rotation of the surveillance camera. According to the prediction result, the surveillance camera is rotated, and the image of the current detection area is taken again. The image fusion technology is used for image stitching, and the stitched image is analyzed again to generate a second calibration object sequence.

2. A road monitoring camera calibration method based on road surface information as claimed in claim 1, characterized in that: Acquiring road-related information and constructing a road coordinate system as described in S1 includes: According to the road-related information, the starting point of the road is selected, and any point on the starting point is used as the origin. The extension direction of the road is used as the x-axis, the direction perpendicular to the road is used as the y-axis, and the direction perpendicular to the ground is used as the z-axis. A road coordinate system is established in three-dimensional space, wherein the road-related information includes road geometry, lane layout, road surface markings, and road facilities.

3. A road monitoring camera calibration method based on road surface information as claimed in claim 1, characterized in that: The step of obtaining the regional image captured by the surveillance camera and performing partition processing on the regional image in S2 includes: The surveillance camera captures regional images under different time periods and lighting conditions, and performs pre-processing operations of denoising, enhancing and geometric correction on the captured regional images; According to the detection task of the monitoring camera, the area image is divided into a primary detection area and a secondary detection area.

4. A road monitoring camera calibration method based on road surface information as claimed in claim 1, characterized in that: The step of matching the position of the calibration object in the image with the actual position in the road model in S4 includes: According to the captured regional image, an image coordinate system is established with the upper left corner of the image as the origin, the horizontal rightward direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis; The feature point coordinates of the calibration objects in the second calibration object sequence in the captured regional image and the road model are extracted by a feature extraction method based on a pyramid network for matching.

5. A road monitoring camera calibration method based on road surface information as claimed in claim 4, characterized in that: Calculating the parameters of the surveillance camera according to the matching result in S5 includes: Select n feature points in the image, and the corresponding coordinates of the feature points in the image coordinate system are (x1, y1), (x2, y2), ..., (x n ,y n ), the corresponding coordinates in the road coordinate system are (X1, Y1, Z1), (X2, Y2, Z2),…, (X n , Y n , Z n ), generate the conversion model: ； Use the least squares method to transform the model to a ij Solve and obtain the conversion relationship between the image coordinate system and the road coordinate system; The parameters of the surveillance camera are calculated according to the feature matching and coordinate conversion results.

Images