Ancient road and gardening ecological dynamic monitoring method and device based on binocular camera

Abstract

The application relates to a method and device for monitoring an ancient road and a garden ecological dynamic based on binocular camera shooting, and belongs to the technical field of photogrammetry and computer vision. The method comprises the following steps: collecting multi-focus section images of the ancient road and the surrounding ecology; inversely calculating camera internal parameters of the corresponding focus section, and performing zoom camera endless zoom distortion calibration; using a collinear equation of binocular images and three-dimensional coordinates of control points to solve camera external orientation elements; taking a reference frame as a reference, calculating a relative offset of a target region in a continuous frame through sub-pixel accuracy least square matching, converting the ground resolution into a real ground deformation variable, and dynamically monitoring ancient road and ancient tree deformation variables; extracting ecological elements through image segmentation, analyzing coverage and texture changes based on a time sequence, and monitoring garden ecological changes. The application realizes joint monitoring of ancient road structure deformation, ancient tree posture change and surrounding ecological state, and improves the intelligent and fine level of cultural site protection.

Description

TECHNICAL FIELD

[0001] The application relates to a method and device for monitoring ancient roads and garden ecologies based on binocular camera shooting, and belongs to the technical field of photogrammetry and computer vision. BACKGROUND

[0002] With the increasing awareness of cultural heritage protection, more and more historical ancient roads, ancient trees and their surrounding garden ecologies are included in the scope of key monitoring and protection. However, such sites are usually distributed in complex outdoor environments and are affected by natural weathering, geological changes, ecological disturbances and human activities for a long time, and there are various risks such as structural damage, ground deformation and vegetation degradation, so it is urgent to establish an efficient and refined dynamic monitoring mechanism.

[0003] Existing deformation monitoring methods, such as total station, GNSS or manual measurement, usually have problems such as high cost, long cycle, inconvenience to lay out, and some devices need to be implanted into the site, which will cause contact damage, and it is difficult to meet the needs of continuous monitoring of large area, long term and non-contact. At the same time, the monitoring of the garden ecology (such as grass coverage, water change, bare soil, etc.) around the site mostly relies on remote sensing or regular patrol, and lacks high-frequency, visual and on-site sensing capabilities.

[0004] In recent years, photogrammetry and computer vision technology has developed rapidly, especially the stereo vision system based on binocular camera, which has the advantages of simple structure, strong real-time performance and strong three-dimensional reconstruction capability. Therefore, it is urgent to develop a method for monitoring the dynamic of ancient roads and garden ecologies to improve the intelligent and refined level of cultural site protection. SUMMARY

[0005] In order to solve the above problems, the application provides a method and device for monitoring the dynamic of ancient roads and garden ecologies based on binocular camera shooting, which can realize the joint monitoring of the structural deformation of ancient roads, the posture change of ancient trees and the surrounding ecological state, and improve the intelligent and refined level of cultural site protection.

[0006] The technical scheme adopted by the application to solve the technical problems is:

[0007] In a first aspect, the application provides a method for monitoring the dynamic of ancient roads and garden ecologies based on binocular camera shooting, which includes the following steps:

[0008] Step S1: collecting multi-focus segment images of ancient roads and surrounding ecologies by using binocular camera shooting;

[0009] Step S2: correcting the distortion of the original images by 1x focal length distortion parameter calibration, calculating the distortion amount under any multiple by combining the strong correlation matching of different focal length images, and inversely solving the camera intrinsic parameters of the corresponding focal length to realize the zoom camera zoom distortion calibration.

[0010] Step S3: Based on the minimum shooting field of view, set up no less than 4 control points, and use the collinearity equation of the binocular images and the three-dimensional coordinates of the control points to solve for the camera exterior orientation elements.

[0011] Step S4: Using the reference frame as a reference, the relative offset of the target area in consecutive frames is calculated by least squares matching with subpixel precision. Combined with the ground resolution, it is converted into real terrain variables to dynamically monitor the ancient road and ancient tree shape variables.

[0012] Step S5 involves extracting ecological elements such as vegetation, water bodies, and bare soil through image segmentation, analyzing their coverage and texture changes based on time series data, monitoring horticultural ecological changes, and generating degradation early warnings.

[0013] As one possible implementation of this embodiment, step S2 includes:

[0014] Step S21: Use the calibration field to check the distortion parameters of the optical lens of the binocular camera at 1x focal length, and obtain the distortion parameters at 1x focal length.

[0015] Step S22: For the same shooting area, since the shooting range of the nx focusing image is smaller than that of the 1x focusing image, the corresponding points on the nx focusing image and the 1x focusing image are identified by the least squares algorithm with full pixel matching. The deviation values ​​dΔx and dΔy of the corresponding points in the nx focusing image relative to the 1x focusing image are calculated. Combined with the obtained distortion variables Δx and Δy under the 1x focal length, the distortion variables Δx′ and Δy′ of the corresponding image points on the nx focusing image are solved.

[0016] Step S23: Based on the geometric relationship between the pixel radius and focal length after distortion correction, the focal length f at nx zoom is calculated using the ratio m0 of the pixel radius of corresponding points on the 1x zoom image and the nx zoom image. n ;

[0017] Step S24: Based on the distortion variables △x′ and △y′ on the obtained n-fold focused image, perform distortion correction on each pixel in the original image and output a distortion-free image.

[0018] As one possible implementation of this embodiment, the formula for calculating the 1x focal length distortion parameter is:

[0019] ,

[0020] In the formula, Δx and Δy are the distortion variables at 1 focal length; x0 and y0 are the principal points; r represents the distance from the image point to the principal point, expressed by the formula... The calculation is obtained; x and y are the image point coordinates in the image coordinate system; k1 and k2 are the radial distortion parameters of the lens; p1 and p2 are the tangential distortion parameters of the lens; a and b are the non-square correction coefficients of the pixels;

[0021] The distortion amounts △x' and △y' of the corresponding image points on the n times focusing image are:

[0022] ,

[0023] Wherein, dΔx and dΔy are the deviation values of the homonymous points on the n times focusing image and the 1 times focusing image.

[0024] As a possible implementation manner of the embodiment, the least square method algorithm of the full-pixel participation matching is used to determine the corresponding relationship of each pixel on the 1 times focusing image and the n times focusing image, so as to accurately identify the homonymous points.

[0025] As a possible implementation manner of the embodiment, the ratio m0 of the pixel radii of the homonymous points on the 1 times focusing image and the n times focusing image is the ratio of the distance r1 of the homonymous points on the 1 times focusing image to the image principal point to the distance r n of the homonymous points on the n times focusing image, that is, m0=r1 / r n ; the focal length f n of the n times focusing satisfies f n =f / m0, wherein f is the focal length of the 1 times focusing.

[0026] As a possible implementation manner of the embodiment, the formula for correcting the distortion of each pixel in the original image is:

[0027]

[0028] In the formula, x i ' and y i ' are the coordinates obtained by correcting the distortion of the i pixel, x i and y i are the original coordinates of the i pixel on the n times focusing image, △x i ' and △y i ' are the distortion amounts on the n times focusing image.

[0029] As a possible implementation manner of the embodiment, the step S3 comprises:

[0030] Step S31, at least four control points with known three-dimensional coordinates are selected and laid out, and it is ensured that the corresponding positions thereof can be accurately identified in the image, and the camera exterior orientation elements are solved;

[0031] Step S32: Distortion correction is performed on the original image acquired by the binocular camera to obtain a distortion-free imaging model; based on the collinearity equation of photogrammetry, the control points selected in step S1 and their corresponding image points in the image are used to establish error equations to inversely calculate the 12 exterior orientation elements of the binocular camera, of which at least 3 ground control points are required, and preferably 4 ground control points are used for inspection.

[0032] Step S33: Based on the exterior orientation elements obtained in step S22, the equations are established using the left and right images from the binocular camera through the inverse transformation of the collinearity equations to solve for the three-dimensional coordinates of any point on the ancient road and ancient tree. Based on the obtained three-dimensional coordinates, the absolute distance between two points on the ancient road or ancient tree is calculated. Through strong correlation matching of multiple sequence image pairs, the dynamic relative change is obtained to determine the size elements of the ancient road / ancient tree.

[0033] As one possible implementation of this embodiment, the specific process of setting up the control points in step S31 is as follows:

[0034] Pre-set control points based on the minimum shooting field of view, that is, plan according to the shooting angle at the minimum zoom or maximum focal length, to ensure that the image always contains no less than 4 control points under any focal length setting.

[0035] Priority should be given to natural or artificial targets with stable features as control points; in areas with features such as stone-built ancient roads, artificially excavated points of suspended ancient roads, and road intersections, these should be prioritized as control points; in areas lacking obvious features, artificial marker targets should be deployed to replace natural features.

[0036] The control point distribution structure meets the geometric stability requirements, that is, it covers the perimeter and central area of ​​the shooting field of view, avoids collinearity and coplanarity, and forms a good geometric distribution structure.

[0037] As one possible implementation of this embodiment, in step S32, the expression for the collinearity equation is:

[0038] (6)

[0039] Where (x, y) are the coordinates of the image point, The coordinates of the camera's center of focus are ( ) represents the coordinates of a point on the ground, a i b i and c i The image is composed of 9 direction cosines formed by the 3 exterior azimuth elements, i=1, 2, 3, and focal length. f Through formula f n =f / m0 is obtained, ( x 0, y 0) is the value for adjusting the image calibration by 1x.

[0040] As a possible implementation manner of the embodiment, in step S33, the expression of the coordinates (X, Y, Z) of any point on the ground is obtained by inverse transformation of the collinear equation as follows:

[0041] (7)

[0042] wherein a i , b i , c i (i=1, 2, 3) are nine direction cosines composed of three exterior orientation elements of the image, is the coordinate of the photograph center of the camera.

[0043] As a possible implementation manner of the embodiment, in step S33, the calculation formula of the absolute distance between two points is as follows:

[0044] (8)

[0045] wherein (X1, Y1, Z1) and (X2, Y2, Z2) are three-dimensional coordinates of two points on the ancient road or the ancient tree.

[0046] As a possible implementation manner of the embodiment, the step S4 comprises:

[0047] In step S41, a first frame image in a video sequence containing a monitoring target is taken as a reference image, and a region of interest (ROI) of a monitoring target region in the reference image is extracted, the monitoring target region including an ancient stone road region and an ancient tree root region.

[0048] In step S42, for each frame image after the first frame in the video sequence, a sub-pixel precision least square image matching algorithm is used to search for an image block most similar to the region of interest (ROI) of the reference frame in a local region of the frame image, and a best matching position is calculated.

[0049] In step S43, a change in offset in a horizontal direction and a vertical direction between the current frame and the reference frame is recorded, and a time sequence offset model is constructed, the offset being a deformation variable or a tilt trend of the monitoring target.

[0050] In step S44, the offset is mapped to a real coordinate scale according to a pixel offset and a ground resolution, and it is judged whether there is a continuous deformation exceeding a threshold value, which is used for risk warning of ancient monument deformation.

[0051] In step S45, a plurality of frame images in the video sequence are registered and adjusted, and a false matching caused by an interference factor is suppressed, so as to improve the overall precision and robustness of the system.

[0052] As a possible implementation manner of the embodiment, in step S43, if the deformation of any two points A and B on the monitoring target is ΔD, ΔD is calculated based on the pixel coordinates of the two points A and B on the two frames of images and the ground resolution gsd:

[0053]

[0054]

[0055]

[0056] wherein (i1, j1) and (i2, j2) represent the coordinates of the point A on the first frame and the second frame of images respectively, and (I1, J1) and (I2, J2) represent the coordinates of the point B on the first frame and the second frame of images respectively; respectively represent the offset of the second frame pixel of the points A and B relative to the first frame, respectively represent the ground resolutions of the points A and B.

[0057] In a second aspect, the embodiment of the present application provides a dynamic monitoring device for ancient roads and garden ecology based on binocular camera, comprising:

[0058] a binocular camera module, configured to collect multi-focus segment images of the ancient road and the surrounding ecology by using a binocular camera mode;

[0059] a distortion calibration module, configured to calculate the distortion at any multiple by 1 times focal length distortion parameter calibration and strong correlation matching of different focal segment images, inversely solve the camera intrinsic parameters of the corresponding focal segment, correct the distortion of the original image, and realize the distortion calibration of the zoom camera;

[0060] a pose solving module, configured to arrange no less than 4 control points based on the minimum shooting field of view, and solve the camera exterior orientation elements by using the collinear equation of the binocular images and the three-dimensional coordinates of the control points;

[0061] a deformation monitoring module, configured to take the reference frame as a reference, calculate the relative offset of the target region in the continuous frames by using the sub-pixel accuracy least square matching, convert the ground deformation into the real ground deformation by using the ground resolution, and dynamically monitor the deformation of the ancient road and the ancient trees;

[0062] an ecological monitoring module, configured to extract ecological elements such as vegetation, water body and bare soil by using image segmentation, analyze the coverage and texture changes based on the time series, monitor the changes of the garden ecology, and generate a degradation warning.

[0063] The technical scheme of the embodiment of the present application has the following beneficial effects:

[0064] ​​​The present application solves the problem of invalidation of the traditional calibration model in the zooming process by constructing a distortion calibration model and imaging geometric formula of the non-polar zoom camera, so that the same device can stably obtain spatial information under different focal lengths, improving the flexibility and precision of the system; the present application introduces image correlation technology and binocular vision constraints to realize camera pose and position solving under the condition of a small number of control points, which is particularly suitable for limited control point layout areas such as cultural relics, and significantly improves the practicability of photogrammetry in cultural relic protection scenarios; the present application is expanded to vegetation coverage, ecological change and other elements on the basis of deformation monitoring, realizing continuous monitoring and early warning of the ecological environment around the ancient road, ancient trees and the like, and improving the comprehensiveness and forward-looking nature of the protection of cultural relics such as ancient roads and ancient trees; the present application analyzes image sequences collected by the camera, without frequent manual intervention and expensive sensor deployment, and the monitoring data can be integrated into a management platform to realize long-term and automatic operation, which has good popularization and application prospect. The present application not only provides a new photogrammetry solving method under zooming conditions at the core algorithm level, ensuring the accuracy of millimeter-level high-precision measurement of cultural relics such as ancient roads and ancient trees, but also provides an efficient solution integrating "structure monitoring + ecological perception + data integration" in the actual protection application of cultural relics, which has significant technological progress and application value. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 is a kind of based on binocular camera's ancient road and horticultural ecological dynamic monitoring method flow chart according to an exemplary embodiment;

[0066] Figure 2 is a kind of based on binocular camera's ancient road and horticultural ecological dynamic monitoring device structure schematic view according to an exemplary embodiment;

[0067] Figure 3 is a kind of distortion difference schematic view between different focusing multiples according to an exemplary embodiment

[0068] Figure 4 is a kind of focal length and imaging radius schematic view according to an exemplary embodiment

[0069] Figure 5 is a kind of ancient road monitoring imaging schematic view using binocular camera according to an exemplary embodiment. DETAILED DESCRIPTION

[0070] To more clearly illustrate the technical features of the present application scheme, the present application will be described in detail below with reference to the specific embodiments and the accompanying drawings.

[0071] As Figure 1 shown, the ancient road and horticultural ecological dynamic monitoring method based on binocular camera provided by the present application embodiment includes the following steps:

[0072] Step S1, a binocular camera mode is used to collect multi-focus segment images of the ancient road and the surrounding ecology;

[0073] Step S2, through 1x focal length distortion parameter calibration, combined with the strong correlation matching calculation of different focal length images, the distortion amount under any multiple is calculated, the corresponding focal length camera internal parameter is solved, the original image distortion correction is performed, the zoom camera infinite zoom distortion calibration is realized, the influence of different focal length distortions is eliminated, the imaging accuracy of each focal length is ensured, and a high-precision image basis is provided for subsequent three-dimensional reconstruction and measurement;

[0074] Step S3, based on the minimum shooting field of view, no less than 4 control points are arranged, the collinear equation of the binocular image and the three-dimensional coordinates of the control points are used to solve the camera external orientation elements, in the limited ruins scene of the control points, the camera position and attitude are accurately determined, and a geometric reference is provided for three-dimensional coordinate solution;

[0075] Step S4, taking the reference frame as a reference, the relative offset of the target area in the continuous frame is calculated through sub-pixel accuracy least square matching, the ground resolution is converted into real ground deformation, the ancient road and the ancient tree deformation are dynamically monitored, the high-precision perception of the ancient road and the ancient tree micro-deformation is realized, the structural risk is found in time, and data support is provided for protection decision;

[0076] Step S5, the ecological elements such as vegetation, water body and bare soil are extracted through image segmentation, the coverage and texture change are analyzed based on time series, the gardening ecological change is monitored, the degradation warning is generated, the dynamic evaluation of the surrounding ecology is realized, the degradation risk is warned in advance, and the integrity of the ancient road surrounding ecological system is ensured.

[0077] As a possible implementation manner of the embodiment, the step S2 comprises:

[0078] Step S21, the distortion parameter calibration of the optical lens of the binocular camera at 1x focal length is performed by using the calibration field, and the distortion parameter at 1x focal length is obtained;

[0079] Step S22, for the same shooting area, based on the fact that the shooting range of the n times focusing image is smaller than that of the 1 times focusing image, the same named points on the n times focusing image and the 1 times focusing image are identified by using the least square method algorithm with full-pixel participation matching, the deviation values dΔx and dΔy of the same named points on the n times focusing image relative to the 1 times focusing image are calculated, the distortion amount △x' and △y' of the corresponding image points on the n times focusing image are solved combined with the obtained distortion amount △x and △y at 1x focal length;

[0080] Step S23, based on the geometric relationship between the pixel radius after distortion correction and the focal length, the pixel radius ratio m0 of the same named points on the 1 times zoom image and the n times zoom image is used to calculate the focal length f of the n times zoom.n ;

[0081] Step S24, based on the obtained distortion amount △x' and △y' on the n times zoom image, distortion correction is performed on each pixel in the original image, and a non-distortion image is output.

[0082] As a possible implementation manner of the present embodiment, the calculation formula of the 1 times focal length distortion parameter is:

[0083] ,

[0084] In the formula, △x and △y are the distortion amounts under 1 times focal length; x0 and y0 are the image principal points; r represents the distance from the image point to the image principal point, which is calculated by the formula ; x and y are the image point coordinates in the image coordinate system; k1 and k2 are the radial distortion parameters of the lens; p1 and p2 are the tangential distortion parameters of the lens; a and b are the non-square correction coefficients of the pixels;

[0085] The distortion amounts △x' and △y' of the corresponding image points on the n times zoom image are:

[0086] ,

[0087] In the formula, dΔx and dΔy are the deviation values of the homonymous points on the n times zoom image and the 1 times zoom image.

[0088] As a possible implementation manner of the present embodiment, the least square method algorithm with full-pixel participation matching is used to determine the corresponding relationship of each pixel on the 1 times zoom image and the n times zoom image, so as to accurately identify the homonymous points.

[0089] As a possible implementation manner of the present embodiment, the ratio m0 of the pixel radii of the homonymous points on the 1 times zoom image and the n times zoom image is the ratio of the distance r1 from the homonymous point on the 1 times zoom image to the image principal point to the distance r n from the homonymous point on the n times zoom image to the image principal point, that is, m0=r1 / r n ; the focal length f n at the n times zoom satisfies f n =f / m0, wherein f is the focal length at the 1 times zoom.

[0090] As a possible implementation manner of the present embodiment, the formula for performing distortion correction on each pixel in the original image is:

[0091]

[0092] In the formula, x i ' and y i ' are the coordinates obtained by performing distortion correction on the i pixel, x i and yi is the original coordinate of i pixel on n times zoom image, △x i and △y i are distortion variables on n times zoom image.

[0093] As a possible implementation manner of the embodiment, the step S3 comprises:

[0094] The step S31 comprises: selecting and arranging at least four control points with known three-dimensional coordinates, and ensuring that the corresponding positions thereof in the image can be accurately identified, so as to solve the camera exterior orientation elements.

[0095] The step S32 comprises: correcting the original image obtained by the binocular camera to obtain a non-distortion imaging model; and based on the collineation equation of photogrammetry, using the control points selected in the step S1 and the corresponding image points thereof in the image, an error equation is set up to inversely solve the 12 exterior orientation elements of the binocular camera, wherein at least three ground control points, preferably four ground control points, are required for checking.

[0096] The step S33 comprises: based on the exterior orientation elements obtained in the step S22, inversely transforming the collineation equation, using the left and right images of the binocular camera to set up equations respectively, and solving the three-dimensional coordinates of any point on the ancient road and the ancient tree; according to the obtained three-dimensional coordinates, calculating the absolute distance between two points on the ancient road or the ancient tree, and through the strong correlation matching of the multi-sequence image pairs, obtaining the dynamic relative change amount, so as to determine the size elements of the ancient road / ancient tree.

[0097] As a possible implementation manner of the embodiment, in the step S31, the specific process of arranging the control points is as follows:

[0098] The control points are pre-arranged according to the minimum shooting field of view range, that is, the shooting angle at the minimum zoom or the maximum focal length is planned to ensure that the image always contains not less than four control points under any focal length setting.

[0099] Natural or artificial targets with stable features are preferentially selected as the control points; in the regions with features such as stone-paved ancient roads, artificial excavation points of suspended ancient roads, and road intersections, the features are preferentially selected as the control points; in the regions lacking obvious ground objects, artificial marker targets are arranged to replace the natural features.

[0100] The distribution structure of the control points meets the geometric stability requirement, that is, the central region and the four surrounding regions of the shooting field of view are covered, and the coplanar and collinear conditions are avoided to form a good geometric distribution structure.

[0101] As a possible implementation manner of the embodiment, in the step S32, the expression of the collineation equation is as follows:

[0102] (6)

[0103] Where (x, y) are the coordinates of the image point, The coordinates of the camera's center of focus are ( ) represents the coordinates of a point on the ground, a i b i c i The image is composed of 9 direction cosines formed by the 3 exterior azimuth elements, i=1, 2, 3, and focal length. f Through formula f n =f / m0 is obtained, ( x 0, y 0) is the value for adjusting the image calibration by 1x.

[0104] As one possible implementation of this embodiment, in step S33, the expression for obtaining the coordinates (X, Y, Z) of any point on the ground through the inverse transformation of the collinearity equation is as follows:

[0105] (7)

[0106] Among them, a i b i c i (i=1, 2, 3) are the nine direction cosines formed by the three exterior azimuth elements of the image. The coordinates are the center of the camera's image capture.

[0107] As one possible implementation of this embodiment, in step S33, the formula for calculating the absolute distance between two points is:

[0108] (8)

[0109] Where (X1, Y1, Z1) and (X2, Y2, Z2) are the three-dimensional coordinates of two points on the ancient road or ancient tree, respectively.

[0110] As one possible implementation of this embodiment, step S4 includes:

[0111] Step S41: Using the first frame image in the video sequence containing the target to be monitored as a reference image, extract the region of interest (ROI) of the target area in the reference image. The target area includes the stone-paved ancient road area and the ancient tree root base area.

[0112] Step S42: For each frame of the video sequence after the first frame, a subpixel precision least squares image matching algorithm is used to search for the image patch most similar to the region of interest (ROI) of the reference frame in the local area of ​​the frame image, and the best matching position is calculated.

[0113] Step S43, record the offset changes between the current frame and the reference frame in the horizontal direction and the vertical direction, construct a time sequence offset model, and the offset is a deformation amount or a tilt trend of the monitoring target;

[0114] Step S44, map the offset to a real coordinate scale according to the pixel offset and the ground resolution, and determine whether there is a continuous deformation exceeding a threshold value, for risk warning of the ancient monument deformation;

[0115] Step S45, register and adjust multiple frames of images in the video sequence, suppress the false matching caused by interference factors, and improve the overall accuracy and robustness of the system.

[0116] As a possible implementation manner of the embodiment, in step S43, if the deformation amount of any two points A and B on the monitoring target is ΔD, ΔD is calculated based on the pixel coordinates of the two points A and B on the two frames of images and the ground resolution gsd:

[0117]

[0118]

[0119]

[0120] wherein (i1, j1) and (i2, j2) respectively represent the coordinates of point A on the first frame and the second frame of images, (I1, J1) and (I2, J2) respectively represent the coordinates of point B on the first frame and the second frame of images; respectively represent the offset of the second frame pixel of points A and B relative to the first frame, respectively represent the ground resolution of points A and B.

[0121] As shown in Figure 2 , the embodiment of the present application provides a kind of based on binocular camera's ancient road and garden ecological dynamic monitoring device, comprising:

[0122] Binocular camera module, for using binocular camera mode to collect the multi-focus segment image of ancient road and surrounding ecology;

[0123] Distortion calibration module, for through 1 times focal length distortion parameter inspection, the strong correlation matching of different focal length images is combined to calculate the distortion amount under any multiple, the camera internal parameter of corresponding focal length is solved back, the distortion correction of original image is carried out, realizes zoom camera zoom lens distortion calibration;

[0124] Attitude solution module, for being based on least shooting field of view, not less than 4 control points are laid out, the collinear equation of binocular image and the three-dimensional coordinates of control point are used to solve camera external orientation element;

[0125] ​​​The deformation monitoring module is used for taking the reference frame as the reference, calculating the relative displacement of the target region in the continuous frames through sub-pixel accuracy least square matching, converting into the actual ground deformation quantity in combination with the ground resolution, and dynamically monitoring the deformation quantity of the ancient road and the ancient tree.

[0126] The ecological monitoring module is used for extracting ecological elements such as vegetation, water body and bare soil through image segmentation, monitoring the change of the horticultural ecology based on time series analysis of the coverage and texture change, and generating the degradation early warning.

[0127] In view of the application challenge of the zoom camera in the dynamic monitoring of the site, a complete set of non-contact, high-precision and sustainable operation monitoring process is established by combining the photogrammetry, image analysis and ecological evaluation, so as to perform millimeter-level dynamic monitoring and early warning on the deformation and growth trend of the ancient Shu Road site and the surrounding ancient trees and original ecological vegetation, and the process is especially suitable for the protection demand of the cultural site in the outdoor natural environment.

[0128] The specific process of the ancient road and horticultural ecological dynamic monitoring based on binocular camera of the application mainly includes the following steps.

[0129] 1. Research and development of zoom camera non-continuous zoom distortion calibration system and formula derivation.

[0130] The zoom camera realizes the change of focal length by changing the position of the lens inside the lens, so as to achieve the purpose of enlarging or reducing the picture. Further, the clarity and image size of the shooting target are adjusted. Since the currently used camera can only jump between fixed multiples (such as only 1x, 2x, 3x multiple zoom), the distortion parameters between fixed zoom cannot be accurately calibrated.

[0131] In view of the problem that the fixed focal length in the traditional camera measurement is difficult to adapt to the actual monitoring demand, the application constructs a dynamic distortion calibration system suitable for the zoom lens, and derives a set of imaging geometric models suitable for the non-continuous zoom continuous change condition, so as to ensure that the imaging precision and spatial reconstruction ability are still stable under different focal lengths, which is one of the core technologies of the application.

[0132] The application derives the calculation formula capable of realizing the non-continuous distortion calibration of the zoom camera on the basis of the geometric relationship between the focal length and the image size. The specific scheme process includes the following:

[0133] 1) 1 multiple focal length distortion parameter calibration.

[0134] The calibration of the distortion parameter of the optical lens of the binocular camera at 1 multiple focal length is carried out by using the calibration field, and the distortion formula is as follows:

[0135] (1),

[0136] wherein, Δx, Δy are the image point correction value; x0, y0 are the image principal point; r represents the distance from the image point to the image principal point, which can be expressed by the formula ; x, y are the image point coordinates in the image coordinate system; k1, k2 are the radial distortion parameters of the lens; p1, p2 are the tangential distortion parameters of the lens; a, b are the non-square correction coefficients of the pixels.

[0137] 2) Calculate the distortion amount at any magnification by strong correlation calculation between images of different focal lengths.

[0138] For images of the same shooting area, the area of n times zoom image is less than 1 times shooting range, so any point on the n times zoom image can be found on the 1 times image. The corresponding relationship of each pixel on the two images can be obtained by using the least square method algorithm with full pixel participating in matching, as shown in Figure 3 , the A1 point on the 1 times zoom image can strictly coincide with the A2 point on the n times zoom image through strong correlation matching of video sequence images, so the deviation values dΔx and dΔy relative to the 1 times zoom image can be calculated. Then the distortion amount △x', △y' of the A2 point on the n times zoom image can be obtained:

[0139] (2).

[0140] 3) Reverse the camera intrinsic parameters at any zoom magnification.

[0141] The pixel radius r after distortion correction and the corresponding camera focal length f satisfy the proportional relationship, as shown in Figure 4 , it can be seen from Figure 4 that in the ideal imaging model, the focal length f and the imaging radius r are inversely proportional, that is, the larger the focal length, the smaller the imaging radius, and their relationship can be simply expressed by the following relationship: assuming that the ratio of the pixel radii of the same name points on the 1 times zoom image and the n times image is m0, then the zoom focal length f n of the n times zoom image can be obtained.

[0142] f n = f / m0 (3),

[0143] The change offset of the optical lens during lens focusing is very small, generally not more than 0.1 times of the pixel, especially the distortion parameters near the image center position are almost close to 0, so the image center position can be considered unchanged during multiple magnification.

[0144] 4) Correction of distortion difference in original image.

[0145] The distortion amount of any one image point of the optical camera of 1 times zoom is found, and the distortion amount of any times zoom corresponding to 1 times zoom is found through strong correlation between zoom images, and the relationship is expressed as follows:

[0146] (4),

[0147] Suppose that the original coordinates of i pixels on n times zoom image are x i and y i , the distortion amounts are △x i ' and △y i ', and the coordinates x i ' and y i ' obtained by correcting the distortion of the pixels are expressed by the following formula:

[0148] (5),

[0149] After the distortion correction of all pixels is completed, an ideal image without distortion is obtained, which provides an ideal data basis for the high-precision measurement of ancient roads and tree diameters.

[0150] The distortion correction of the zoom camera can realize the maximum times of high-definition shooting of the observed target, greatly improve the image resolution, and thus improve the millimeter-level dynamic monitoring of the ancient roads and trees.

[0151] 2. Determination of the size elements of the ancient roads and trees based on a few control points on the scene.

[0152] Considering that the number of control points arranged on the cultural relic site is limited, the application designs a method for solving the exterior orientation elements by combining binocular image matching and a small number of control points for auxiliary constraint, which is especially suitable for dynamically adjusting the shooting field of view under zoom conditions while ensuring the accuracy of the solution of the exterior orientation elements. The technology breaks through the strict dependence of traditional photogrammetry on the number and distribution of control points, and improves the adaptability and deployment flexibility of the system in actual scenes.

[0153] 1) Selection and arrangement of control points.

[0154] In photogrammetry, in order to realize the resection solution of a single image, at least four control points with known three-dimensional coordinates are needed, and their corresponding positions in the image are accurately identified, so as to solve the exterior orientation elements of the camera.

[0155] This invention addresses the characteristic of zoom functionality in practical applications of camera systems by proposing a control point layout method that takes into account the zoom range. Its technical features include: (1) Pre-laying control points based on the minimum shooting field of view. Since the field of view of zoom lenses changes significantly at different focal lengths, the traditional fixed point layout method may cause some control points to exceed the field of view in telephoto mode, affecting the calculation accuracy. Therefore, the layout stage is planned according to the shooting angle at the minimum zoom or maximum focal length to ensure that the image always contains no less than 4 control points under any focal length setting. (2) Prioritizing the selection of natural or artificial targets with stable features as control points. In areas with features such as stone-built ancient roads, artificially excavated points of suspended ancient roads, and road intersections, these are prioritized as control points; in areas lacking obvious features, artificially marked targets are used to replace natural features to ensure that the control points are highly identifiable and have high positioning accuracy in the image. (3) The distribution structure of control points meets the requirements of geometric stability. Control points should cover the perimeter and central area of ​​the shooting field of view, avoiding collinearity and coplanarity, and forming a good geometric distribution structure to enhance the stability and accuracy of single-image resection calculation. Through this arrangement, even when adjusting the focal length during shooting, the system can still acquire sufficient and reasonably distributed control points in a single image from the camera, thereby ensuring accurate calculation of the camera's exterior orientation elements.

[0156] 2) Determine the camera position and orientation, as well as the measurement of the ancient road and the ground features to be measured, by using the correspondence between control points and images of different focal lengths.

[0157] like Figure 5 As shown, binocular cameras were used for monitoring and imaging the ancient road. The coordinates of ground features were measured using binocular images, and the classical collinearity equation of photogrammetry was applied, its expression being as follows:

[0158] (6),

[0159] Where (x, y) are the coordinates of the image point, The coordinates of the camera's center of focus are ( () represents the coordinates of a point on the ground. i b i c i (i=1, 2, 3) are the nine direction cosines composed of the three exterior azimuth elements of the image.

[0160] By correcting the distortion of the original image, the binocular camera installed in the field has an ideal distortion-free imaging model, where f is obtained by formula (3), and (x0, y0) is the value of the image calibration using 1x adjustment. That is to say, from the center of the camera projection, the image point and the object point completely satisfy the collinearity equation above.

[0161] From the collineation equation, one ground control point and its corresponding image point can list two equations of equation (6), and since each binocular image needs 12 exterior orientation elements (X ) to solve the coordinates of any ground feature point.

[0162] All at least 3 ground control points (usually need 4 for inspection) to solve 12 exterior orientation elements (X ).

[0163] When the 12 exterior orientation elements of the binocular camera are solved, the deformation of the ancient road and the change trend of the diameter of the ancient tree can be calculated.

[0164] The following describes how to calculate the three-dimensional coordinates of any point on the ancient road and the ancient tree through the binocular camera.

[0165] In the case where the exterior orientation elements and distortion are solved and corrected, the coordinates (X, Y, Z) of any point on the ground can be solved by the inverse transformation of equation (6). The expression is as follows:

[0166] (7),

[0167] In this equation, a i , b i , c i (i=1, 2, 3) are the 9 direction cosines composed of 3 exterior orientation angles of the image, which are obtained in the previous and solved. The position elements of the left and right cameras are also known. One equation (7) can be listed for each left and right image, so there are 4 equations to solve 3 unknowns (X, Y, Z).

[0168] Through this step, the three-dimensional coordinates of any point on the ancient road and the ancient tree can be solved, such as assuming that the absolute distance between a point A (X1, Y1, Z1) at the bottom and a point B (X2, Y2, Z2) at the top of the ancient road is:

[0169] (8),

[0170] Similarly, through the strong correlation matching between multiple sequence images, the relative change amount from the bottom to the top of the ancient road can be solved.

[0171] 3. Dynamic monitoring of deformation based on image analysis.

[0172] Through the sub-region pixel matching analysis of consecutive frame images in the binocular video sequence, the sub-pixel offset of the deformation region relative to the initial reference frame is calculated, and the quantitative monitoring of the expansion of the ancient road cracks and the inclination of the ancient tree is realized.

[0173] In the case of calculating the absolute length of the size of the ancient road and ancient tree, the continuous video frame sequence is collected by the ordinary camera, and the relative displacement of the micro displacement of the site area as the monitoring target is realized by using image processing and matching algorithm. The displacement is the change of the deformation or tilt of the detection target.

[0174] (1) Reference frame setting: the first frame image in the video sequence is taken as the reference image, and the region of interest (ROI) of the target region to be monitored (such as stone ancient road, ancient tree root, etc.) is extracted.

[0175] (2) Pixel by pixel least square matching: for each subsequent frame image, the least square image matching algorithm with sub-pixel accuracy is used to search for the image block most similar to the reference frame ROI in the local area, and the best matching position is calculated.

[0176] (3) Displacement calculation and accumulation: record the displacement change between the current frame and the reference frame in the horizontal and vertical directions, and construct a time series displacement model. The displacement can be regarded as the deformation or tilt trend of the monitoring target;

[0177] Similarly, assuming the deformation amount of the ancient road at any two points A and B is ΔD, then based on the pixel coordinates of points A and B in the two frames of images and the ground resolution gsd, ΔD is calculated:

[0178] (9),

[0179] (10),

[0180] (11),

[0181] Where (i1, j1) and (i2, j2) represent the coordinates of points A in the first and second frames of images respectively, and (I1, J1) and (I2, J2) represent the coordinates of points B in the first and second frames of images respectively; represents the displacement of points A and B in the second frame of pixels relative to the first frame, represents the ground resolution of points A and B respectively.

[0182] (4) Heritage deformation evaluation: map the displacement to the real coordinate scale according to the pixel displacement and the ground resolution, and judge whether there is continuous deformation exceeding the threshold value, which is used for early warning of ancient tree tilt, ancient road landslide, structure loosening and other risks;

[0183] (5) Multi-frame joint adjustment to enhance stability: through multi-frame registration and adjustment calculation, the false matching caused by light, shaking or noise is suppressed, and the overall accuracy and robustness of the system is improved.

[0184] This method boasts advantages such as contactless camera sensors, low cost, high accuracy, and wide applicability, making it particularly suitable for continuous observation and protection management of historical sites in field environments. It can achieve millimeter-level displacement sensing, demonstrating significant practical value and innovative significance.

[0185] 4. Quantitative monitoring and early warning model for changes in original vegetation cover.

[0186] In addition to monitoring the dynamic deformation of ancient paths and trees, the system is further extended to the synchronous monitoring of the surrounding ecological environment of cultural relics, aiming to protect and manage the overall ecosystem of the site. Continuous video sequences captured by cameras not only contain image information of the target ancient paths, trees, and other cultural relics, but also cover the surrounding ecological characteristics, such as grassland cover, vegetation growth status, area of ​​water accumulation or bare land, and seasonal changes. The system utilizes image processing and change detection technologies to analyze the changing trends of ecological elements over time.

[0187] By utilizing the color, texture, and temporal variations of vegetation areas in imagery, horticultural ecological cover indicators such as grassland and shrubs are extracted to establish dynamic change models, enabling the identification and early warning of ecological degradation. Specific technical methods include, but are not limited to:

[0188] (1) Extraction of regional ecological elements: Based on traditional image segmentation algorithms, ecological elements such as grassland, shrubs, tree canopy, water body, and bare soil in the video sequence are classified and extracted;

[0189] (2) Time series change detection: Compare the changes of ecological elements in the same area over time to identify signs of ecological degradation, such as reduced green coverage, changes in diseases, increased soil bareness, and invasion of alien species.

[0190] (3) Color and texture analysis: Combine color index (such as NDVI fitting) with texture change characteristics to assess the health of vegetation;

[0191] (4) Abnormal event identification: Through rate of change analysis and abnormal behavior detection, it helps to determine the ecological disturbances caused by human damage (such as trampling), animal activities, and extreme weather.

[0192] (5) Visualization of ecological status and linkage early warning: The monitoring results will be presented in the form of heat maps, change curves, etc., and ecological early warning prompts will be issued in conjunction with the site protection system when necessary.

[0193] This invention constructs a complete dynamic monitoring technology framework for ancient sites and their ecological environment. Combined with binocular camera measurement technology, it can achieve joint monitoring of the structural deformation of ancient roads, the changes in the posture of ancient trees, and the surrounding ecological conditions without interfering with the site itself and the natural environment, thereby improving the level of intelligence and precision in the protection of cultural sites.

[0194] Compared with the prior art, the present application has the following characteristics:

[0195] 1. Strong zoom adaptability: Through the non-polar zoom distortion calibration, the problem of precision decline in zooming of traditional fixed focal length calibration is solved, and the same device can cover different monitoring ranges, and the flexibility is significantly improved;

[0196] 2. Low control point dependence: Only a small amount of control points are needed to solve the camera pose, which reduces the demand for control points compared with traditional methods, adapts to the layout restrictions of the site scene, and reduces the interference to the site;

[0197] 3. High-precision monitoring: Realize millimeter-level deformation monitoring, meet the early warning needs of ancient road crack expansion and ancient tree tilt, and the precision is better than that of existing remote sensing or conventional photogrammetry methods;

[0198] 4. Multi-dimensional joint monitoring: Simultaneously realize dynamic monitoring of ancient road structure and surrounding ecology, provide more comprehensive protection basis, and overcome the defect of single monitoring dimension of existing technology;

[0199] 5. Non-contact operation: No need to contact the site body and natural environment, avoid damaging cultural relics and ecology such as ancient roads and trees, suitable for sensitive cultural heritage scenes.

[0200] Finally, it should be explained that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it, although the present application has been described in detail with reference to the above examples, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced by the equivalent, without departing from the spirit and scope of the present application. Any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method for monitoring the dynamic of ancient road and garden ecology based on binocular camera, characterized in that, It comprises the following steps: Step S1, a binocular camera mode is used to collect multi-focus segment images of the ancient road and the surrounding ecology; Step S2, through 1 times focal length distortion parameter calibration, combined with the strong correlation matching calculation of different focal length images, the distortion amount under any multiple is calculated, the camera internal parameter of the corresponding focal length is inversely solved, the distortion correction of the original image is carried out, and the zoom camera zoom distortion calibration is realized; Step S3, based on the minimum shooting field of view, at least 4 control points are arranged, the collinear equation of the binocular image and the three-dimensional coordinates of the control points are used to solve the camera exterior orientation elements; Step S4, taking the reference frame as the reference, the relative offset of the target area in the continuous frame is calculated by sub-pixel accuracy least square matching, combined with the ground resolution conversion into the actual deformation, the ancient road and the ancient tree deformation are dynamically monitored; Step S5, the image segmentation is used to extract the ecological elements, the coverage and texture change are analyzed based on the time series, the gardening ecological change is monitored, and the degradation warning is generated; The step S2 comprises: Step S21, the calibration field is used to calibrate the distortion parameters of the optical lens of the binocular camera at 1 times focal length, and the distortion parameters at 1 times focal length are obtained; Step S22, for the same shooting area, based on the fact that the shooting range of n times focusing image is smaller than that of 1 times focusing image, the same named points on the n times focusing image and the 1 times focusing image are identified by the least square algorithm of full pixel matching, the deviation values dΔx and dΔy of the same named points on the n times focusing image relative to the 1 times focusing image are calculated, combined with the obtained distortion amount △x and △y at 1 times focal length, the distortion amount △x' and △y' of the corresponding image points on the n times focusing image are solved; Step S23, based on the geometric relationship between the pixel radius after distortion correction and the focal length, the ratio m0 of the pixel radii of the same points on the 1 times zoom image and the n times zoom image is used to calculate the focal length f at the n times zoom n ; Step S24, based on the obtained distortion amount △x' and △y' of the n times focusing image, the distortion correction of each pixel in the original image is carried out, and the non-distortion image is output.

2. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 1, characterized in that, The calculation formula of the 1 times focal length distortion parameter is: ， wherein Δx and Δy are distortion values at 1x focal length; x0 and y0 are image principal points; r represents a distance from an image point to the image principal point, and is calculated by the formula ; x and y are image point coordinates in an image coordinate system; k1 and k2 are radial distortion parameters of the lens; p1 and p2 are tangential distortion parameters of the lens; a and b are non-square correction coefficients of the pixels; The distortion amount △x' and △y' of the corresponding image points on the n times focusing image are: ， Wherein, dΔx and dΔy are the deviation values of the same named points on the n times focusing image and the 1 times focusing image. 3.The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 1, characterized in that, The ratio m0 of the pixel radius of the same point on the 1-time zoom image and the n-time zoom image is the ratio of the distance r1 of the same point on the 1-time zoom image to the image principal point to the distance r n of the same point on the n-time zoom image to the image principal point, that is, m0=r1 / r n ; the focal length f n of the n-time zoom satisfies f n =f / m0, wherein f is the focal length of the 1-time zoom.

4. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 1, characterized in that, The formula for carrying out the distortion correction of each pixel in the original image is: wherein x i and y i are the coordinates of the i pixel after distortion correction, x i and y i are the original coordinates of the i pixel on the n times zoom image, and Δx i and Δy i are the distortion amounts on the n times zoom image.

5. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 1, characterized in that, The step S3 comprises: Step S31, at least 4 control points with known three-dimensional coordinates are selected and arranged, and the corresponding positions in the image are accurately identified, which are used for solving the camera exterior orientation elements; Step S32, the original image obtained by the binocular camera is corrected for distortion to obtain a non-distortion imaging model; based on the collinear equation of photogrammetry, the 12 exterior orientation elements of the binocular camera are inversely solved by using the control points selected in step S1 and the corresponding image points thereof in the image, wherein at least 3 ground control points are required; Step S33, based on the exterior orientation elements obtained in step S22, the three-dimensional coordinates of any point on the ancient road and the ancient tree are solved by inverse transformation of the collinear equation, using the left and right images of the binocular camera to respectively list equations; according to the obtained three-dimensional coordinates, the absolute distance between two points on the ancient road or the ancient tree is calculated, the dynamic relative change amount is solved through the strong correlation matching of multiple sequence images, so as to determine the size elements of the ancient road / ancient tree.

6. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 5, characterized in that, In step S31, the specific process of the control point layout is as follows: The control point is pre-laid according to the minimum shooting field range, that is, the shooting angle at the minimum zoom or the maximum focal length is planned to ensure that no less than four control points are contained in the image at any focal length setting; The natural or artificial target with stable characteristics is preferentially selected as the control point; in the area with the artificial excavation point of the stone-paved ancient road, the suspended ancient road and the road intersection feature, the control point is preferentially selected; in the area lacking obvious ground objects, the artificial marker target is laid to replace the natural feature; The control point distribution structure meets the geometric stability requirement, that is, the central and peripheral areas of the shooting field are covered to avoid collinearity and coplanarity and form a good geometric distribution structure.

7. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to any one of claims 1-6, characterized in that, The step S4 comprises: In step S41, a first frame image in a video sequence containing a target to be monitored is taken as a reference image, and a region of interest of a target area to be monitored in the reference image is extracted, the target area to be monitored including a stone-paved ancient road area and an ancient tree root area; In step S42, for each frame image after the first frame in the video sequence, a sub-pixel precision least square image matching algorithm is used to search for an image block most similar to the region of interest of the reference frame in a local area of the frame image, and a best matching position is calculated; In step S43, a change in a horizontal direction and a vertical direction between the current frame and the reference frame is recorded, and a time series offset model is constructed, the offset being a deformation of the target to be monitored or a tilt trend; In step S44, the offset is mapped to a real coordinate scale according to a pixel offset and a ground resolution, and it is judged whether there is a continuous deformation exceeding a threshold value, which is used for risk early warning of the ancient site deformation; In step S45, a plurality of frame images in the video sequence are registered and adjusted.

8. The ancient road and garden ecological dynamic monitoring method based on binocular camera according to claim 7, characterized in that, In step S43, if the deformation of any two points A and B on the target to be monitored is ΔD, ΔD is calculated based on pixel coordinates of the two points A and B on the two frames of images and the ground resolution: ， ， ， wherein (i1 j1) and (i2, j2) represent the coordinates of point A on the first and second frames of images, respectively, and (I1, J1) and (I2, J2) represent the coordinates of point B on the first and second frames of images, respectively; respectively represent the offset of the second frame pixel of points A and B relative to the first frame, respectively represent the ground resolutions of points A and B.

9. A device for monitoring the dynamic of ancient road and garden ecology based on binocular camera, characterized in that, It comprises: A binocular camera module is configured to collect multi-focus segment images of the ancient road and the surrounding ecology by using a binocular camera mode; A distortion calibration module is configured to calibrate the distortion parameters at 1 times focal length, calculate the distortion at any multiple based on the strong correlation matching of different focal segment images, inversely solve the camera intrinsic parameters of the corresponding focal segment, correct the distortion of the original image, and realize the distortion calibration of the zoom camera without zooming; An attitude solving module is configured to solve the camera exterior orientation elements based on the minimum shooting field layout of no less than four control points and the collinearity equation of the binocular images and the three-dimensional coordinates of the control points; A deformation monitoring module is configured to take the reference frame as a reference, calculate the relative offset of the target area in the continuous frame by using the sub-pixel precision least square matching, convert the real ground deformation based on the ground resolution, and dynamically monitor the deformation of the ancient road and the ancient tree; An ecological monitoring module is configured to extract ecological elements by image segmentation, analyze the coverage and texture changes based on time series, monitor the changes of the horticultural ecology, and generate a degradation early warning; The specific process of the distortion calibration module to realize the distortion calibration of the zoom camera without zooming is as follows: Step S21, using the calibration field to calibrate the distortion parameters of the optical lens of the binocular camera at 1 times focal length, and obtaining the distortion parameters at 1 times focal length; Step S22, for the same shooting area, based on the fact that the shooting range of the n times focusing image is smaller than that of the 1 times focusing image, identifying the homonymic points on the n times focusing image and the 1 times focusing image by the least square method algorithm with full pixel participation in matching, calculating the deviation values dΔx and dΔy of the homonymic points on the n times focusing image relative to the 1 times focusing image, combining the obtained distortion variables △x and △y at 1 times focal length, and solving the distortion variables △x' and △y' of the corresponding image points on the n times focusing image; Step S23, based on the geometric relationship between the pixel radius after distortion correction and the focal length, the ratio m0 of the pixel radii of the same points on the 1 times zoom image and the n times zoom image is used to calculate the focal length f at the n times zoom n ; Step S24, based on the obtained distortion variables △x' and △y' on the n times focusing image, correcting the distortion of each pixel in the original image, and outputting a non-distortion image.

Images