Wide-angle camera distortion image correction method and device based on track visual detection

Abstract

The invention discloses a wide-angle camera distorted image correction method and device based on track visual detection. The method comprises the following steps: acquiring a track distorted image; the projection imaging intersection point between the sleeper and the steel rail is adopted to form a mark point; solving the distortion center of the camera by combining a distortion model according to the mapping relation between the three-dimensional world coordinate and the projection coordinate of the three-dimensional world coordinate in the distorted image; solving distortion parameters of a camera by combining a Levenberg-Marquardt algorithm according to the fact that blanking points formed by parallel steel rails in the image must be subjected to collinear constraint; re-projection correction is carried out on the mark points participating in calculation; screening out bad mark points and removing the bad mark points; the distortion center and the distortion parameters are calculated again until all the mark points meet the 3 sigma criterion; and according to the optimal distortion center and the distortion parameters, track image correction is carried out through a distortion model. According to the invention, high-efficiency and accurate distortion correction can be carried out on line without the participation of a special cooperative calibration object.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of track intelligent visual detection, and particularly relates to a wide-angle camera distortion image correction method and device based on track visual detection. BACKGROUND

[0002] With the rise of low-altitude economy and the development of low-altitude technology, it has attracted extensive attention and research in the field of track equipment detection. In view of the advantages of current machine vision detection in speed, cost and visualization, with the help of a wide-angle camera with a large field of view, a longer track area can be covered by a single shot, which is suitable for a high-speed inspection vehicle or a drone to quickly obtain a large range of images, and the overall appearance image of the track equipment to be detected can be quickly obtained, thereby reducing the data acquisition time and speeding up the subsequent visual analysis process. In a space-limited area such as a subway tunnel or under a bridge, a wide-angle camera can capture a complete track section (such as a rail waist, a fastener, and a track bed) at a close distance, avoiding frequent adjustment of the camera position. However, due to the serious distortion problem of the image captured by the wide-angle camera, how to efficiently and accurately calibrate the camera parameters of the wide-angle camera with a large field of view and correct the distorted image has become a key problem to be solved, which restricts the application of the wide-angle camera in the field of track detection.

[0003] To this end, a camera distortion correction method and device, equipment, and storage medium are disclosed in Chinese Patent Application No. CN110738707A. The disclosed distortion correction method of the application obtains a planar array image captured by a camera and a normal planar array image obtained by correcting the distortion of the planar array image, obtains the pixel coordinates of a preset feature point in the planar array image as a distortion pixel coordinate, and obtains the pixel coordinates of the preset feature point in the normal planar array image as a normal pixel coordinate. According to the normal pixel coordinate, the predicted pixel coordinates of the preset feature point are obtained, and at least according to the distortion pixel coordinates and the predicted pixel coordinates, the distortion parameters of the camera are calculated, and the distortion parameters are used to correct the image captured by the camera. This method can correct the distorted image, but it needs to use a high-precision artificial calibration object, which limits the application in the field or in the scene without a calibration object, and cannot adapt to dynamic environments. SUMMARY

[0004] The purpose of the present application is to provide a wide-angle camera distortion image correction method and device based on track visual detection.

[0005] In order to solve the problems existing in the prior art, the technical scheme adopted by the present application is:

[0006] In a first aspect, the present application provides a wide-angle camera distortion image correction method based on track visual detection, comprising the following steps:

[0007] S10: obtaining a track distortion image captured by a wide-angle camera based on track visual detection;

[0008] S20: a spatial non-specific mark point is formed by the intersection of the naturally existing projection imaging between the tie and the steel rail, and the three-dimensional world coordinates of the mark point are obtained;

[0009] S30: the mapping relationship between the three-dimensional world coordinates of the mark point and the projection coordinates of the mark point in the distorted image is used to solve the distortion center e of the camera in combination with the distortion model;

[0010] S40: the second-order radial distortion parameters of the camera are solved by using the Levenberg-Marquardt algorithm in combination with the collinear constraint of the blanking points formed by the parallel steel rails in the image;

[0011] S50: the mark points participating in the calculation are corrected by re-projection through the distortion model;

[0012] S60: the 3σ criterion is used to screen out bad mark points with too large fitting distance and eliminate them;

[0013] S70: the calculation of the distortion center and the second-order radial distortion parameters is re-performed on the basis of the mark points screened through, until all the mark points meet the 3σ criterion;

[0014] S80: the track image is corrected through the distortion model according to the obtained optimal distortion center and second-order radial distortion parameters.

[0015] Further, the mark points participating in the calculation in step S50 are ≥8.

[0016] Further, the mapping relationship between the three-dimensional world coordinates of the mark point and the projection coordinates of the mark point in the distorted image is represented by a homography matrix H:

[0017]

[0018] In the formula, λ represents an arbitrary scale factor; p=p(x u y u 1) T represents the two-dimensional coordinates of the non-distorted image point projected on the imaging plane; (R, t) is called an external parameter, R represents the rotation relationship between the two coordinate systems, and t represents the translation relationship between the two coordinate systems; P=P(X w Y w Z w 1) T represents the three-dimensional coordinates of a point in space; A represents a camera intrinsic parameter matrix, (u0, v0) is the principal point coordinates, f u and f v are effective focal lengths (in pixels), and s is a parameter describing the skew degree of the x and y two image axes.

[0019] Further, the distortion model is a division model (DM), and the distortion model is:

[0020]

[0021] where p is an image point of the pinhole imaging in an ideal non-distortion case, p' is an actually imaged image point, k1 and k2 are second-order radial distortion parameters, e = (e u0 ,e v0 ,1) T represents a homographic coordinate of a center of distortion (COD), r d is a pixel radius of the pixel point p' to the center of distortion e.

[0022] Further, the step S40 comprises the following steps:

[0023] (S401) establishing a second-order radial distortion model, the second-order radial distortion model being a vanishing point collinearity constraint equation, and the vanishing point collinearity constraint equation being:

[0024]

[0025] (S402) optimizing the above second-order radial distortion model using a Levenberg-Marquardt algorithm to solve optimal second-order radial distortion parameters, so that the corrected vanishing point satisfies collinearity.

[0026] Further, the bad landmark points with excessively large fitting distances in the step S60 are spatial landmark points containing gross errors.

[0027] Further, the method for screening the bad landmark points with excessively large fitting distances using a 3σ criterion in the step S60 comprises the following steps:

[0028] A fitting distance {x1, x2,..., xn} of a group of landmark points is obtained through equal-precision measurement, the arithmetic mean x and the residual v n =x i (i = 1, 2,..., n) are calculated, and then the standard deviation σ is calculated, if the residual v i of a certain measurement value x b satisfies the following formula:

[0029] |v b |=|x b -x|>3σ b

[0030] Then x b is regarded as a bad value containing a gross error and should be removed.

[0031] ​wherein {x1x2...x n}.

[0032] In a second aspect, the present application provides a wide-angle camera distortion image correction device based on track visual detection, comprising:

[0033] an acquisition unit configured to acquire a track distortion image collected by a wide-angle camera based on track visual detection;

[0034] a constituting unit configured to constitute a spatial non-specific mark point by using a projection imaging intersection naturally existing between a tie and a steel rail, and obtain a three-dimensional world coordinate of the mark point;

[0035] a distortion center solving unit configured to solve a distortion center e of the camera according to a mapping relationship between the three-dimensional world coordinate of the mark point and a projection coordinate of the mark point in the distortion image, and in combination with a distortion model;

[0036] a second-order radial distortion parameter solving unit configured to solve a second-order radial distortion parameter of the camera according to a must-collinear constraint of a blanking point formed by a parallel steel rail in the image, and in combination with a Levenberg-Marquardt algorithm;

[0037] a correction unit configured to perform re-projection correction on the mark points involved in the calculation by the distortion model;

[0038] a screening unit configured to screen out bad mark points with a too large fitting distance and eliminate them by using a 3σ criterion;

[0039] a repeating unit configured to perform calculation of the distortion center and the second-order radial distortion parameter again on the basis of the mark points screened through until all the mark points satisfy the 3σ criterion;

[0040] a track image correction unit configured to perform track image correction by the distortion model according to the obtained optimal distortion center and second-order radial distortion parameter.

[0041] Further, the second-order radial distortion parameter solving unit comprises:

[0042] a establishing unit configured to establish a blanking point collinear constraint equation based on a second-order radial distortion model,

[0043] the blanking point collinear constraint equation is:

[0044]

[0045] an optimization unit configured to optimize the above second-order radial distortion model by using a Levenberg-Marquardt algorithm, and solve optimal second-order radial distortion parameters so that the corrected blanking points satisfy collinearity.

[0046] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the wide-angle camera distortion image correction method based on track visual detection according to the first aspect when running the computer program.

[0047] The present application has the advantages and beneficial effects that:

[0048] The wide-angle camera distortion image correction method based on track visual detection of the present application is based on a radial distortion exclusion model. First, the spatial coordinates of natural space marker points are constructed according to the track gauge value between two known parallel rails and the spacing value between sleepers and sleepers. The radial distortion center point coordinates are solved according to the image coordinate relationship corresponding to these spatial coordinates based on the radial distortion basic matrix. Then, the constraint equation is constructed according to the vanishing point collinearity constraint in the railway image, and the Levenberg-Marquardt algorithm is used to solve the second-order radial distortion parameters. The distortion coefficients and distortion center coordinates are re-optimized according to the minimum deviation error principle of points and fitting straight lines. Finally, the track distortion image collected by the camera is corrected based on the above. The present application can efficiently and accurately correct the distortion of the wide-angle camera with a large field of view used in railway detection without the participation of specially designed cooperative calibration objects. The whole method is simple and easy to implement, has high correction accuracy, and is suitable for fast correction of camera images in online railway visual detection. BRIEF DESCRIPTION OF DRAWINGS

[0049] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0050] Figure 1 The flowchart of the wide-angle camera distortion image correction method based on track visual detection provided by the present application is shown in the figure;

[0051] Figure 2 The camera imaging model and the radial distortion formation diagram are shown in the figure;

[0052] Figure 3 The track marker point diagram is shown in the figure. In the figure: steel rail 1; sleeper 2;

[0053] Figure 3 a is a spatial plane track diagram, Figure 3 b is a diagram showing the track after non-parallel overhead projection;

[0054] Figure 4 The formation diagram of the vanishing point after parallel rail straight line distortion projection is shown in the figure;

[0055] Figure 5 The structure diagram of the wide-angle camera distortion image correction device based on track visual detection provided by the present application is shown in the figure;

[0056] Figure 6 A structural block diagram of an electronic device according to an embodiment of the present application is provided;

[0057] Figure 7 A distorted image before correction;

[0058] Figure 8 An image after distortion correction using the method of embodiment 1. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0060] The experimental methods in the following embodiments are all conventional methods, and are performed according to the techniques or conditions described in the literature in the art or according to the product instructions, unless otherwise specified. The materials, reagents and the like used in the following embodiments can be obtained from commercial channels, unless otherwise specified.

[0061] Embodiment 1

[0062] As shown in the following, the present embodiment is based on a wide-angle camera distortion image correction method for orbit visual detection, which includes the following steps: Figure 1

[0063] S10: Obtain an orbit distorted image collected by a wide-angle camera based on orbit visual detection;

[0064] The process of projecting a point in a three-dimensional scene to a two-dimensional image plane of a camera can be described by a pinhole imaging model, and a relevant demonstration is shown in FIG. 1. Figure 2

[0065] Figure 2 In the formula, O W X W Y W Z W represents a three-dimensional world coordinate system in which an actual object point is located, O D X D Y D represents a two-dimensional image coordinate system of a camera imaging plane, O C X C Y C Z C represents a three-dimensional camera coordinate system, in which O C and O D respectively represent a camera optical center and an image center, O C O D is the focal length f of the camera, X C ​​axis, Y C axis is parallel to the x D axis, y D axis, Z C axis is perpendicular to the image plane. In the ideal pinhole imaging model, a space object point P(X w Y w Z w 1) T The projection on the image plane is a pixel p(x u y u 1) T However, due to the radial distortion of the camera, the actual imaging point will be shifted to p'(x d y d 1) T This point.

[0066] A space object point P(X w Y w Z w 1) T is projected to a two-dimensional image plane through a pinhole to form a pixel p(x u y u 1) T The corresponding conversion relationship between the three-dimensional world coordinate system and the two-dimensional image coordinate system in the process can be described by the following formula.

[0067]

[0068] In the above formula, λ represents an arbitrary scale factor; p = p(x u y u 1) T represents the two-dimensional coordinates of the un-distorted pixel formed by the projection on the imaging plane; A represents the camera intrinsic parameter matrix, (R, t) is called the external parameter, R represents the rotation relationship between the two coordinate systems, r represents each column of R, t represents the translation relationship between the two coordinate systems; (u0, v0) is the principal point coordinate, f u and f v are the effective focal lengths (in pixels), P = P(X w Y w Z w 1) T represents the three-dimensional coordinates of a space object point; s is a parameter describing the degree of skew of the x and y image axes.

[0069] S20: A space non-specific marker point is formed by the intersection point of the projection imaging between the tie and the steel rail, and the three-dimensional world coordinates of the marker point are obtained;

[0070] In the ideal pinhole imaging model, if there are parallel lines in space and the lines are not parallel to the image plane, the projection lines of the parallel lines in the image plane will intersect at a point, which is the vanishing point corresponding to the parallel lines in space. For a railway track, there are parallel relationships between sleepers and between two rails, and these parallel lines will have corresponding vanishing points after projection imaging. The imaging of the vanishing points in a railway track image can be described by Figure 3 .

[0071] Figure 3 (a) is a schematic diagram of a space plane track, in which Zw=0 is assumed for convenience of representation, Figure 3 (b) is a schematic diagram of the track after non-parallel overhead projection, X V and Y V are the vanishing points corresponding to the horizontal and vertical directions of the track plane, respectively. The projection relationship can be represented by the following formula:

[0072]

[0073] Therefore, a homography matrix H can be used to relate the three-dimensional world coordinates of a landmark point to the projection coordinates of the landmark point in the distorted image, and the mapping relationship therebetween is shown in the following formula:

[0074]

[0075] In the formula, λ represents an arbitrary scale factor; p=p(x u y u 1) T represents the two-dimensional coordinates of the non-distorted image point projected on the imaging plane; (R, t) is referred to as an external parameter, R represents the rotation relationship between the two coordinate systems, and t represents the translation relationship between the two coordinate systems; A represents an internal parameter matrix of the camera, (u0, v0) is the principal point coordinate, f u and f v are the effective focal lengths (in pixels), P=P(X w Y w Z w 1) T represents the three-dimensional coordinates of a point in space; and s is a parameter describing the skew degree of the x and y image axes.

[0076] The mapping relationship formula is simplified as follows:

[0077] λp=HP, with H=A[r1r2t]

[0078] S30: According to the mapping relationship (homography relationship) between the three-dimensional world coordinates of the landmark point and the projection coordinates of the landmark point in the distorted image, and in combination with the distortion model, the distortion center e of the camera is solved.

[0079] For camera distortion, a division model (DM) is proposed to describe the distortion process, which can be expressed as:

[0080]

[0081] where p is the image point of pinhole imaging without distortion, p' is the actual image point, k1, k2 are the second order radial distortion parameters, e = (e u0 ,e v0 ,1) T is the homography of the center of distortion (COD), r d is the pixel radius of p' to the center of distortion e.

[0082] For e, it can also be expressed as:

[0083]

[0084] For the above equation, if let then the above equation can be rewritten as follows:

[0085] p i '=e+C i (p i -e)

[0086] For the above equation, simultaneously multiply [p i ' ] T [e] x ([e]x represents the 3x3 skew-symmetric matrix of [e], According to p i =HP i and C i is not equal to 0, the following equation can be obtained:

[0087] 0=C i [p i '] T ([e] × H)P i =[p i '] T ([e] × H)P

[0088] Let F r =[e] x H, which is called the radial distortion basis matrix, then the above equation can be written as:

[0089] [p i '] T F r P i=0

[0090] If the correspondence between a series of three-dimensional coordinate points and image points is known (the number of corresponding points ≥ 8), the distortion center can be obtained through the basic matrix F r The left extreme point is extracted, and the correlation can be expressed as follows:

[0091] e T F r =e T [e] × H=0.

[0092] S40: Based on the collinear constraint of the vanishing points formed by the parallel rails in the image, the second-order radial distortion parameters of the camera are solved in combination with the Levenberg-Marquardt algorithm.

[0093] The step S40 includes the following steps:

[0094] (S401) Establishing a constraint equation based on a second-order radial distortion model.

[0095] (S402) The second-order radial distortion model is optimized using the Levenberg-Marquardt algorithm to obtain the optimal second-order radial distortion parameters so that the corrected vanishing points satisfy collinearity.

[0096] Specifically, in the ideal undistorted case, the vanishing points between the clusters of parallel straight lines in space should intersect at the same point after projection. However, due to the influence of distortion, there may be multiple vanishing points between the clusters of parallel straight lines after straight line fitting. Therefore, the distortion coefficient of the distorted image can be solved based on the collinearity constraint between the vanishing points. The vanishing points formed by the projection distortion between the parallel rail lines can be used Figure 4 Provide a description.

[0097] Figure 4 In the example, four parallel lines l1, l2, l3, and l4 between the rails are selected, and the intersection points (i.e., vanishing points) formed by the projection lines are v1, v2, v3, v4, v5, and v6 respectively. Assume that the actual vanishing point coordinates formed by the parallel lines in this direction without camera distortion are v(x v ,y v ), take any two marker points A(x A ,y A )、B(x B ,y B ), the straight line l connecting points A and v Av The mathematical formula y can be expressed as follows:

[0098]

[0099] The distance d between point B and the straight line l Av at this time can be expressed as follows:

[0100]

[0101] In the formula, S Av represents the length of the line segment Av. If there is no distortion in the camera, the three points A, B, and v will be on the same straight line in the image plane, and the distance d will also be equal to 0. Assuming that the pixel coordinates of the projection points formed by the two points A and B on the image with distortion are (x A ’, y A ) and (x B ’, y B ) respectively, and r A′ and r B′ are the distortion radii of the distortion points (x A ’, y A ) and (x B ’, y B ) to the distortion center e (e u0 , e v0 ). If there are j = 1, 2, …, m straight lines, and on each straight line there are i = 1, 2, …, n pairs of mark points, the coordinates of the ideal vanishing points formed by these parallel straight lines are v (x v , y v ). Using a second-order radial distortion model, the distance of point B’ to the straight line l A’v can be expressed as follows:

[0102]

[0103] F ij represents the distance calculation of the i-th point of the j-th straight line. For F ij , its value should be close to 0. Thus, the objective function can be constructed as follows:

[0104]

[0105] The above formula is a nonlinear equation with respect to the four parameters (x v , y v , k1, k2). The distortion center e (e u0 , e v0 ) has been obtained by previous solving. At this time, the intersection point (x v , y v ) of any two straight lines and k1, k2 equal to 0 are selected as the initial iteration values for parameter solving. The Levenberg-Marquardt algorithm is used to iteratively optimize this equation, and after iterative optimization, (x v , yv ,k1,k2) numerical solutions of the four parameters.

[0106] S50: reprojecting and correcting the landmark points involved in the calculation using the distortion model. The landmark points involved in the calculation are ≥8.

[0107] S60: Use the 3σ criterion to screen out and remove bad landmarks with too large a fitting distance. Bad landmarks with too large a fitting distance refer to spatial landmarks containing gross errors.

[0108] Specifically, the fitting distances of a set of landmark points {x1x2...x n}. Calculate its arithmetic mean x and residual v i =x i -x(i=1,2,...,n), and then calculate the standard deviation σ. b The residual v b (1≤b≤n) satisfies the following formula:

[0109] |v b |=|x b -x|>3σ

[0110] Then x b Bad values ​​are considered to contain gross errors and should be eliminated.

[0111] S70: recalculating the distortion center and the second-order radial distortion parameters based on the marker points that have been screened out until all marker points meet the 3σ criterion.

[0112] S80: performing orbital image correction using a distortion model according to the obtained optimal distortion center and second-order radial distortion parameters.

[0113] Thus, an embodiment of the present application provides a method for correcting distorted images of a wide-angle camera based on track visual inspection. This method uses the naturally existing projected imaging intersections between sleepers and rails to form non-specialized spatial markers, which does not require the special production of high-precision cooperative calibration objects and is suitable for online correction of multiple terrains. Since this method uses the naturally existing projected imaging intersections between sleepers and rails to form non-specialized spatial markers, it does not require the special production of high-precision cooperative calibration objects and is suitable for online correction of multiple terrains. The correction process of this method separately considers the distortion center of the distorted image, making the distortion correction effect closer to the actual image. This method considers and calibrates the distortion center of the camera. Since the entire calibration process is decoupled, it can effectively avoid the mutual influence between the various parameters during the calibration process. This method solves the distortion center and the distortion parameters separately, which can avoid the coupling problem between the parameters and improve the robustness of the system correction. This method uses the 3σ criterion to screen the marker points to improve the accuracy of the correction parameters. Experiments have shown that the method of the present invention can accurately correct the distortion of track distortion images collected by a wide-angle camera with a large field of view, and can well serve the application scenarios of track visual inspection.

[0114] The above is a specific implementation method of a wide-angle camera distortion image correction method based on track vision detection provided in an embodiment of the present application. Based on this, an embodiment of the present application also provides a wide-angle camera distortion image correction device based on track vision detection.

[0115] Example 2

[0116] like Figure 5 As shown, this embodiment provides a wide-angle camera distortion image correction device based on track visual detection, the device comprising:

[0117] An acquisition unit (1) is used to acquire a track distortion image collected by a wide-angle camera based on track visual detection;

[0118] A constituting unit (2) is used to form a non-specialized spatial marker point using a naturally existing projection imaging intersection point between the sleeper and the rail, and obtain the three-dimensional world coordinates of the marker point;

[0119] A distortion center solving unit (3) is used to solve the distortion center e of the camera based on the mapping relationship between the three-dimensional world coordinates of the marker point and the projection coordinates of the marker point in the distorted image and in combination with the distortion model;

[0120] A second-order radial distortion parameter solving unit (4) is used to solve the second-order radial distortion parameters of the camera based on the constraint that the vanishing points formed by the parallel rails in the image must be collinear, and in combination with the Levenberg-Marquardt algorithm;

[0121] A correction unit (5) is used to perform reprojection correction on the landmark points involved in the calculation using a distortion model;

[0122] A screening unit (6) is used to screen out bad landmarks with too large fitting distances using the 3σ criterion and remove them;

[0123] Repeating unit (7) is used to recalculate the distortion center and the second-order radial distortion parameters based on the marker points that have been screened until all marker points meet the 3σ criterion;

[0124] The track image correction unit (8) is used to perform track image correction through a distortion model according to the obtained optimal distortion center and second-order radial distortion parameters.

[0125] Furthermore, the second-order radial distortion parameter solving unit (4) includes:

[0126] Establishing unit (41), used for establishing error function, wherein the error function is a collinear constraint equation of the vanishing points,

[0127] The collinear constraint equation of the vanishing points is:

[0128]

[0129] The optimization unit (42) is used to optimize the error function using the Levenberg-Marquardt algorithm to solve the optimal second-order radial distortion parameter so that the corrected vanishing point satisfies collinearity.

[0130] The device can perform efficient and accurate online distortion correction on the large-field-of-view wide-angle camera used in the rail intelligent visual inspection system without the participation of special cooperative calibration objects. It has high correction accuracy and is suitable for rapid image correction of the rail intelligent visual inspection system.

[0131] The above describes the wide-angle camera distortion image correction device based on track vision detection provided by the embodiment of the present application from the perspective of functional modularization. Next, the electronic device provided by the embodiment of the present application is introduced from the perspective of hardware processing.

[0132] Example 3

[0133] like Figure 6 As shown, this embodiment provides an electronic device, including an image acquisition module 100 , a GPU 200 , a memory 400 , a processor 300 , a memory 400 , and an image display module 500 .

[0134] In the embodiments of the present application, the number of the image acquisition module 100, the GPU 200, the memory 400, the processor 300, the memory 400 and the image display module 500 is at least one, the communication between the image acquisition module 100, the GPU 200, the memory 400, the processor 300, the memory 400 and the image display module 500 is completed through the communication bus, the GPU 200 is used to process the image, extract the pixel marker point coordinate condition, and simultaneously perform the image correction operation output. The memory 400 stores a program, the processor can call the program stored in the memory, and the processor implements the wide-angle camera distortion image correction method based on track visual detection as described in Embodiment 1 when executing the computer program.

[0135] It should be noted that the embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the system or device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0136] In this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0137] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0138] Application example

[0139] In order to further verify the effectiveness of the method in practical application, the experiment selects the HBVCAM-4M2214HD camera equipped with a 3.6mm lens to perform on-site correction experiment. The experiment collects 6 track distortion images of different angles, and corrects the distortion images by using the method of the present application and Zhang's calibration method. The distortion images before correction are shown in Figure 7 , and the images after correction by using the method of the present application are shown in Figure 8 .

[0140] pass Figure 7 、 8 It can be seen from the comparison that the method of the present invention can perform limited correction on the large-distortion track images collected by track visual inspection. The corrected images can truly reflect the specific track status and provide a good basis for subsequent inspection and analysis.

[0141] The average values ​​of the correction effects of the method described in Example 1 of the present invention and the traditional Zhang calibration method are statistically analyzed, and the results are shown in Table 1. It should be noted that, on the one hand, the Zhang calibration method does not take the distortion center e (e u0 ,e v0 ), while the present method calculates the center of distortion. On the other hand, Zhang's calibration method uses a polynomial model to describe lens distortion, while the present method is based on the division model (DM), resulting in different ways of expressing the obtained distortion coefficients.

[0142] Table 1 Actual image correction results

[0143]

[0144] The mean square reprojection error E in the table msre The calculation formula is shown as follows:

[0145]

[0146] Where x id ,y id Indicates the horizontal and vertical pixel coordinates of the relevant landmark points in the distorted image, x iu ,y iu Indicates the horizontal and vertical pixel coordinates of the marker point after distortion correction and reprojection.

[0147] From the calculation in Table 1, it can be seen that for the six test images, the average mean square reprojection error after correction by the method of the present invention is comparable to the accuracy of Zhang's calibration method. However, considering that the influence of individual maximum error points on the correction is taken into account, the method of the present invention can avoid the problem of pulling away from the center caused by individual error data points compared to Zhang's calibration method, and can better serve subsequent image processing.

[0148] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A wide-angle camera distortion image correction method based on track visual detection, characterized in that: The following steps are involved: S10: Acquire a track distortion image collected by a wide-angle camera based on track visual detection; S20: using the naturally existing projection imaging intersection between the sleeper and the rail to form a non-specialized spatial marker point, and obtaining the three-dimensional world coordinates of the marker point; S30: solving the distortion center e of the camera according to the mapping relationship between the 3D world coordinates of the marker point and the projection coordinates of the marker point in the distorted image and in combination with the distortion model; S40: Based on the collinearity constraint imposed by the vanishing points formed by the parallel rails in the image, the second-order radial distortion parameters of the camera are solved by combining the Levenberg-Marquardt algorithm. S50: reprojecting and correcting the landmark points involved in the calculation through the distortion model; S60: Use the 3σ criterion to filter out bad landmarks with large fitting distances and remove them; S70: recalculating the distortion center and the second-order radial distortion parameters based on the marker points that have passed the screening, until all marker points meet the 3σ criterion; S80: performing orbital image correction using a distortion model according to the obtained optimal distortion center and second-order radial distortion parameters.

2. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 1, characterized in that: The number of landmark points involved in the calculation in step S50 is ≥8.

3. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 1 or 2, characterized in that: The mapping relationship between the three-dimensional world coordinates of the marker point and the projection coordinates of the marker point in the distorted image is represented by a homography matrix H: Where λ represents an arbitrary scale factor; p = p(x u y u 1) T , represents the two-dimensional coordinates of the undistorted image point projected on the imaging plane; A represents the camera intrinsic parameter matrix; (R, t) is called the external parameter, R represents the rotation relationship between the two coordinate systems, and t represents the translation relationship between the two coordinate systems; P = P(X w Y w Z w 1) T Represents the three-dimensional coordinates of an object point in space; f u and f v is the effective focal length (in pixels); (u0, v0) is the coordinate of the principal point; s is a parameter describing the degree of deflection of the x and y image axes.

4. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 1, characterized in that: The distortion model is a division model, and the distortion model is: Where p is the ideal image point of the pinhole under undistorted conditions, p' is the actual image point, k1 and k2 are the second-order radial distortion parameters, and e = (e u0 ,e v0 ,1) T It represents the homography coordinates of the center of distortion (COD), r d It is the pixel radius from pixel point p' to the distortion center e.

5. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 1 or 2, characterized in that: The step S40 includes the following steps: (S401) A second-order radial distortion model is established. The second-order radial distortion model is a vanishing point collinearity constraint equation. The vanishing point collinearity constraint equation is: (S402) The second-order radial distortion model is optimized using the Levenberg-Marquardt algorithm to obtain the optimal second-order radial distortion parameters so that the corrected vanishing points satisfy collinearity.

6. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 1, characterized in that: The bad landmark points with too large fitting distances in step S60 refer to spatial landmark points containing gross errors.

7. The method for correcting wide-angle camera distortion images based on track visual detection according to claim 6, characterized in that: The method of using the 3σ criterion to screen out bad marker points with large fitting distances in step S60 includes the following steps: The fitting distances of a set of landmark points {x1 x2...x n }, calculate its arithmetic mean x and residual v i =x i -x(i=1,2,...,n), then calculate the standard deviation σ, if a certain measurement value x b The residual v b (1≤b≤n) satisfies the following formula: |v b |=|x b -x|>3σ Then x b Bad values ​​that are considered to contain gross errors are eliminated.

8. A wide-angle camera distortion image correction device based on track visual detection, characterized in that: include: An acquisition unit (1) is used to acquire a track distortion image collected by a wide-angle camera based on track visual detection; A constituting unit (2) is used to form a non-specialized spatial marker point using a naturally existing projection imaging intersection point between the sleeper and the rail, and obtain the three-dimensional world coordinates of the marker point; A distortion center solving unit (3) is used to solve the distortion center e of the camera based on the mapping relationship between the three-dimensional world coordinates of the marker point and the projection coordinates of the marker point in the distorted image and in combination with the distortion model; A second-order radial distortion parameter solving unit (4) is used to solve the second-order radial distortion parameters of the camera based on the constraint that the vanishing points formed by the parallel rails in the image must be collinear, and in combination with the Levenberg-Marquardt algorithm; A correction unit (5) is used to perform reprojection correction on the landmark points involved in the calculation using a distortion model; A screening unit (6) is used to screen out bad landmarks with too large fitting distances using the 3σ criterion and remove them; Repeating unit (7) is used to recalculate the distortion center and the second-order radial distortion parameters based on the marker points that have been screened until all marker points meet the 3σ criterion; The track image correction unit (8) is used to perform track image correction through a distortion model according to the obtained optimal distortion center and second-order radial distortion parameters.

9. The wide-angle camera distortion image correction device based on track visual detection according to claim 8, characterized in that: The second-order radial distortion parameter solving unit (4) comprises: The establishment unit (41) is used to establish a vanishing point collinearity constraint equation based on a second-order radial distortion model, wherein the vanishing point collinearity constraint equation is: The optimization unit (42) is used to optimize the second-order radial distortion model using the Levenberg-Marquardt algorithm to solve the optimal second-order radial distortion parameters so that the corrected vanishing points meet collinearity.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the wide-angle camera distortion image correction method based on track visual detection is implemented as described in any one of claims 1 to 7.

Images