Image radial distortion coefficient estimation method, device and computer equipment
By performing discrete sampling and statistical trend analysis within the range of the radial distortion coefficient of an image, the optimal radial distortion coefficient is automatically estimated, solving the problem of relying on calibration objects and scene geometric constraints in existing technologies, and realizing efficient and robust radial distortion coefficient estimation in a single image.
Patent Information
- Application Number
- CN202610183377.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-06
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies rely on specific calibration objects or scene geometric constraints in the estimation of radial distortion coefficients in images. The operation process is complex and lacks robustness, making it difficult to automatically and robustly estimate radial distortion coefficients in a single image.
By discretely sampling within the radial distortion coefficient range of the image to be corrected, a distortion coefficient sequence is generated. Then, by parameter traversal, line detection, intersection calculation, and statistical trend analysis, the optimal radial distortion coefficient is automatically estimated using the statistical regularity of the image's own linear features.
It achieves fully automated radial distortion coefficient estimation, is suitable for a wide range of applications where high-precision calibration conditions are lacking or scene structures are unknown, has good robustness and reliability, can effectively filter out noise and texture loss interference, and improves the stability and credibility of the estimation results.
Smart Images

Figure CN122134821A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, and computer device for estimating the radial distortion coefficient of an image. Background Technology
[0002] In camera calibration technology, accurately determining the radial distortion coefficient is a crucial step in improving the accuracy of applications such as 3D reconstruction and visual measurement. Existing technologies mainly determine the radial distortion coefficient through the following methods, but each has its limitations.
[0003] The first category is methods based on specific calibration objects. These are represented by the classic Zhang Zhengyou planar calibration method and the Tsai two-step method. These methods establish the correspondence between two-dimensional image points and three-dimensional spatial points by photographing known high-precision patterns (such as checkerboards or planar grids), and then use these corresponding points to solve for camera model parameters including distortion coefficients. Their drawback is that they heavily rely on dedicated calibration objects and meticulously designed shooting procedures, making them unsuitable for scenarios such as industrial inspections and security monitoring where deploying calibration boards is not permitted or convenient. Furthermore, calibration accuracy is easily affected by corner detection errors, and traditional models often only consider low-order distortions, leading to decreased accuracy in situations with significant high-order distortions, such as fisheye lenses.
[0004] The second category is self-calibration methods based on scene geometry. These methods do not require a calibration object and aim to utilize the inherent geometric constraints in natural images. A typical approach is to leverage the constraint that a straight line in the real world should still be an image of a straight line. Distortion parameters are solved inversely by detecting curves (actually distorted straight lines) in the image that are bent due to distortion, and fitting or evaluating their degree of bending. Some methods also require prior knowledge or additional optimization of the distortion center, increasing model complexity and solution instability.
[0005] The third category is based on multiple views or other special constraints. For example, using the fundamental matrix between multiple images, controlling the camera to perform specific movements, or utilizing the principle of cross-ratio invariance. These methods either have strict limitations on image acquisition conditions or are computationally complex, making it difficult to meet the needs of fast and automated single-image calibration.
[0006] In summary, existing technologies still face challenges in terms of scene adaptability (relying on specific calibration objects or abundant straight lines), ease of operation (requiring manual intervention or multiple images), and algorithm robustness (sensitive to noise and missing features). Therefore, there is an urgent need for a practical method that can estimate radial distortion coefficients automatically and robustly from a single image, without relying on calibration objects, with relaxed requirements on scene features. Summary of the Invention
[0007] Based on this, it is necessary to address the technical problems of existing methods for determining radial distortion coefficients, such as dependence on specific calibration objects or scene geometric constraints, complex operation procedures, and insufficient robustness under feature loss or noise interference. Therefore, a method, apparatus, and computer equipment for estimating the radial distortion coefficient of an image are proposed.
[0008] Firstly, a method for estimating the radial distortion coefficient of an image is provided, the method comprising: Discrete sampling is performed within the radial distortion coefficient range corresponding to the original image to be corrected to generate an ordered distortion coefficient sequence; For each current distortion coefficient in the distortion coefficient sequence, a parameter traversal operation is performed to generate a sequence of x-direction dispersion statistics and a sequence of y-direction dispersion statistics. The parameter traversal operation includes: calculating the output image size based on the current distortion coefficient and the size of the original image; performing trial correction on the original image using the current distortion coefficient and the output image size to obtain a current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of valid line segments; performing random sampling and calculation on the set of valid line segments to obtain an array of intersection coordinates; removing outliers from the array of intersection coordinates to obtain a cleaned array of intersection coordinates; and calculating the x-direction dispersion statistics and y-direction dispersion statistics based on each coordinate in the cleaned array of intersection coordinates. Perform a downward convexity trend test on the dispersion statistics sequence in the x-direction and the dispersion statistics sequence in the y-direction, respectively; When both sequences pass the downward convexity trend test, calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, and perform a correlation test based on the correlation coefficient; When the correlation test passes, the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistics sequence or the y-direction dispersion statistics sequence is determined, and this distortion coefficient is used as the estimated optimal radial distortion coefficient.
[0009] Secondly, an image radial distortion coefficient estimation device is provided, the device comprising: The sampling module is used to perform discrete sampling within the range of radial distortion coefficients corresponding to the original image to be corrected, and generate an ordered sequence of distortion coefficients. A generation module is used to generate a sequence of x-direction dispersion statistics and a sequence of y-direction dispersion statistics after performing a parameter traversal operation on each current distortion coefficient in the distortion coefficient sequence. The parameter traversal operation includes: calculating the output image size based on the current distortion coefficient and the size of the original image; performing trial correction on the original image using the current distortion coefficient and the output image size to obtain a current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of valid line segments; performing random sampling and calculation on the set of valid line segments to obtain an array of intersection coordinates; removing outliers from the array of intersection coordinates to obtain a cleaned array of intersection coordinates; and calculating the x-direction dispersion statistics and y-direction dispersion statistics based on each coordinate in the cleaned array of intersection coordinates. The trend testing module is used to perform a downward convexity trend test on the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, respectively. The correlation test module is used to calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence when both sequences pass the convexity trend test, and to perform a correlation test based on the correlation coefficient. The determination module is used to determine the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistics sequence or the y-direction dispersion statistics sequence when the correlation test passes, and to use the distortion coefficient as the estimated optimal radial distortion coefficient.
[0010] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for estimating the radial distortion coefficients of an image.
[0011] Beneficial effects: This application realizes a fully automated process for estimating the radial distortion coefficient of images. It replaces the key steps in traditional methods that rely on manual placement of calibration objects, adjustment of shooting posture, or selection of image sequences by using discrete sampling, systematic parameter traversal, and a decision mechanism based on statistical trends. This allows the entire calibration process to be completed without any manual intervention.
[0012] This application abandons the reliance on physical calibration objects or specific known patterns. Its core principle lies in mining and utilizing the statistical regularity exhibited by the straight-line features of the image itself during the distortion correction process. That is, by analyzing the changing trend of the dispersion of the intersection points of straight lines under different trial correction coefficients, the optimal coefficients are solved in reverse. This fundamental shift in principle makes this method applicable to a wide range of applications where high-precision calibration conditions are lacking or the scene structure is unknown.
[0013] This application demonstrates good robustness and reliability. It employs a dual verification mechanism consisting of a downward convexity trend test and a sequence correlation test. This not only mathematically guarantees that the sought optimal solution conforms to the theoretically expected single-valley variation trend, but also physically ensures that the correction effect of the solution is consistent in the horizontal and vertical directions of the image. This design effectively filters out interference caused by image noise, local texture loss, or accidental line mismatches, significantly improving the stability and reliability of the estimation results in complex real-world scenes. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] in: Figure 1 This is a schematic diagram of the structure of a video surveillance system provided in an embodiment of this application; Figure 2 This is a flowchart of a method for estimating the radial distortion coefficients of an image in one embodiment; Figure 3 A schematic diagram of a process for generating a distortion coefficient sequence provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the process of calculating the output image size provided in an embodiment of this application; Figure 5 A schematic diagram of the process for generating trial corrections provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the process of generating a set of valid line segments provided in an embodiment of this application; Figure 7 A schematic diagram of the detection results of a straight line segment provided in an embodiment of this application; Figure 8 A flowchart illustrating the process of generating a cleaned array of intersection coordinates provided in an embodiment of this application; Figure 9 This is a structural block diagram of an image radial distortion coefficient estimation device in one embodiment; Figure 10 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] See Figure 1 , Figure 1 This is a schematic diagram of the structure of a video surveillance system provided in an embodiment of this application. The video surveillance system includes a camera 101 and a computer device 102.
[0018] like Figure 1 As shown, camera 101 and computer device 102 can communicate via a network, which serves as the medium for providing communication links between the aforementioned units. The network can include various types of wired or wireless communication links, such as: wired communication links including optical fiber, twisted pair, or coaxial cable, and wireless communication links including Bluetooth, Wi-Fi, or microwave communication links.
[0019] The computer device described in this application can be equipped with various communication client applications, such as video recording applications, video playback applications, voice interaction applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0020] Computer equipment can also be equipped with display devices, which can be various devices that can perform display functions. Cameras are used to capture video streams; for example, display devices can be cathode ray tube displays (CR), light-emitting diode displays (LED), electronic ink screens, liquid crystal displays (LCD), plasma display panels (PDP), etc.
[0021] It should be understood that Figure 1 The number of cameras, networks, and computer devices shown is for illustrative purposes only. Any number of cameras, networks, and computer devices can be used depending on the implementation requirements.
[0022] Please see Figure 2 As shown, Figure 2 A flowchart illustrating an image radial distortion coefficient estimation method provided in an embodiment of the present invention includes the following steps: S1. Discrete sampling is performed within the range of radial distortion coefficients corresponding to the original image to be corrected to generate an ordered sequence of distortion coefficients.
[0023] The computer equipment first sets a reasonable physical range for the radial distortion coefficients to be estimated based on the fundamental principles of optical imaging and the distortion characteristics of common lenses. This range should cover the typical distortion levels that the lens may exhibit in the target application scenario. After determining the range, discretization is performed within this continuous range, i.e., uniform sampling is performed according to a preset fixed step size, thereby transforming the continuous coefficient values into a series of discrete points that can be tested one by one. The choice of sampling step size is a key engineering trade-off, striking a balance between the final estimation accuracy and the overall computational cost. The smaller the step size, the denser the sampling points, and the more refined the search for the optimal coefficients, but correspondingly, more computationally intensive image correction and feature analysis operations are required subsequently; the larger the step size, the higher the computational efficiency, but the optimal solution may be missed due to sparse sampling. After completing the discrete sampling, the computer equipment sorts all the generated sampling points according to their numerical values, forming an ordered sequence of distortion coefficients.
[0024] For example, the computer device estimates the first-order radial distortion coefficients, a common manifestation of barrel distortion. Based on extensive engineering experience, these coefficients for ordinary wide-angle lenses are typically positive, with absolute values exceeding 0.6. Therefore, the computer device sets the lower limit of the search range to 0, representing a distortion-free state, and the upper limit to 0.6 to cover stronger barrel distortion cases. The computer device sets the sampling step size to 0.05. Thus, within the closed interval from 0 to 0.6, the discrete coefficient points obtained by uniform sampling include 0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60. Subsequently, the computer device arranges these sampling points in ascending order, generating an ordered sequence of distortion coefficients [0, 0.05, 0.10, ..., 0.60]. In the model, barrel distortion corresponds to k>0, and during correction, the scaling factor S>1 indicates that the pixel needs to be moved outward.
[0025] S2. For each current distortion coefficient in the distortion coefficient sequence, perform parameter traversal operation to generate the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence.
[0026] The parameter traversal operation includes: calculating the output image size based on the current distortion coefficients and the size of the original image; performing trial correction on the original image using the current distortion coefficients and the output image size to obtain the current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of valid line segments; performing random sampling and calculation on the set of valid line segments to obtain an array of intersection coordinates; removing outliers from the array of intersection coordinates to obtain a cleaned array of intersection coordinates; and calculating the x-direction dispersion statistics and y-direction dispersion statistics based on each coordinate in the cleaned array of intersection coordinates.
[0027] In the outlier removal step, the preset removal ratio β is a configurable parameter, with a typical recommended range of 1% to 5%. This range is set primarily based on two factors: First, from a statistical validity perspective, this ratio is sufficient to remove significant extreme values caused by noise and random errors, while ensuring that the vast majority (95% to 99%) of valid intersection data are retained to maintain the stability of subsequent dispersion statistics. Second, to improve the method's adaptability and robustness, the value of β can be dynamically fine-tuned according to the actual number of intersection coordinates M obtained. For example, when M is large, a higher fixed ratio within the range can be used; when M is small, a lower ratio within the range can be used, or the formula (e.g., n = max(1, int(β*M))) can be used to ensure that at least one most extreme point is removed, thus achieving effective cleaning results in various scenarios.
[0028] Specifically, for each current distortion coefficient in the distortion coefficient sequence, an independent set of operation procedures is executed to evaluate the potential effect of the distortion coefficient on image correction and output a quantitative evaluation index.
[0029] First, the theoretical output image size is calculated based on the current distortion coefficients and the dimensions of the original image. This calculation is based on a radial distortion model and aims to determine the size of the image canvas required after the correction transformation to prevent the loss of effective pixel information at image edges due to geometric transformation.
[0030] Specifically, the radial distortion model is used to quantitatively describe the geometric displacement of pixels in an image due to their varying radial distances from the optical axis center. This model forms the basis for all subsequent geometric transformation operations, such as trial correction and calculation of output image dimensions.
[0031] In one optional implementation, the radial distortion model is a polynomial model, and its mathematical expression is: .
[0032] A point on the corrected image ( , ) and corresponding points on the distorted original image ( , The relationship between the two can be established using the following formula: .
[0033] The more commonly used equivalent form for inverse mapping (finding the original coordinates from the corrected coordinates), where the scale factor formula directly applies to the normalized radius, is: .
[0034] R norm or r corrected The normalized radius is the distance from a point in the image to the distortion center (usually assumed to be the image center (x-axis)). c y c The Euclidean distance of the vector is normalized. First, the actual radial distance is calculated. Then, it is divided by a normalized reference length. In one specific embodiment of the invention, this reference length is the maximum distance R from the image center to its corner point. max R norm =r / R max Thus, R norm The value range of is restricted to [0, 1], and the R value of the image edge is... norm =1. (xc,yc) are the coordinates of the image distortion center, which is usually set to the image center.
[0035] k1, k2, and k3 are called the first-order, second-order, and third-order radial distortion coefficients, respectively, used to determine the type and intensity of distortion. The core objective of this application is to accurately estimate these coefficients. This application typically uses first-order and second-order coefficients for correction, where the first-order coefficients are equal to the second-order coefficients.
[0036] S is a scaling factor related to the normalized radius. It represents the scaling factor located at a certain R in the original image to correct distortion. norm The point at that location needs to be moved radially by a certain proportion from its ideal position (distortion-free position).
[0037] S>1 indicates that outward movement is needed (to correct barrel distortion), and S<1 indicates that inward movement is needed (to correct pincushion distortion).
[0038] Next, using the current distortion coefficients and the calculated output image size, a trial correction is performed on the original image. This trial correction is achieved through inverse coordinate mapping and pixel interpolation algorithms, generating an intermediate image that is assumed to have been corrected under the current coefficients; this is called the current trial-corrected image. The pixel interpolation algorithm refers to the calculation method used to obtain the precise color value of a pixel in the original distorted image after determining its non-integer floating-point coordinates within the original distorted image through geometric inverse mapping. Since the calculated source coordinates usually do not correspond to integer index positions in the original image's pixel grid, the algorithm calculates a weighted average based on the known color values of several neighboring actual pixels (e.g., the four nearest pixels) around the floating-point coordinate, according to their relative distances to the target floating-point coordinate, thereby synthesizing an estimated pixel value and assigning it to the output pixel. Commonly used algorithms include bilinear interpolation, which uses four neighboring pixels for two linear interpolations; more complex algorithms such as bicubic interpolation can also be used in more demanding scenarios.
[0039] Then, straight line features are extracted from the current trial correction image. A line segment detection algorithm is typically used to identify line segment structures in the image. To improve the reliability of subsequent analysis, the initial set of detected line segments is filtered, removing segments that are too short or do not conform to the expected geometric orientation according to preset rules, thus obtaining a higher-quality set of effective line segments. The line segment detection algorithm is an image processing algorithm that automatically identifies and extracts approximate straight line structures from digital images. Its function is to detect pixel clusters in a grayscale image that conform to the geometric features of a straight line and express them in the form of discrete line segments, typically outputting a set of pixel coordinates of the two endpoints of each line segment. In the specific implementation of this application, a classic algorithm such as LSD (Line Segment Detector) can be used. This algorithm can quickly detect straight line segments in an image without pre-setting parameters and has a certain robustness to noise.
[0040] After obtaining the set of valid line segments, multiple pairs of different line segments are selected from the set through random sampling, and the coordinates of the intersection point of the lines containing each pair of line segments are calculated. A large number of intersection point coordinates are collected to form an initial intersection point coordinate array.
[0041] Due to line detection errors and scene complexity, the initial intersection coordinate array may contain some coordinate points that deviate significantly from the main subject distribution, i.e., outliers. A statistical distribution-based removal method is used to remove a certain proportion of extreme values from both ends of the x-coordinate and y-coordinate sequences of the intersection coordinates, ultimately resulting in a cleaned intersection coordinate array that better represents the distribution trend of the main subject. If the number of line detections is insufficient, the user can be prompted to replace the image or enable multi-frame fusion mode.
[0042] Finally, based on the cleaned intersection point coordinate array, the dispersion of the coordinate values of all intersection points in the horizontal direction of the image is calculated as the X-direction dispersion statistic, and the dispersion of the coordinate values of all intersection points in the vertical direction of the image is calculated as the Y-direction dispersion statistic. The dispersion is typically measured using robust statistics such as interquartile range (IMR), which are insensitive to outliers. To facilitate subsequent comparative analysis and eliminate scale differences caused by perspective, the Y-direction statistics need to be normalized by multiplying by a preset scaling factor (e.g., 0.1).
[0043] After performing the above operation on each distortion coefficient in the sequence, two statistical sequences corresponding one-to-one with the distortion coefficient sequence are obtained, namely the X-direction dispersion statistics sequence and the Y-direction dispersion statistics sequence.
[0044] For example, we are currently processing an ordered sequence with a distortion coefficient of 0.25. The original image has a resolution of 1200 pixels wide and 800 pixels high. Substituting the distortion coefficient 0.25 into the polynomial model of radial distortion, we calculate the maximum scaling factor required at the four corners of the image, thus determining that the output image size should be adjusted to approximately 1260 pixels wide and 840 pixels high to prevent information loss. In the distortion model, k>0 corresponds to barrel distortion, with S>1 during correction; k<0 corresponds to pincushion distortion, with S<1 during correction.
[0045] Subsequently, an empty canvas of size 1260 by 840 is created. Through reverse mapping, the position of each target pixel on the new canvas is calculated back to the corresponding position in the original distorted image according to the distortion correction relationship corresponding to the coefficient 0.25. Then, the pixel value at that position is obtained by bilinear interpolation and filled into the new canvas, thereby generating a trial correction image.
[0046] Next, a line segment detection algorithm was used on this trial correction image to obtain 300 initial detection line segments. According to the rules, short line segments with a length of less than 20 pixels were filtered out, and according to the preset slope threshold, only line segments with a slope greater than 0.3 on the left side of the image and a slope less than -0.3 on the right side of the image were retained, finally obtaining a set of effective line segments containing 85 line segments.
[0047] The sampling ratio was set to 1%, and approximately 72 different pairs of line segments were randomly selected from 85 line segments. The intersection points of their extensions were calculated for each pair. During the calculation, parallel or approximately parallel line segment pairs were discarded, resulting in 65 valid intersection point coordinates, which formed the initial array.
[0048] For these 65 intersection points, their X and Y coordinates are sorted, and the smallest and largest 3 percent of the data points in each sequence are removed, that is, about 2 extreme values are removed from each sequence, thus obtaining a cleaned intersection point coordinate array containing about 61 intersection points.
[0049] Based on these 61 intersection points, the interquartile range of the X-coordinate is calculated, assuming a value of 18.5. This value is used as the X-direction dispersion statistic corresponding to the current coefficient 0.25. Simultaneously, the interquartile range of the Y-coordinate is calculated, assuming a value of 210. To match the magnitude of the X-direction, this value is multiplied by a coefficient of 0.1 for normalization, resulting in 21.0, which is used as the Y-direction dispersion statistic corresponding to the current coefficient 0.25.
[0050] Repeat all the above operations for each coefficient in the sequence, such as 0, 0.05, 0.10, and you will eventually get two complete sequences. One sequence records the X-direction statistic value for each coefficient, and the other sequence records the normalized Y-direction statistic value for each coefficient.
[0051] S3. Perform a downward convexity test on the dispersion statistics sequences in the x-direction and y-direction respectively.
[0052] The purpose of step S3 is to perform a mathematical morphological validity test on the two statistical sequences generated in step S2, specifically to verify whether each sequence exhibits the expected downward convex single-valley trend. Physically, this trend corresponds to the process where the convergence of the linear features in the image first improves and then deteriorates as the trial correction coefficient approaches the optimal value and then overestimates. This test needs to be performed independently on the X-direction dispersion statistics sequence and the Y-direction dispersion statistics sequence.
[0053] First, check the basic validity of the sequence, such as whether the sequence length meets the minimum requirements for trend analysis. Then, locate the global minimum point in the sequence and determine whether this minimum point is located within the sequence itself, rather than at the start or end point. This is a fundamental prerequisite for establishing a complete downward-then-upward trend.
[0054] Next, using the global minimum point as the dividing point, the original sequence is divided into a left subsequence and a right subsequence. For the left subsequence, the overall downward strength of the left subsequence is quantified by calculating the difference between its adjacent data points and counting the proportion of negative differences (i.e., showing a downward trend) out of the total number of differences. For the right subsequence, a similar operation is performed, counting the proportion of positive differences (i.e., showing an upward trend) to quantify its overall upward strength.
[0055] Finally, the calculated percentage decrease on the left and percentage increase on the right are compared with a preset percentage threshold. Only when both percentages exceed the threshold is the current sequence considered to have passed the downward convexity trend test. This test must be applied separately to the two statistical sequences in the X and Y directions, and both sequences must pass the test independently for step S3 to be considered successful. This double-test mechanism ensures that the data trend relied upon for subsequent analysis is mathematically reasonable and reliable. A typical preset percentage threshold can be set to 0.5 (i.e., 50%). This value is determined primarily based on the logic and experience of the method design: theoretically, an ideal downward convexity trend requires that the left side of the minimum point should show an overall downward trend, and the right side should show an overall upward trend. Setting the threshold to 0.5 means that at least half of the adjacent point pairs on the left side should show a downward relationship, and at least half of the adjacent point pairs on the right side should show an upward relationship. This effectively distinguishes between the true overall trend and random local fluctuations, thus ensuring the rigor of the test while having a certain tolerance for data noise. This threshold can also be used as a configurable parameter, which can be adjusted adaptively according to the different requirements of the strictness of trend judgment in specific application scenarios. For example, in scenarios with high noise, the threshold can be appropriately increased to enhance robustness.
[0056] For example, the X-direction dispersion statistic sequence contains 12 values, corresponding to 12 sampling distortion coefficients. The sequence length is confirmed to be 12, satisfying the operational conditions. The minimum value in the sequence is calculated to be the 5th element. It is determined that the 5th position is neither the beginning nor the end of the sequence, thus meeting the requirement of an internal minimum.
[0057] The sequence is then divided into a left subsequence and a right subsequence. The left subsequence consists of the first four values. The three differences between any three adjacent values in this subsequence are calculated, and the number of negative differences is counted. Assuming there are three negative differences, the left subsequence represents a 100% decrease. The right subsequence consists of the 6th to 12th values, a total of seven values. The six differences between any six adjacent values in this subsequence are calculated, and the number of positive differences is counted. Assuming there are five positive differences, the right subsequence represents an approximately 83% increase.
[0058] Next, the same procedure is performed on the Y-direction dispersion statistics sequence. If the Y-direction sequence also passes the test, then step S3 is complete. If any sequence fails the test, for example, if its global minimum is located at the end of the sequence, or if the trend proportion on one side does not reach the threshold, it indicates that the current data does not show a reasonable theoretical trend, and the entire estimation process may terminate here or proceed to the parameter adjustment stage.
[0059] S4. When both sequences pass the downward convexity test, calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, and perform a correlation test based on the correlation coefficient.
[0060] The purpose of step S4 is to further examine whether the trends of the two statistical sequences fluctuating with the distortion coefficient remain synchronized, after step S3 has verified that each of the two statistical sequences has a reasonable downward convex shape. The physical significance of this test is to ensure that an ideal distortion correction coefficient should be able to improve the geometric consistency of the image in both the horizontal and vertical directions. Therefore, the dispersion indices in the two directions should show synergy in their changes.
[0061] The downward convexity trend test determines whether a sequence exhibits a monotonically decreasing trend to the left of the global minimum and a monotonically increasing trend to the right. This is achieved by calculating the confidence scores for the monotonically decreasing trend of the left subsequence and the monotonically increasing trend of the right subsequence, and comparing each score with a preset proportional threshold (e.g., 0.5). The confidence score for the monotonically decreasing trend is defined as the proportion of adjacent pairs in the left subsequence that show a decreasing relationship; the confidence score for the monotonically increasing trend is defined as the proportion of adjacent pairs in the right subsequence that show an increasing relationship.
[0062] Specifically, the Pearson correlation coefficient between the X-direction dispersion statistics sequence and the Y-direction dispersion statistics sequence is first calculated. This coefficient is calculated based on all pairs of corresponding data points in both sequences. The means of each sequence need to be calculated separately, and then the sum of the products of the deviations of each pair of data points from their respective sequence means is calculated. Simultaneously, the sum of the squares of the deviations of all data points in each sequence from their means is calculated separately. The Pearson correlation coefficient is the ratio of the former to the square root of the product of the latter two.
[0063] After obtaining the correlation coefficient, it is compared with a preset correlation threshold. The correlation threshold represents the minimum standard for judging whether the changing trends of two sequences have a significant positive correlation. If the calculated correlation coefficient is greater than or equal to the correlation threshold, the trends of dispersion in the X and Y directions are considered to have sufficient positive correlation; that is, when the distortion coefficient changes, if the dispersion of one sequence increases or decreases, the other sequence tends to change in the same direction. This judgment means that the correlation test is passed. Conversely, if the correlation coefficient is less than the correlation threshold, the changing trends of the two sequences are considered to lack consistent synchronicity, and the correlation test fails.
[0064] For example, suppose the X-direction dispersion statistics sequence is [22.1, 18.5, 15.2, 12.8, 11.0, 10.5, 11.3, 13.6, 16.9, 20.5, 24.8, 29.1], and the Y-direction dispersion statistics sequence is [25.3, 21.0, 17.1, 14.4, 12.5, 12.0, 13.1, 15.9, 19.8, 24.0, 28.9, 34.2]. Both sequences have passed the downward convexity trend test in step S3.
[0065] First, the mean of sequence X is calculated to be approximately 17.8, and the mean of sequence Y is approximately 21.0. Next, the covariance numerator is calculated; that is, for each position i, (X[i] - 17.8) * (Y[i] - 21.0) is calculated, and the 12 results are summed. Simultaneously, the variances of sequences X and Y are calculated separately, i.e., the sum of squared deviations of their respective variances. Finally, the covariance numerator is divided by the square root of the product of the variances of sequences X and Y to obtain the Pearson correlation coefficient. Assume the calculated result is approximately 0.98.
[0066] The preset correlation threshold is usually set to a moderately strong positive number, such as 0.4. Since the calculated correlation coefficient of 0.98 is much greater than the threshold of 0.4, it is determined that the dispersion trends in the X and Y directions are highly positively correlated, passing the correlation test in step S4. This means that the two indicators exhibit a highly coordinated change pattern during the search for the optimal coefficient.
[0067] S5. When the correlation test passes, determine the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistic sequence or the y-direction dispersion statistic sequence, and use this distortion coefficient as the estimated optimal radial distortion coefficient.
[0068] The optimal distortion coefficient is determined when the dispersion statistics sequences in the X and Y directions each exhibit a reasonable downward convex single-valley shape, and their trends are confirmed to have a significant positive correlation. Since the trends of the two sequences are highly synchronized, the positions corresponding to their respective global minimum points—that is, their index values in the ordered distortion coefficient sequence—are theoretically the same or very close. Therefore, either the X-direction or Y-direction dispersion statistics sequence can be selected to locate its global minimum point's index position within that sequence.
[0069] Next, using this index position, we trace back to the ordered sequence of distortion coefficients generated in step S1. We directly read the specific distortion coefficient value stored at that position using the index. This value is then determined as the optimal radial distortion coefficient estimated in this process.
[0070] Finally, the coefficient value is output. This optimal radial distortion coefficient can be used as a core parameter to directly apply precise geometric correction transformation to the original image, thereby completing the entire automatic calibration process.
[0071] For example, after the aforementioned steps, it has been confirmed that the X-direction dispersion statistics sequence [22.1, 18.5, 15.2, 12.8, 11.0, 10.5, 11.3, 13.6, 16.9, 20.5, 24.8, 29.1] and the Y-direction sequence have both passed all tests.
[0072] First, we process the X-direction sequence. By iterating and comparing, we find that the minimum value in this sequence is 10.5. Upon inspection, this minimum value is located at the 6th position of the sequence (assuming the starting index of the sequence is 1). Since the correlation test has passed, it is reasonable to assume that the minimum value of the Y-direction sequence also appears at the same or very close position.
[0073] Subsequently, the ordered distortion coefficient sequence [0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55] generated and stored in step S1 is invoked. Based on the determined index position 6, the 6th coefficient value, 0.25, is read from this sequence.
[0074] Therefore, the distortion coefficient of 0.25 is determined and output as the final estimated optimal radial distortion coefficient. This coefficient can then be used by subsequent image processing modules to perform high-precision correction on the original distorted image.
[0075] In one possible embodiment, see Figure 3 , Figure 3 This is a schematic flowchart of generating a distortion coefficient sequence provided in an embodiment of this application, including the following steps: S211. Based on the distortion characteristics of the lens used for the original image, determine the range of the radial distortion coefficients for the corresponding first-order radial distortion coefficients.
[0076] The first step is to define a reasonable search range for the first-order radial distortion coefficients to be estimated. This range is not determined arbitrarily, but rather based on prior knowledge of the optical characteristics of the imaging system. For common perspective lenses, the radial distortion introduced is mainly divided into two types: barrel distortion and pincushion distortion. In polynomial distortion models, this is usually reflected in the first-order coefficients taking positive or negative values. Based on known or typical information about the imaging system to which the image to be corrected belongs, such as whether the lens is a wide-angle lens, fisheye lens, or ordinary lens, the main type of distortion and its approximate intensity are determined. Based on this optical common sense and engineering experience, the computing device sets a numerical range sufficient to cover the degree of distortion that may occur in the actual use of this type of lens as the search range. The lower and upper limits of this range should be able to encompass various situations from slight inverse distortion to strong positive distortion (or vice versa), ensuring that the true distortion coefficients have a high probability of falling within this range.
[0077] For example, analysis revealed that the original image to be processed was captured by a typical wide-angle surveillance lens, which usually exhibits significant barrel distortion. In commonly used distortion models, barrel distortion corresponds to a positive first-order radial distortion coefficient. Based on extensive calibration experience with similar lenses, the coefficient value is typically between 0 and 0.8. Therefore, to ensure coverage of all possible scenarios and allow for some margin, the lower limit of the search range is set to 0, representing no distortion; the upper limit is set to 1.0 to cover cases with stronger distortion. Thus, the initial radial distortion coefficient range is determined to be 0 to 1.0.
[0078] S212. Within the range of radial distortion coefficients, uniform sampling is performed at a preset fixed sampling interval to generate a set of sampling points containing multiple sampling points.
[0079] After determining the continuous range of coefficients, discretization sampling is required within this range to generate a finite set of candidate coefficient values that can be tested one by one. A uniform sampling strategy is adopted, that is, starting from the initial value of the range and increasing the step size sequentially according to a pre-set fixed step size until the end value of the range is reached or exceeded, thus obtaining a series of equally spaced sampling points. The choice of sampling interval is a key parameter, directly affecting the accuracy of the final estimation and the computational efficiency of the entire algorithm. The smaller the interval, the denser the sampling points, and the more refined the search for the optimal coefficients, but the more image correction and feature analysis are required, and the longer the computation time. Therefore, the preset interval needs to be determined after a trade-off between the required accuracy and available computing resources.
[0080] Continuing the previous example, the search range has been determined to be 0 to 1.0. A fixed sampling interval of 0.05 is now set. Starting from the lower limit of 0, increments of 0.05 are generated sequentially, producing the following sampling points: 0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00. All these sampling points constitute the initial set of sampling points for this parameter traversal.
[0081] S213. Sort the sampling points in the sampling point set according to their numerical values to form an ordered sequence of distortion coefficients.
[0082] After generating the set of sampling points, the elements within this set need to be organized to form an ordered structure. Since the sampling points are generated at fixed intervals, they already possess a certain order. However, to ensure logical rigor and accommodate other possible sampling methods, a sorting operation is performed. This operation arranges all sampling points in the set in ascending order of their values. After sorting, these sampling points constitute an ordered sequence, where the position index of each element strictly corresponds to its numerical value. This ordered sequence provides the foundation for systematically traversing and testing each candidate distortion coefficient in subsequent steps, and ensures that the sequence generated by the dispersion statistics as the coefficients change is also ordered, which is crucial for subsequent trend analysis.
[0083] Obtain the set of sampling points containing 0, 0.05, 0.10, ..., 1.00. Sort these points in ascending order by executing a sorting algorithm, such as quicksort. Since the sampling points in this example were generated in ascending order, the sorted result is consistent with the original generation order, ultimately forming an ordered sequence of distortion coefficients [0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00].
[0084] See Figure 4 The following is a flowchart illustrating the calculation of the output image size provided in an embodiment of this application, including the following steps: S221. Obtain the width and height of the original image.
[0085] The process begins by reading the original image file to be corrected and parsing its image header information or pixel data array to obtain the image's basic geometric parameters. Among these parameters, the image width refers to the total number of pixels contained in the horizontal direction of the image, and the image height refers to the total number of pixels contained in the vertical direction of the image. These two values are the basis for defining the image's spatial range and performing all subsequent geometric calculations.
[0086] S222. Calculate the coordinates of the center point of the original image based on its width and height.
[0087] After obtaining the width and height of the original image, it is necessary to determine the geometric center of the image plane. This center point is usually assumed to be the intersection of the lens optical axis and the imaging plane, and is the center of symmetry for radial distortion. Dividing the image width value by two gives the x-coordinate of the center point; dividing the image height value by two gives the y-coordinate of the center point. This coordinate is in pixels, and its origin is usually defined at the top left corner of the image.
[0088] Continuing the previous example, given that the image width w is 1920 and the height h is 1080, the x-coordinate of the center point (x_center) is calculated as 1920 divided by 2, resulting in 960. The y-coordinate of the center point (y_center) is calculated as 1080 divided by 2, resulting in 540. Therefore, the coordinates of the image center point are (960, 540).
[0089] S223. Based on the center point coordinates, calculate the Euclidean distance from the center point to multiple boundary points in the original image, and take the maximum value of the Euclidean distances among the multiple boundary points as the maximum distance.
[0090] To assess the most extreme distortion levels in an image, it's necessary to find the image boundary locations furthest from the center point. Several key boundary points are selected from the original image, typically including at least four corner points. For each selected boundary point, the distance between the two points is calculated using the Euclidean distance formula, based on its pixel coordinates and the already calculated center point coordinates. The Euclidean distance is the square root of the sum of the squares of the differences in the x-coordinates and y-coordinates between the two points. After calculating the distances from all selected boundary points to the center point, these distance values are compared, and the maximum value is defined as the maximum distance R_max. This distance represents the maximum radial displacement that an image pixel can potentially have.
[0091] Continuing with the previous example, the center point coordinates are (960, 540). We select the four corner points of the image as boundary points, with coordinates of top left (0, 0), top right (1920, 0), bottom left (0, 1080), and bottom right (1920, 1080). We then calculate the Euclidean distances from the center point to each of these four points: The distance to (0, 0) is: sqrt((960-0)^2+(540-0)^2)≈1100.5.
[0092] The distance to (1920, 0) is: sqrt((960-1920)^2+(540-0)^2)≈1100.5.
[0093] The distance to (0, 1080) is: sqrt((960-0)^2+(540-1080)^2)≈1100.5.
[0094] The distance to (1920, 1080) is: sqrt((960-1920)^2+(540-1080)^2)≈1100.5.
[0095] After calculation, all four distance values are approximately 1100.5. The maximum value is selected, i.e., the maximum distance R_max is 1100.5 (pixels). Here, sqrt is used to calculate the square root.
[0096] S224. Based on the current distortion coefficients and the preset radial distortion model, calculate the maximum scaling factor corresponding to the maximum distance.
[0097] This method employs a pre-defined radial distortion mathematical model that describes the functional relationship between the normalized radius and the coordinate scaling factor. The maximum distance R_max corresponds to the maximum normalized radius in the image. First, the maximum distance R_max is normalized, typically by dividing it by a reference value (e.g., the length of the sensor's half-diagonal or the image's half-width / half-height; however, in this method, R_max itself can be used as the normalization reference, making the maximum normalized radius 1). Then, the distortion coefficient currently being tested is substituted into the distortion model formula, and the normalized radius is set to 1 to calculate the corresponding scaling factor. This calculated scaling factor represents the required scaling ratio of the pixel furthest from the center (i.e., the corner point) in the original image when correcting the distortion coefficient currently being tested; it is called the maximum scaling factor S_max.
[0098] Assume the preset radial distortion model is a simplified first-order polynomial model: scaling factor S = 1 + k * r_norm, where k is the current distortion coefficient and r_norm is the normalized radius. The distortion coefficient k in the current test is 0.25. Since the normalized radius r_norm corresponding to the maximum distance R_max is 1, substituting it into the formula: S_max = 1 + 0.25 * 1 = 1.25. Therefore, the calculated maximum scaling factor S_max is 1.25.
[0099] S225. Multiply the width by the maximum scaling factor to obtain the output image width, and multiply the height by the maximum scaling factor to obtain the output image height.
[0100] To avoid losing information in the corrected image, the output image canvas size must be large enough to accommodate all the geometrically transformed pixels. The theoretical output image width is obtained by multiplying the width of the original image by the calculated maximum scaling factor S_max. Similarly, the theoretical output image height is obtained by multiplying the height of the original image by the same maximum scaling factor S_max. Since the number of pixels must be an integer, the product result is usually rounded up to ensure the canvas size is large enough. Ultimately, the output image width and output image height together constitute the output image size.
[0101] Continuing with the examples above, the original image width w is 1920, the height h is 1080, and the maximum scaling factor S_max is 1.25.
[0102] Calculate the output image width as 1920 * 1.25 = 2400.0. Calculate the output image height as 1080 * 1.25 = 1350.0.
[0103] To ensure that the pixel values are integers and the canvas completely contains the corrected image, both results are rounded up, resulting in a final output image width of 2400 pixels and an output image height of 1350 pixels. Therefore, the output image dimensions are 2400 pixels wide and 1350 pixels high.
[0104] In one possible embodiment, see Figure 5 As shown, Figure 5 The flowchart for generating trial corrections provided in this application embodiment includes the following steps: S231. Create an initial output image canvas based on the width and height of the output image.
[0105] Based on the output image width and height parameters calculated in step S225, a corresponding storage area is allocated in memory; this area serves as the output image canvas. The size of this canvas is entirely defined by the aforementioned width and height values; the number of pixels it can accommodate horizontally equals the output image width, and the number of pixels it can accommodate vertically equals the output image height. The initialization operation involves setting the color value of each pixel in this canvas to a predefined default value, typically zero (representing black), or other neutral background colors depending on application requirements. The purpose of creating and initializing the canvas is to provide a blank target carrier for subsequent pixel filling, ensuring that all pixels to be calculated have a defined storage location.
[0106] For example, based on the calculations in the previous steps, the output image width is 2400 pixels and the output image height is 1350 pixels. Therefore, a contiguous buffer in memory is allocated to store 2400 x 1350 pixels of data. Then, all pixel positions in this buffer are iterated through, and the values of the red, green, and blue channels for each pixel are set to 0. After this operation, a pure black output image canvas with a size of 2400 pixels wide and 1350 pixels high is ready.
[0107] S232. For each pixel in the output image canvas, perform a reverse mapping operation to determine the corresponding pixel value in the original image.
[0108] The reverse mapping operation includes: calculating the original image distortion position corresponding to the pixel based on the current distortion coefficient and the preset distortion model; calculating the pixel value in the original image by interpolation based on the original image distortion position; and assigning the calculated pixel value to the corresponding pixel in the output image canvas.
[0109] This process requires processing each pixel on the output image canvas one by one, assigning it the correct color value through inverse mapping and interpolation. For a specific pixel on the canvas, its coordinates in the canvas coordinate system are first determined. Then, based on a preset radial distortion model and the distortion coefficients currently being tested, the theoretical position coordinates of that pixel in the original distorted image before correction are calculated. This position is called the original image distortion position. Since distortion correction is a geometric transformation, these theoretical position coordinates are usually floating-point numbers and may not necessarily coincide with the integer pixel grid of the original image.
[0110] Next, based on the calculated floating-point coordinates, the surrounding pixels in the original image are located. An interpolation algorithm is used to calculate a weighted approximate color value using the color values of several actual pixels surrounding the calculated floating-point coordinates. Common interpolation methods include bilinear interpolation, which uses the four nearest neighbors for calculation. Finally, this interpolated color value is assigned to the pixel being processed on the output image canvas. This process is performed sequentially or in parallel for each pixel on the canvas.
[0111] Suppose we are currently processing a pixel at coordinates (100, 200) on the output canvas. First, convert these coordinates to normalized coordinates relative to the center of the canvas. Assume the distortion coefficient k is 0.25, using a first-order distortion model. Calculate the theoretical source coordinates (Src_x, Src_y) of this point in the original image using the inverse function of the model, which is (95.2, 210.8). Since 95.2 and 210.8 are not integers, use bilinear interpolation to calculate the color of this point. This finds four integer pixels in the original image surrounding this floating-point position, with coordinates (95, 210), (96, 210), (95, 211), and (96, 211). Read the color values of these four points, and then perform a weighted blending calculation based on the decimal parts 0.2 and 0.8 to obtain a final composite color value. This composite color value is then assigned to the pixel (100, 200) on the output canvas.
[0112] S233. After all pixels have been processed, the output image canvas is used as the current trial correction image.
[0113] After the reverse mapping and color assignment operations for each pixel on the output image canvas are completed, all pixels on the entire canvas have been filled with color information corrected based on the current distortion coefficients. At this point, the canvas containing complete pixel data constitutes a new image. This filled canvas data is then formally identified and output as the current trial correction image. This image is the visual result of correcting the original image using the currently traversed distortion coefficients, and it will serve as the direct input for subsequent line detection and feature extraction steps.
[0114] Following step S232, all 3.24 million pixels from (0, 0) to (2399, 1349) on the output canvas were processed sequentially. Once the color value of the last pixel was calculated and filled, the 2400 x 1350 buffer in memory stored the complete, corrected image data. This data in the buffer was encapsulated in a standard image format, for example, labeled in memory as "Trial Correction Image_k0.25", where "k0.25" represents the distortion coefficient used. This image was then sent to the next processing module for line detection.
[0115] See Figure 6 The diagram shown is a flowchart illustrating the process of generating a set of valid line segments according to an embodiment of this application, including the following steps: S241. Convert the current trial correction image from a color image to a grayscale image.
[0116] The first step involves processing the current trial-correction image generated in the preceding steps. This image is typically a color image represented by three channels: red, green, and blue. To simplify subsequent calculations and focus on the image's brightness and structural information, it needs to be converted into a single-channel grayscale image. The conversion process is based on a recognized brightness formula that takes into account the differences in sensitivity of the human visual system to different colors. For each pixel in the color image, its red, green, and blue component values are weighted and summed according to predetermined weighting coefficients to calculate a single grayscale intensity value. This value represents the brightness of that pixel. This calculated grayscale value replaces the original three-channel color values, thus generating a grayscale image containing only brightness information.
[0117] S242. Detect at least one original line segment in the grayscale image using a line segment detection algorithm, forming an original set. Each original line segment in the original set is represented by the coordinates of its two endpoints.
[0118] S243. Perform filtering on each original line segment in the original set in sequence.
[0119] The filtering process includes: determining whether the coordinates of any endpoint of the original line segment exceed the boundary of the current trial correction image; if so, it is discarded. Calculate the pixel length of the original line segment, and determine whether the pixel length is less than the preset minimum length threshold. If it is, discard it. The slope is calculated based on the coordinates of the two endpoints of the original line segment, and based on the image region where the original line segment is located and the slope, it is determined whether it belongs to the preset line segment category pointing towards the vanishing point. If not, it is removed.
[0120] Next, a line segment detection algorithm is applied to the grayscale image to identify the edge features of straight lines. This algorithm can directly output the geometric parameters of each detected line segment, typically represented by the pixel coordinates of the two endpoints of the line segment. All detected line segments constitute an initial raw set.
[0121] Subsequently, each line segment in the original set undergoes a triple-condition filter to eliminate unreliable or irrelevant detection results. The first filter is a boundary check, which determines whether the start or end coordinates of the line segment exceed the valid boundary range of the grayscale image. If any endpoint crosses the boundary, the line segment is considered incomplete or may be caused by image border artifacts and is therefore discarded.
[0122] The second layer of filtering is length checking. Based on the coordinates of the two endpoints of the line segment, the Euclidean geometric length of the line segment in the pixel plane is calculated. This length is then compared to a preset minimum length threshold. If the line segment length is less than this threshold, it is considered to be possibly generated by noise or is too short, exhibiting poor directional stability, and is therefore discarded. A typical value for the preset minimum length threshold can be set based on the resolution of the input image and experience with the application scenario; for example, it can be set to one to two percent of the image diagonal length, or expressed in absolute pixel count as 20 to 50 pixels. The method for determining this threshold is based on the following considerations: excessively short line segments are usually generated by image noise, texture details, or detection errors; their geometric direction calculation is extremely unstable, contributing little to subsequent analysis based on a large number of line intersection statistics and easily introducing interference. By setting this threshold, these unreliable short line segments can be effectively filtered out, thereby improving the overall quality of the effective line segment set and the robustness of subsequent statistics. This threshold can also be used as a configurable parameter, adaptively adjusted according to the specific resolution of the image to be processed, the noise level, and the requirements for feature strictness.
[0123] The third layer of filtering is a direction and region consistency check. This filtering aims to retain line segments that are more likely to point in the direction of the expected vanishing point. First, the slope of the line segment is calculated based on its endpoints. Simultaneously, the position of the line segment's midpoint is used to determine whether it is roughly located in the left or right region of the image. A pre-defined rule is established, for example, line segments expected to point to a vertical vanishing point should have a larger positive slope on the left side of the image and a larger negative slope on the right side. The line segment is compared with a pre-defined slope threshold based on its region and the calculated slope. Only when its slope meets the requirements of its region is it retained; otherwise, it is considered to point to a vanishing point in another direction or is an interfering line segment and is discarded.
[0124] The preset categories of line segments pointing in the direction of the vanishing point are set based on prior knowledge of common scenarios. For example, for indoor and outdoor scenes containing a large number of vertical structures, it can be preset that line segments in the left area of the image should have a positive slope (e.g., greater than 0.3), and line segments in the right area of the image should have a negative slope (e.g., less than -0.3), thereby filtering line segments that may point to the vertical vanishing point. Those skilled in the art can adjust these preset conditions according to the geometric characteristics of the target scene.
[0125] Suppose the line segment detection algorithm detects 500 initial line segments in a grayscale image, forming the original set. For one of these line segments, the endpoint coordinates are reported as (-5, 100) and (100, 200). Upon inspection, it is found that the x-coordinate of the first endpoint, -5, is less than the left boundary of the image (0), therefore, the endpoint of this line segment is determined to be out of bounds and is discarded.
[0126] For another line segment not removed by the boundary check, its endpoint coordinates are (10, 10) and (12, 12). Its length is calculated as the Euclidean distance between the two points, which is approximately 2.83 pixels. The preset minimum length threshold is 30 pixels. Since 2.83 is less than 30, the line segment is determined to be too short and is removed.
[0127] For another line segment that passed the first two checks, its endpoint coordinates are (300, 50) and (350, 400). Its slope is calculated to be approximately (400-50) / (350-300), which equals 7.0. The x-coordinate of the midpoint of this line segment is 325. The image is vertically bisected in the middle; segments with x-coordinates less than 1200 are considered the left region. Therefore, this line segment is located in the left region. The default rule requires that the slope of line segments in the left region must be greater than a positive threshold (e.g., 0.5) to point towards the vanishing point at the top of the image. Since 7.0 is greater than 0.5, this line segment passes this check and is retained. If there is a line segment located in the left region but with a slope close to 0, it will be filtered out by this method.
[0128] In the specific implementation of this application, the definition of "line segment category pointing towards the vanishing point" can be adopted in various flexible ways. Firstly, the most common implementation is based on image region division and slope threshold judgment. For example, the image is divided into left, center, and right regions horizontally, and a slope threshold range corresponding to the approximate direction of the target vanishing point is set for each region based on scene priors (e.g., line segments in the left region must have a positive slope greater than threshold T1, and line segments in the right region must have a negative slope less than threshold T2), thereby filtering out candidate line segments pointing towards the vertical vanishing point. Secondly, dynamic thresholds based on statistical distribution can be used, such as calculating the histogram or quantiles of the slopes of all detected line segments, and considering line segments whose slope values fall within the main distribution range as valid categories. Thirdly, in structured scenes, constraints can also be imposed based on the absolute position of the line segments in the image. For example, only line segments located in the upper part of the image and satisfying a specific angle range are retained, as they are more likely to point towards the vanishing point near the horizon. Fourth, for more complex multi-vanishing-point scenarios, multiple major line segment orientation groups can be identified in advance through cluster analysis (such as Hough transform or mean-shift clustering), and line segments that conform to the preset group orientations can be classified as valid. These definition methods can be used individually or in combination to ensure that geometric features that contribute to radial distortion correction analysis can be effectively screened in different scenarios.
[0129] S244. Based on the filtered original line segments, form a set of valid line segments.
[0130] After performing the aforementioned triple filtering process on all line segments in the original set, all the remaining line segments form a new set. The line segments in this new set all meet the integrity requirements, have significant lengths, and their directions conform to the prior geometric constraints for the target vanishing point. This filtered set of line segments is formally defined as the valid line segment set. This set serves as the direct data basis for subsequent intersection sampling and statistical calculations, and its quality directly affects the accuracy of the final distortion coefficient estimation.
[0131] The initial 500 line segments were individually checked for boundaries, length, and direction. Assuming that after filtering, 325 line segments were removed due to exceeding boundaries, 100 due to insufficient length, and 50 due to direction not conforming to the preset category, the remaining 25 line segments passed all filtering conditions. The endpoint coordinates of these 25 line segments were then reorganized into a data structure called the "Set of Valid Line Segments".
[0132] See Figure 7 As shown, Figure 7 This is a schematic diagram of the detection results of a straight line segment provided in an embodiment of this application.
[0133] Figure 7This is an image containing the entrance gates and surrounding environment of a subway station. First, the acquired color image of the subway station scene is converted to a grayscale image. Then, a line segment detection algorithm (such as the LSD algorithm) is used to analyze the grayscale image. Pixel regions in the image with significant grayscale changes and arranged in a linear pattern are automatically identified and mapped using virtual rectangles (within...). Figure 7 (Illustrated with dashed boxes) marks each image region identified as a line segment. For example, in Figure 7 In this process, the vertical borders of multiple turnstiles are identified by the algorithm and their pixel range is marked with a thin, vertical dashed box. The horizontal bars or edges of the turnstiles are marked with a separate horizontal dashed box. Gaps extending from the ground, edges of decorative lines on walls, etc., may also be marked with their own independent dashed boxes. This application calculates and outputs the pixel coordinates of the two endpoints in the image for each region marked by a dashed box (i.e., a detected line segment). The coordinate information corresponding to all line segments marked by dashed boxes is collected by a computer device to form an initial raw set.
[0134] In one embodiment of this application, the process of generating the intersection point coordinate array includes: first, determining the scale of this random sampling based on the number of line segments in the set of valid line segments. The sampling scale is usually set to a small proportion of the total number of all possible line segment combinations, thereby achieving a balance between computational efficiency and statistical representativeness.
[0135] After determining the sampling size, a cyclic sampling process is initiated. In each cycle, a random sampling method without replacement is used to arbitrarily select two different line segments from the set of valid line segments. For each selected pair of line segments, it is treated as an infinitely extending line, and the coordinates of the intersection point of these two lines are calculated.
[0136] When calculating the intersection point, the first step is to determine whether the two lines are parallel or approximately parallel. This is achieved by calculating the absolute value of the cross product of the direction vectors of the two lines. If this absolute value is less than a very small positive threshold, the two lines are considered parallel, with no unique intersection point, and the current sample pair is discarded. If they are not parallel, the precise intersection point coordinates are obtained by solving a system of two linear equations in two variables, commonly using Cramer's rule. The calculated intersection point coordinates may lie outside the image plane, and a preliminary validity check is usually performed, such as discarding intersection points with negative x-coordinates or y-coordinates.
[0137] The above iterative process continues until the number of successful sampled logs reaches the preset sampling scale. All successfully calculated intersection coordinates are collected and stored in a two-dimensional array, where each row represents the x and y coordinates of an intersection point. The final output two-dimensional array is the intersection coordinate array.
[0138] Assume the valid set of line segments contains 50 line segments. First, calculate the total number of possible line segment combinations: 50 multiplied by 49 and then divided by 2, which equals 1225 pairs. Set the sampling ratio to 1%, then the sampling size is rounded down (0.01 multiplied by 1225), which equals 12 pairs. Twelve successful sampling calculations will be performed.
[0139] In the first sampling, line segments 7 and 23 were randomly selected. Assume the endpoint coordinates of line segment 7 are (100, 150) and (200, 300), and its direction vector is (100, 150). The endpoint coordinates of line segment 23 are (50, 400) and (150, 350), and its direction vector is (100, -50). The absolute value of the cross product of the two direction vectors is |100*(-50) - 150*100| = 20000, which is greater than the threshold 1e. -6 The two lines are determined to be non-parallel.
[0140] Next, the parametric equations of the two lines are established and solved. The final calculated intersection point coordinates are (125.0, 225.0). All these coordinates are positive, and no other invalid cases were found. Therefore, these coordinates (125.0, 225.0) are stored in the intersection point coordinate array.
[0141] Continue the process of random selection, parallelism determination, and intersection point calculation. In a subsequent sampling, lines 5 and 19 might be randomly selected. If the absolute value of their direction vector cross product is found to be less than a threshold, they are determined to be parallel. In this case, the sampling is discarded, no valid intersection point is generated, and it is not counted in the number of successful sampling pairs. The computer will continue to supplement the sampling.
[0142] The loop ends after successfully calculating and collecting the coordinates of 12 valid intersection points. These 12 coordinates are stored as a 12-row, 2-column array, with the first column representing the x-coordinate of each intersection point and the second column representing the y-coordinate. This array is the output of this step.
[0143] See Figure 8 The diagram shown illustrates the process of generating a cleaned array of intersection coordinates according to an embodiment of this application, including the following steps: S261. Obtain the intersection point coordinate array, which consists of M rows and 2 columns of coordinate data.
[0144] The first column contains the x-coordinates of each intersection point, and the second column contains the y-coordinates. The process receives the intersection point coordinate array from the previous processing step. This array is a structured two-dimensional data set containing M rows, each corresponding to a valid intersection point coordinate. Each row contains two data elements: the first column stores the horizontal (x-coordinate) of the intersection point, and the second column stores the vertical (y-coordinate). The number of rows M in this array is explicitly known and used as the basis for subsequent calculations.
[0145] For example, we receive an array of intersection coordinates, represented in memory as a list or matrix containing 85 rows of data. Upon inspection, the first row is [120.5, 450.2], indicating that the x-coordinate of the first intersection is 120.5 and the y-coordinate is 450.2. The second row is [115.8, 445.9], and so on. This confirms that the array has a total of 85 rows, meaning M equals 85.
[0146] S262. Calculate the number of extreme points n to be removed based on the number of rows M and the preset removal ratio β.
[0147] The process involves calculating the number of extreme data points to be removed based on a preset removal ratio β. The removal ratio β is a small percentage, such as 3%, determined during algorithm design, defining the proportion of extreme values to be removed at each end of the data set. Multiplying the total number of data rows M by the removal ratio β yields a floating-point number. Since the number of data points to be removed must be an integer, the product is rounded down, discarding the decimal part, thus obtaining the final number n of extreme points to be removed from each end of the data set.
[0148] Continuing the previous example, the number of rows M in the intersection point coordinate array is known to be 85. The preset elimination ratio β is three percent, or 0.03. Multiplying M by β, i.e., 85 multiplied by 0.03, yields 2.55. Rounding this result down, we get the integer 2. Therefore, the number of extreme points n to be eliminated is determined to be 2. This means that subsequently, the rows corresponding to the two smallest and two largest values in the x-coordinate and y-coordinate respectively will be eliminated.
[0149] S263. For the x-coordinate data in the first column, determine the first row index set corresponding to the n smallest values and the second row index set corresponding to the n largest values.
[0150] The process begins by focusing on the first column of the intersection coordinate array, which contains all x-coordinate values. To identify the extreme x-values that need to be removed, their specific positions within the array must be determined. A copy of this column's data is created, and its values are sorted in ascending order. After sorting, the first n values in the sequence represent the n smallest x-values. The row position number of the first occurrence of each of these n minimum values in the original unsorted x-coordinate column is recorded, and these row position numbers are collected to form the first row's index set, Index_x_min.
[0151] Next, starting from the end of the sorted x-coordinate sequence, take the last n values; these are the n largest x-values. Similarly, find the row position number of the first occurrence of each of these n largest values in the original unsorted sequence, and collect these row position numbers to form the second row index set Index_x_max.
[0152] Suppose the original sequence of x-coordinates with 5 intersection points is [120.5, 115.8, 800.2, 118.0, 810.5], with row indices 1 to 5. Given n=2, sorting this sequence yields [115.8, 118.0, 120.5, 800.2, 810.5]. The two smallest values are 115.8 and 118.0. In the original sequence, 115.8 is at row index 2, and 118.0 is at row index 4. Therefore, Index_x_min = {2, 4}. The two largest values are 800.2 and 810.5. In the original sequence, 800.2 is at row index 3, and 810.5 is at row index 5. Therefore, Index_x_max = {3, 5}.
[0153] S264. For the second column of y-coordinate data, determine the set of the third row indices corresponding to the n smallest values and the set of the fourth row indices corresponding to the n largest values.
[0154] The second column of the intersection coordinate array, i.e., all y-coordinate values, is processed using the same logic. The y-coordinate column is independently sorted, and the smallest n y-values and the largest n y-values in the sorted sequence are identified. Then, the original row positions of these smallest and largest y-values in the original y-coordinate column are found. The row positions corresponding to the smallest n y-values are collected into a third row index set, Index_y_min, and the row positions corresponding to the largest n y-values are collected into a fourth row index set, Index_y_max.
[0155] Continuing with the previous example, suppose the original sequence of the y-coordinates of these 5 intersection points is [450.2, 445.9, 100.0, 448.1, 50.5], with row indices from 1 to 5. Sorting this results in [50.5, 100.0, 445.9, 448.1, 450.2]. The two smallest values are 50.5 and 100.0, with original row indices of 5 and 3 respectively. Therefore, Index_y_min = {5, 3}. The two largest values are 450.2 and 448.1, with original row indices of 1 and 4 respectively. Therefore, Index_y_max = {1, 4}.
[0156] S265. Merge and deduplicate the above four row index sets to obtain the total set of row indexes to be removed.
[0157] After identifying the row indices for extreme values in both the x and y coordinates, four independent index sets were obtained. These sets may contain duplicate row indices because a row might be marked for extreme values in both its x and y coordinates. All elements from these four sets were merged into a larger temporary set. Then, a deduplication operation was performed, removing all duplicate index values from this temporary set to ensure that each row index to be removed appears only once in the final set. This unique set of indices, obtained after merging and deduplication, is the final set of indices for all rows that need to be completely removed from the original intersection coordinate array.
[0158] Based on the previous example, we have obtained four sets: Index_x_min={2, 4}, Index_x_max={3, 5}, Index_y_min={5, 3}, and Index_y_max={1, 4}. We merge them to obtain a temporary set {2, 4, 3, 5, 5, 3, 1, 4}. We then perform a deduplication operation to remove duplicate numbers. After deduplication, we obtain a unique index set Total_Index={1, 2, 3, 4, 5}.
[0159] S266. Delete all rows belonging to the total set from the intersection coordinate array, and construct a cleaned intersection coordinate array from the remaining coordinate data.
[0160] After obtaining the total set of row indices to be removed, the original M-row intersection coordinate array is pruned. Each row index in this total set is traversed, and the corresponding row in the original array is located based on the index. These marked rows are then removed from the array. Deletion can be done all at once or sequentially. After removing all specified rows, the remaining unmarked rows in the original array are automatically arranged compactly. The remaining row data is extracted and reorganized into a new two-dimensional array with fewer rows, maintaining its original two-column coordinate format. This new array is the cleaned intersection coordinate array, where outliers considered too extreme in the x or y directions have been excluded.
[0161] For example, the original intersection coordinate array has 5 rows of data. The total set of row indices to be removed, Total_Index, contains indices {1, 2, 3, 4, 5}. This means that all 5 rows are marked as needing removal. These 5 rows are then deleted from the array. After deletion, no data rows remain. Therefore, the cleaned intersection coordinate array is an empty array containing 0 rows of data. In practical applications, since the removal ratio β is usually small (e.g., 3%) and M is large, the cleaned array usually contains most of the original data. For example, if M=85, removing n=2 extreme values may result in the removal of approximately 4 unique rows, leaving approximately 81 rows of data forming the cleaned array.
[0162] In one possible embodiment, the dispersion statistics in the x-direction and y-direction are calculated based on the coordinates in the cleaned intersection coordinate array.
[0163] This step aims to quantify the distribution information contained in the cleaned intersection point coordinate array into two independent scalar statistics, representing the degree of clustering and dispersion of the intersection points in the horizontal and vertical directions of the image, respectively. First, the cleaned intersection point coordinate array is logically separated into two independent one-dimensional data sequences: one sequence contains the x-coordinate values of all intersection points, and the other sequence contains the y-coordinate values of all intersection points.
[0164] For a sequence of x-coordinates, the interquartile range (IQR) is calculated as the x-direction dispersion statistic. The calculation process is as follows: Sort all values in the sequence in ascending order. In the sorted sequence, determine the positions of the lower quartile (Q1_x) and the upper quartile (Q3_x). The lower quartile corresponds to the value at the 25th percentile of the sequence, and the upper quartile corresponds to the value at the 75th percentile. Since the position indices may be non-integers, linear interpolation is typically used to calculate the accurate quartile values. Subsequently, calculate the difference between the upper and lower quartiles, i.e., IQR_x = Q3_x - Q1_x. This value is the x-direction dispersion statistic; the smaller the value, the more concentrated the intersection points are in that horizontal direction.
[0165] For the y-coordinate sequence, the original interquartile range (IQR_y) is calculated using the same procedure. However, due to the geometric characteristics of scene perspective projection, the coordinate distribution range in the vertical direction is usually much larger than that in the horizontal direction. To make the statistics in the two directions comparable, IQR_y needs to be scaled. Specifically, IQR_y is multiplied by a preset scaling factor, which is an empirical value determined based on a large number of experiments, used to offset the inherent scale differences caused by perspective. The normalized value Y_norm = IQR_y * S is the y-direction dispersion statistic, where S represents the preset scaling factor. The preset scaling factor S is used to balance the scale differences in the X and Y directions caused by image perspective projection, and its typical value can be obtained through empirical statistics, for example, it can be set to 0.1. In specific implementations, S can also be adaptively adjusted according to the image content or prior camera parameters. The normalization factor of 0.1 is an empirical value used to balance the scale expansion in the Y direction caused by perspective, and can be determined through calibration experiments.
[0166] Through the above operations, a pair of evaluation metrics were obtained for the distortion coefficient of the current test, namely the x-direction dispersion statistic IQR_x and the y-direction dispersion statistic Y_norm.
[0167] For example, the cleaned intersection coordinate array contains 60 intersections. First, extract all x-coordinates to form a sequence X, for example, its first few values are [115.2, 118.5, 120.1, ...]. Similarly, extract all y-coordinates to form a sequence Y, for example, [452.3, 448.9, 455.0, ...].
[0168] For the sequence X, sort it. Assume the sorted sequence has 60 values. The lower quartile Q1_x is located at 0.25*(60+1)=15.25, between the 15th and 16th ordered values. Assuming the 15th value is 122.5 and the 16th value is 123.0, we can calculate Q1_x = 122.5 + 0.25*(123.0 - 122.5) = 122.625 using linear interpolation. The upper quartile Q3_x is located at 0.75*(60+1)=45.75, between the 45th and 46th ordered values. Assuming the 45th value is 135.2 and the 46th value is 135.8, then Q3_x = 135.2 + 0.75*(135.8 - 135.2) = 135.65. Then IQR_x was calculated as 135.65 - 122.625 = 13.025. This value is the dispersion statistic in the x-direction.
[0169] For the Y sequence, the same calculation is performed. Assuming that after sorting, Q1_y = 438.2 and Q3_y = 462.8, then the original IQR_y = 462.8 - 438.2 = 24.6. The preset scaling factor S is 0.1. The normalized y-direction statistic Y_norm is calculated as 24.6 * 0.1 = 2.46.
[0170] Finally, for the current distortion coefficient, two statistics are recorded: IQR_x = 13.025 and Y_norm = 2.46. These two values will be stored in the corresponding statistical series for subsequent trend analysis.
[0171] In one possible embodiment, the method of this application further includes: When the convex trend test or the correlation test fails, a judgment result indicating that the indicator test has failed is generated. Based on the judgment results, the target parameters are adjusted. The target parameters include at least one of the following: the boundary value of the radial distortion coefficient range, the sampling interval of discrete sampling, the proportional threshold in the convexity trend test, and the correlation threshold in the correlation test. Using the adjusted target parameters, trigger a re-execution of the image radial distortion coefficient estimation method. If the downward convexity trend test in step S3 or the correlation test in step S4 fails, it indicates that under the current parameter configuration, the data fails to exhibit the statistical regularity expected by theory, and the optimal coefficient cannot be reliably determined. In this case, the application does not simply declare failure, but rather initiates a closed-loop process of parameter adjustment and retry.
[0172] First, a structured decision result is generated. This result clearly records the specific type of test failure, such as failure of the X-sequence convexity trend test, failure of the Y-sequence convexity trend test, or failure of the correlation test between sequences, and may include a snapshot of the key parameters used at the time of the failure.
[0173] Next, based on the above judgment results, one or more preset target parameters are intelligently selected and adjusted. The adjustment strategy aims to relax the conditions, expand the search range, or improve the search accuracy to cope with different failure scenarios. The adjustable target parameters mainly include: first, the upper and lower boundary values of the radial distortion coefficient range, which expands the range to cover the true distortion coefficients that may be missed; second, the fixed sampling interval during discrete sampling, which reduces the interval to improve the search resolution; third, the proportional threshold t used in the downward convex trend test to determine whether the downward or upward trend is significant, which lowers the threshold to tolerate weaker trends; and fourth, the correlation threshold T used in the correlation test to determine the synchronicity of trends, which lowers the threshold to accept weaker correlations.
[0174] The selected target parameters are numerically modified according to predefined adjustment logic or heuristic rules. After the parameter adjustment is completed, the entire image radial distortion coefficient estimation process, which started from step S1, is automatically triggered and re-executed using this new set of parameter configurations. This process can be iterated until the verification is passed and the optimal coefficients are successfully output, or until the preset maximum number of retries is reached.
[0175] For example, in one execution, the test for the downward convexity trend of the Y-direction dispersion statistic sequence failed. The generated result is recorded as "Trend test failed, failed sequence: Y, current coefficient range [0, 0.6], current sampling interval 0.05".
[0176] Based on this result, the analysis logic suggests that the true distortion coefficient may be outside the current test range, causing the Y sequence to not form a complete single-valley trend. Therefore, the target parameter "boundary value of radial distortion coefficient range" is adjusted. It expands the upper limit of the range from 0.6 to 0.8, while keeping the lower limit unchanged at 0, thus adjusting the search range to [0, 0.8].
[0177] Subsequently, using the adjusted new range [0, 0.8], along with the original sampling interval of 0.05, the proportional threshold t=0.5, the correlation threshold T=0.4, and other parameters, the entire method is re-executed from step S1 (discrete sampling in the range of 0 to 0.8).
[0178] For example, if the correlation test fails (the Pearson correlation coefficient is only 0.3, less than the threshold of 0.4), while the downward convexity trend tests of both sequences barely pass, then one might choose to adjust the "correlation threshold T" in the correlation test, temporarily relaxing it from 0.4 to 0.3, and then re-execute the subsequent steps starting from trend analysis (S3) using the original parameters to determine whether the test can be passed under the new threshold. If it still fails, one might further try adjusting other parameters, such as expanding the coefficient range or reducing the sampling interval, and then re-execute the process from the starting point again.
[0179] Please see Figure 9 As shown, in one embodiment, an image radial distortion coefficient estimation apparatus is provided, the apparatus comprising: The sampling module 901 is used to perform discrete sampling within the radial distortion coefficient range corresponding to the original image to be corrected, and generate an ordered distortion coefficient sequence. The generation module 902 is used to generate an x-direction dispersion statistic sequence and a y-direction dispersion statistic sequence after performing a parameter traversal operation on each current distortion coefficient in the distortion coefficient sequence; wherein, the parameter traversal operation includes: calculating the output image size based on the current distortion coefficient and the size of the original image; performing trial correction on the original image using the current distortion coefficient and the output image size to obtain a current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of effective line segments; performing random sampling and calculation on the set of effective line segments to obtain an intersection point coordinate array; removing outliers from the intersection point coordinate array to obtain a cleaned intersection point coordinate array; and calculating the x-direction dispersion statistic and the y-direction dispersion statistic based on each coordinate in the cleaned intersection point coordinate array. The trend testing module 903 is used to perform a downward convexity trend test on the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, respectively. The correlation test module 904 is used to calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence when both sequences pass the convexity trend test, and to perform a correlation test based on the correlation coefficient. The determination module 905 is used to determine the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistics sequence or the y-direction dispersion statistics sequence when the correlation test passes, and to use the distortion coefficient as the estimated optimal radial distortion coefficient.
[0180] In one possible embodiment, the step of discretely sampling within the radial distortion coefficient range corresponding to the original image to be corrected to generate an ordered sequence of distortion coefficients includes: Based on the distortion characteristics of the lens used for the original image, the range of the radial distortion coefficients of the corresponding first-order radial distortion coefficients is determined. Within the radial distortion coefficient range, uniform sampling is performed at a preset fixed sampling interval to generate a set of sampling points containing multiple sampling points; The sampling points in the sampling point set are sorted according to their numerical values to form the ordered distortion coefficient sequence.
[0181] In one possible embodiment, calculating the output image size based on the current distortion coefficients and the size of the original image includes: Obtain the width and height of the original image; Calculate the coordinates of the center point of the original image based on the width and the height; Based on the coordinates of the center point, calculate the Euclidean distance from the center point to multiple boundary points in the original image, and take the maximum value among the multiple boundary points as the maximum distance; Based on the current distortion coefficient and the preset radial distortion model, calculate the maximum scaling factor corresponding to the maximum distance; Multiply the width by the maximum scaling factor to obtain the output image width, and multiply the height by the maximum scaling factor to obtain the output image height; The output image size is composed of the output image width and the output image height.
[0182] In one possible embodiment, the step of performing trial correction on the original image using the current distortion coefficients and the output image size to obtain a current trial-corrected image includes: An initialized output image canvas is created based on the width and height of the output image. For each pixel in the output image canvas, a reverse mapping operation is performed to determine the pixel value corresponding to that pixel in the original image. The reverse mapping operation includes: calculating the distortion position of the original image corresponding to that pixel based on the current distortion coefficient and a preset distortion model; calculating the pixel value in the original image by interpolation based on the distortion position of the original image; and assigning the calculated pixel value to the corresponding pixel in the output image canvas. After all pixels have been processed, the output image canvas is used as the current trial correction image.
[0183] In one possible embodiment, the step of performing line detection and filtering on the current trial correction image to obtain a set of valid line segments includes: Convert the current trial correction image from a color image to a grayscale image; At least one original line segment is detected in the grayscale image using a line segment detection algorithm, forming an original set; each original line segment in the original set is represented by the coordinates of its two endpoints; For each original line segment in the original set, a filtering process is performed sequentially, the filtering process including: Determine whether the coordinates of any endpoint of the original straight line segment exceed the boundary of the current trial correction image; if so, discard it. Calculate the pixel length of the original line segment, and determine whether the pixel length is less than a preset minimum length threshold. If it is, discard it. The slope is calculated based on the coordinates of the two endpoints of the original straight line segment, and based on the image region where the original straight line segment is located and the slope, it is determined whether it belongs to the preset line segment category pointing towards the vanishing point. If not, it is removed. Based on the filtered original line segments, the set of effective line segments is formed.
[0184] In one possible embodiment, the outlier removal process for the intersection point coordinate array to obtain a cleaned intersection point coordinate array includes: Obtain the intersection point coordinate array, which consists of M rows and 2 columns of coordinate data, with the first column being the x-coordinate of each intersection point and the second column being the y-coordinate of each intersection point; Calculate the number of extreme points n to be removed based on the number of rows M and the preset removal ratio β; For the first column of x-coordinate data, determine the first row index set corresponding to the n smallest values, and the second row index set corresponding to the n largest values; For the second column of y-coordinate data, determine the set of the third row indices corresponding to the n smallest values, and the set of the fourth row indices corresponding to the n largest values; Merge and deduplicate the four row index sets mentioned above to obtain the total set of row indexes to be removed. Remove all rows belonging to the total set from the intersection coordinate array, and use the remaining coordinate data to form the cleaned intersection coordinate array.
[0185] In one possible embodiment, calculating the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, and performing a correlation test based on the correlation coefficient, includes: Calculate the Pearson correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence; The Pearson correlation coefficient is compared with a preset correlation threshold; If the Pearson correlation coefficient is greater than or equal to the correlation threshold, the correlation test is deemed to have passed. If the Pearson correlation coefficient is less than the correlation threshold, the correlation test is deemed to have failed.
[0186] In one possible embodiment, it also includes: The adjustment module is used to generate a judgment result indicating that the test has failed when the convex trend test or the correlation test fails. Based on the determination result, the target parameters are adjusted, and the target parameters include at least one of the following: the boundary value of the radial distortion coefficient range, the sampling interval of the discrete sampling, the proportional threshold in the convexity trend test, and the correlation threshold in the correlation test; Using the adjusted target parameters, the method for estimating the radial distortion coefficients of the image is re-executed.
[0187] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the function or steps of a method for estimating the radial distortion coefficients of an image.
[0188] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements... Figure 2 The method shown can be referred to for details. Figure 2 As shown, it will not be elaborated further here.
[0189] In one embodiment, a computer-readable storage medium is provided that stores a computer program, which is loaded and executed by a processor as described above. Figure 1 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 2 The specific details of the illustrated embodiments will not be elaborated here. It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0190] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0192] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for estimating the radial distortion coefficient of an image, characterized in that, include: Discrete sampling is performed within the radial distortion coefficient range corresponding to the original image to be corrected to generate an ordered distortion coefficient sequence; For each current distortion coefficient in the distortion coefficient sequence, a parameter traversal operation is performed to generate a sequence of x-direction dispersion statistics and a sequence of y-direction dispersion statistics. The parameter traversal operation includes: calculating the output image size based on the current distortion coefficient and the size of the original image; performing trial correction on the original image using the current distortion coefficient and the output image size to obtain a current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of valid line segments; performing random sampling and calculation on the set of valid line segments to obtain an array of intersection coordinates; removing outliers from the array of intersection coordinates to obtain a cleaned array of intersection coordinates; and calculating the x-direction dispersion statistics and y-direction dispersion statistics based on each coordinate in the cleaned array of intersection coordinates. Perform a downward convexity trend test on the dispersion statistics sequence in the x-direction and the dispersion statistics sequence in the y-direction, respectively; When both sequences pass the downward convexity trend test, calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, and perform a correlation test based on the correlation coefficient; When the correlation test passes, the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistics sequence or the y-direction dispersion statistics sequence is determined, and this distortion coefficient is used as the estimated optimal radial distortion coefficient.
2. The method according to claim 1, characterized in that, The step of discretely sampling within the radial distortion coefficient range corresponding to the original image to be corrected to generate an ordered distortion coefficient sequence includes: Based on the distortion characteristics of the lens used for the original image, the range of the radial distortion coefficients of the corresponding first-order radial distortion coefficients is determined. Within the radial distortion coefficient range, uniform sampling is performed at a preset fixed sampling interval to generate a set of sampling points containing multiple sampling points; The sampling points in the sampling point set are sorted according to their numerical values to form the ordered distortion coefficient sequence.
3. The method according to claim 1, characterized in that, The step of calculating the output image size based on the current distortion coefficients and the size of the original image includes: Obtain the width and height of the original image; Calculate the coordinates of the center point of the original image based on the width and the height; Based on the coordinates of the center point, calculate the Euclidean distance from the center point to multiple boundary points in the original image, and take the maximum value among the multiple boundary points as the maximum distance; Based on the current distortion coefficient and the preset radial distortion model, calculate the maximum scaling factor corresponding to the maximum distance; Multiply the width by the maximum scaling factor to obtain the output image width, and multiply the height by the maximum scaling factor to obtain the output image height; The output image size is composed of the output image width and the output image height.
4. The method according to claim 3, characterized in that, The step of using the current distortion coefficients and the output image size to perform trial correction on the original image to obtain the current trial-corrected image includes: An initialized output image canvas is created based on the width and height of the output image. For each pixel in the output image canvas, a reverse mapping operation is performed to determine the pixel value corresponding to that pixel in the original image. The reverse mapping operation includes: calculating the distortion position of the original image corresponding to that pixel based on the current distortion coefficient and a preset distortion model; calculating the pixel value in the original image by interpolation based on the distortion position of the original image; and assigning the calculated pixel value to the corresponding pixel in the output image canvas. After all pixels have been processed, the output image canvas is used as the current trial correction image.
5. The method according to claim 1, characterized in that, The process of detecting and filtering lines in the current trial correction image to obtain a set of valid line segments includes: Convert the current trial correction image from a color image to a grayscale image; At least one original line segment is detected in the grayscale image using a line segment detection algorithm, forming an original set; each original line segment in the original set is represented by the coordinates of its two endpoints; For each original line segment in the original set, a filtering process is performed sequentially, the filtering process including: Determine whether the coordinates of any endpoint of the original straight line segment exceed the boundary of the current trial correction image; if so, discard it. Calculate the pixel length of the original line segment, and determine whether the pixel length is less than a preset minimum length threshold. If it is, discard it. The slope is calculated based on the coordinates of the two endpoints of the original straight line segment, and based on the image region where the original straight line segment is located and the slope, it is determined whether it belongs to the preset line segment category pointing towards the vanishing point. If not, it is removed. Based on the filtered original line segments, the set of effective line segments is formed.
6. The method according to claim 1, characterized in that, The step of removing outliers from the intersection point coordinate array to obtain a cleaned intersection point coordinate array includes: Obtain the intersection point coordinate array, which consists of M rows and 2 columns of coordinate data, with the first column being the x-coordinate of each intersection point and the second column being the y-coordinate of each intersection point; Calculate the number of extreme points n to be removed based on the number of rows M and the preset removal ratio β; For the first column of x-coordinate data, determine the first row index set corresponding to the n smallest values, and the second row index set corresponding to the n largest values; For the second column of y-coordinate data, determine the set of the third row indices corresponding to the n smallest values, and the set of the fourth row indices corresponding to the n largest values; Merge and deduplicate the four row index sets mentioned above to obtain the total set of row indexes to be removed. Remove all rows belonging to the total set from the intersection coordinate array, and use the remaining coordinate data to form the cleaned intersection coordinate array.
7. The method according to claim 1, characterized in that, The calculation of the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, and the correlation test based on the correlation coefficient, include: Calculate the Pearson correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence; The Pearson correlation coefficient is compared with a preset correlation threshold; If the Pearson correlation coefficient is greater than or equal to the correlation threshold, the correlation test is deemed to have passed. If the Pearson correlation coefficient is less than the correlation threshold, the correlation test is deemed to have failed.
8. The method according to claim 7, characterized in that, Also includes: When the convex trend test or the correlation test fails, a judgment result indicating that the indicator test has failed is generated. Based on the determination result, the target parameters are adjusted, and the target parameters include at least one of the following: the boundary value of the radial distortion coefficient range, the sampling interval of the discrete sampling, the proportional threshold in the convexity trend test, and the correlation threshold in the correlation test; Using the adjusted target parameters, the method for estimating the radial distortion coefficients of the image is re-executed.
9. An apparatus for estimating the radial distortion coefficient of an image, characterized in that, The device includes: The sampling module is used to perform discrete sampling within the range of radial distortion coefficients corresponding to the original image to be corrected, and generate an ordered sequence of distortion coefficients. A generation module is used to generate a sequence of x-direction dispersion statistics and a sequence of y-direction dispersion statistics after performing a parameter traversal operation on each current distortion coefficient in the distortion coefficient sequence. The parameter traversal operation includes: calculating the output image size based on the current distortion coefficient and the size of the original image; performing trial correction on the original image using the current distortion coefficient and the output image size to obtain a current trial-corrected image; performing line detection and filtering on the current trial-corrected image to obtain a set of valid line segments; performing random sampling and calculation on the set of valid line segments to obtain an array of intersection coordinates; removing outliers from the array of intersection coordinates to obtain a cleaned array of intersection coordinates; and calculating the x-direction dispersion statistics and y-direction dispersion statistics based on each coordinate in the cleaned array of intersection coordinates. The trend testing module is used to perform a downward convexity trend test on the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence, respectively. The correlation test module is used to calculate the correlation coefficient between the x-direction dispersion statistics sequence and the y-direction dispersion statistics sequence when both sequences pass the convexity trend test, and to perform a correlation test based on the correlation coefficient. The determination module is used to determine the distortion coefficient corresponding to the global minimum point in the x-direction dispersion statistics sequence or the y-direction dispersion statistics sequence when the correlation test passes, and to use the distortion coefficient as the estimated optimal radial distortion coefficient.
10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method for estimating the radial distortion coefficients of an image as described in any one of claims 1 to 8.
