A curve detection method suitable for a camera model with ultra-large field of view distortion

Abstract

The application provides a curve detection method suitable for a camera model with ultra-large field of view distortion, which can realize ultra-high processing speed, extract more curve features in the same image, and significantly enhance the relevance and interpretability of the detection results. The initial rapid exploration and refinement stage includes: S11, mapping the local gradient of the curve to multiple equidistant directions, and quantizing the direction of the curve; S12, saving the state of the curve when the curve reaches the extension limit in the current direction, and selecting a new direction with the greatest extension possibility in the remaining directions to continue growing; S13, using gradient constraint and distance constraint to quickly select the candidate extension point of the initial extended curve; S2, the final integrity verification stage includes: S21, verifying the geometric relevance of two curves; S22, if the curve fails to pass the geometric relevance verification, the optimal splicing candidate is screened through global search matching, and splicing is completed; S23, abnormal detection and elimination are performed on the spliced curve.

Description

Technical Field

[0001] This invention relates to a curve detection method applicable to ultra-large field-of-view distortion camera models, and relates to the fields of machine vision and image feature extraction technology. Background Technology

[0002] Robot visual positioning technology, as a core direction of current artificial intelligence and robotics, has significant advantages. By acquiring image or video information from the environment, visual positioning technology can achieve 3D modeling and dynamic perception of complex scenes. Compared with traditional positioning methods (such as LiDAR or inertial navigation), visual positioning technology is characterized by lower cost, lighter weight, and stronger environmental adaptability. Furthermore, visual positioning can simultaneously perform pose estimation and environmental perception, providing robots with rich perceptual data and showing broad application potential in autonomous navigation, path planning, and obstacle avoidance.

[0003] Despite its numerous advantages, visual positioning technology still faces a series of challenges. First, in complex or dynamic environments, such as scenes with drastic lighting changes, monotonous textures, or significant occlusion, traditional visual positioning algorithms are prone to positioning drift or failure. Second, the widespread use of ultra-large field-of-view distortion cameras has increased the difficulty of nonlinear distortion processing for visual positioning, placing higher demands on the accurate extraction and description of image features. Furthermore, visual positioning is computationally intensive, especially in scenarios with high real-time requirements; balancing processing speed and resource consumption remains a critical issue to be addressed.

[0004] Current research primarily focuses on line segment detection and visual-inertial odometry methods for ultra-large field-of-view distortion cameras. Existing technologies include: Unified Line Segment Detection (ULSD), which uses Bézier curves to achieve unified detection of line segments in both distorted and undistorted images; however, it extracts a limited number of curves, failing to fully utilize image information, and suffers from speed limitations in ultra-large field-of-view images, while also lacking object-level correlation in the output line segments. The LF-PGVIO framework, through Omnidirectional Curve Segment Detection (OCSD), directly extracts great circle curve segments from distorted images, solving the line segment extraction problem in negative plane field-of-view scenes; however, it suffers from insufficient number and diversity of curves extracted from a single image, and struggles to achieve fine object-level description. In contrast, the algorithm proposed in this invention supports a unified ultra-large field-of-view distortion camera model, achieving ultra-high processing speed and extracting more curve features from the same image. Furthermore, it significantly enhances the correlation and interpretability of the detection results through lightweight object-level description, thus overcoming the shortcomings of existing methods in processing efficiency, number of curves, and object-level description. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned shortcomings in the prior art and provide a curve detection method suitable for ultra-large field-of-view distortion camera models. It supports a unified ultra-large field-of-view distortion camera model, can achieve ultra-high processing speed, and extract more curve features in the same image. At the same time, it significantly enhances the correlation and interpretability of the detection results, thereby making up for the shortcomings of existing methods in terms of processing efficiency, number of curves, and object-level description.

[0006] The technical solution adopted by the present invention to solve the above problems is: a curve detection method applicable to ultra-large field-of-view distortion camera models, including an initial rapid exploration and refinement stage and a final integrity verification stage; S1, Initial Rapid Exploration and Refinement Phase, this phase includes: S11, Multi-directional edge drawing mechanism: Maps the local gradient of the curve to multiple equidistant directions, and then quantizes the direction of the curve; S12, Omnidirectional growth mechanism: When a curve reaches its extension limit in the current direction, it first saves its state, and then selects the new direction with the greatest potential for extension in the remaining directions to continue growing. S13. Rapid exploration and refinement mechanism: Using gradient constraints and distance constraints, candidate extension points of the initial expansion curve are quickly selected. If the candidate extension point is verified, the curve is rapidly extended; otherwise, fine fitting is performed. S2, Final Integrity Verification Phase, which includes: S21. Geometric correlation verification: Verify the geometric correlation between two curves. When two curves are identified as having a geometric correlation, further determine whether they can be seamlessly spliced ​​together. S22. Global Search Matching: If the curve fails the geometric correlation verification, the optimal splicing candidate is selected through global search matching and the splicing is completed. S23. Abnormal Curve Removal: Perform abnormal detection and removal on the spliced ​​curves.

[0007] Step (2) of S11 of the present invention includes the following steps: (1) Taking the pixel P(x,y) in the image as the center, calculate the gradient dx and dy of the gray level of the pixel's neighborhood in the horizontal and vertical directions, and then calculate the azimuth angle θ and the positive guiding direction Dir of the pixel. pos ; (2) Calculate the inversion guide direction Dir corresponding to the pixel. neg .

[0008] In S11 of the present invention, the local gradient is mapped to eight equidistant directions; The azimuth angle θ of the pixel and the positive guiding direction Dir pos The calculation method is as follows: , ; The reversed guiding direction Dir neg The calculation method is as follows: .

[0009] S13 of the present invention includes the following steps: (1) Gradient constraint: Calculate the pixel gradient sequence {G} through a sliding window. i If the median difference ΔG of the interval meets the threshold, then the distance constraint process begins. (2) Distance constraint: First, calculate the three-dimensional coordinates P of the current extension point pixel. 3D The three-dimensional planar distance d between the target curve and the target curve plane If the three-dimensional plane distance d plane If the threshold is met, then the 3D coordinates P of the current extended point pixel will be... 3D Project onto a two-dimensional plane and calculate the reprojection error d. reproj ; (3) Rapid exploration and evaluation: combining the median difference ΔG of gradient constraints with the reprojection error d of distance constraints reproj Define the evaluation index η; if the evaluation index η meets the threshold, the current extension point is verified through the rapid exploration phase and the curve is rapidly extended; otherwise, it needs to enter the refinement phase. (4) Refinement stage: The curve is finely fitted using the least squares method, and the projection deviation e at each point is calculated again. i ; Summing the projection deviations e of all points i The sum of these values ​​is used as the cumulative error. If the cumulative error exceeds the threshold, growth in the current direction is stopped and the omnidirectional growth mechanism is given a new decision.

[0010] The median difference ΔG and the three-dimensional planar distance d described in this invention plane Reprojection error d reproj Evaluation index η, projection bias e i The calculation method is as follows: ΔG=median(|G i -G d |), i∈[1,5], , , , , Among them, G dLet P be the current pixel gradient, n be the direction vector of the curve, Π be the projection matrix of the equivalent sphere onto the pixel plane, and P be the gradient of the current pixel. 2D Let T be the coordinates of the target pixel in the two-dimensional plane, ρ be the weight factor for the gradient constraint and the distance constraint, and T be the distance. G and T d These are the normalized thresholds for the gradient and distance, respectively, P. i For all three-dimensional points in the curve.

[0011] S21 of the present invention includes the following steps: (1) Normalize the direction vector of the curve and calculate the angle Δθ between its direction vectors. If the angle Δθ between the direction vectors meets the threshold, it means that the two curves tend to be consistent in direction. (2) After ensuring the consistency of direction, verify the coordinate continuity of the curve.

[0012] The method for calculating the angle Δθ between the direction vectors described in this invention is as follows: , Where n1 and n2 are the direction vectors of the two curves.

[0013] In step (2) of S21 of the present invention, the coordinate continuity is verified by calculating the average Euclidean distance Δcoord between adjacent points of two geometrically related curves. The calculation method is as follows: , Where k is the number of adjacent pairs, dis i Let be the Euclidean distance between the i-th pair of adjacent points.

[0014] S22 of the present invention includes the following steps: (1) Based on directional similarity S θ Similarity S by distance d The weighted average global similarity S is calculated as follows: S=ωS θ +(1-ω)S d Where ω is the weighting coefficient; (2) Define the directional similarity S by the angle Δθ between the direction vectors of the two curves. θ The distance d between the two endpoints of the curve in three-dimensional space 3D Measuring distance similarity S d The calculation method is as follows: , , Where, when Δθ→0, S θ →1, the more similar the directions; d max d represents the maximum allowable spacing between endpoints. 3D →0, Sd →1, the closer the distance; (3) If the global similarity S exceeds the preset threshold, the curve is spliced ​​with the optimal matching curve.

[0015] In S23 of this invention, the variance of the local gradient features of the spliced ​​curve is statistically analyzed using a sliding window. With gradient mean Abnormal fluctuation areas are detected and removed, and the calculation method is as follows: , , Where N is the number of pixels within the sliding window, G i This represents the gradient value of the i-th pixel after downsampling. When gradient variance With gradient mean If all values ​​meet the preset threshold, the local features of the curve segment are determined to be stable; otherwise, it is considered an abnormal curve and is removed.

[0016] Compared with the prior art, the present invention has the following advantages and effects: 1. This invention enables fast, accurate, and efficient curve feature detection in any ultra-large field-of-view distortion camera model, significantly improving the completeness and structural correlation of the detection results. Furthermore, this invention supports lightweight object-level detection, providing a unified description of curve features within the same object, making it suitable for high-performance vision tasks in complex scenarios.

[0017] 2. The multi-directional edge drawing mechanism enables accurate tracking of curve edges under nonlinear distortion. The least squares method is used to complete the fine fitting of the curve in the refinement stage. In the final stage, geometric correlation verification and global retrieval and splicing are used to ensure the spatial consistency of the curve.

[0018] 3. The omnidirectional growth mechanism enables multi-directional dynamic exploration of curves, breaking through the limitations of traditional unidirectional growth. It can extract more and more diverse curve features, while effectively solving the problems of missed detection and breakage of curves in strongly distorted regions. It can still stably extract long line segments in the edge regions of ultra-large field-of-view distorted images.

[0019] 4. It achieves unified aggregation of curve features within the same object, while ensuring the structural coherence of the curves in the final integrity verification stage. This solves the problems of discrete line segments and lack of object-level correlation in existing algorithms, providing a more reliable feature foundation for subsequent object-level analysis and 3D reconstruction.

[0020] 5. This invention surpasses existing ULSD and OCSD algorithms in several key indicators, particularly demonstrating significant advantages in detection accuracy, robustness, and processing speed. In complex fisheye and panoramic distortion scenarios, this invention exhibits stronger adaptability and stability, providing strong support for efficient curve detection in practical applications. This invention not only improves the accuracy and robustness of curve detection but also effectively increases processing speed, providing reliable technical support for large-scale, efficient image analysis tasks. Attached Figure Description

[0021] Figure 1 This is a framework diagram of an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of a curve segment in the two-dimensional pixel plane of the input image in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the curve in the three-dimensional equivalent sphere of the input image in an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of eight equidistant directions in an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram of the detection curve using the traditional unidirectional growth mechanism, which detects the complete object curve as cyan and brown respectively.

[0026] Figure 6 This is a schematic diagram of the detection curve of the omnidirectional growth mechanism in an embodiment of the present invention. The omnidirectional growth mechanism detects the complete object curve as dark blue.

[0027] Figure 7 This is a schematic diagram of the calculation of the median difference ΔG in the gradient constraint process according to an embodiment of the present invention; in the figure, G1~G5 are five pixel gradients in the sliding window, which slide as the current pixel is extended.

[0028] Figure 8 This is a schematic diagram of calculating the three-dimensional plane distance during the distance constraint process in an embodiment of the present invention. The red squares simulate the trend of the target curve, and the blue squares simulate the trend of the quasi-curve being extracted.

[0029] Figure 9 This is a schematic diagram illustrating the calculation of reprojection error during the distance constraint process in an embodiment of the present invention. The red squares simulate the trend of the target curve segment, and the blue squares simulate the trend of the quasi-curve segment being extracted.

[0030] Figure 10 This is a schematic diagram illustrating the calculation of the angle between direction vectors and the average Euclidean distance between adjacent points in the geometric correlation verification of an embodiment of the present invention.

[0031] Figure 11 This is a schematic diagram illustrating the calculation of gradient variance and gradient mean within a sliding window for the spliced ​​curve in an embodiment of the present invention.

[0032] Figure 12 The images show a comparison between the original image and the image detected according to the embodiments of the present invention, where (a) is the original image and (b) is the image detected.

[0033] Figure 13 This is a comparison chart of the curve detection performance of the present invention and the existing OCSD method on the fisheye dataset in a light object-level curve detection experiment.

[0034] Figure 14 This image shows a comparison of the curve detection performance of this invention and the existing OCSD method on a panoramic dataset in a light-object-level curve detection experiment.

[0035] Figure 15 The figure shows a comparison of the curve detection performance of this invention with existing OCSD and ULSD methods on a panoramic dataset in a rotation robustness experiment. In the figure, the green curve represents the detection result of the algorithm in the original image, and the red curve represents the curve that can still be repeatedly detected in the original image after the viewpoint is rotated. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0037] I. Embodiments of the present invention.

[0038] See Figure 1 This invention provides an efficient curve detection method suitable for ultra-large field-of-view distortion camera models. It addresses the problems of low processing speed, insufficient detection accuracy, and limited curve feature extraction in existing technologies by proposing a systematic solution. A significant feature of this method is the introduction of an innovative technical framework, including an initial rapid exploration and refinement stage and a final integrity verification stage. The initial rapid exploration and refinement stage consists of a multi-directional edge drawing mechanism, an omnidirectional growth mechanism, and a rapid exploration and refinement mechanism. The final integrity verification mechanism consists of geometric correlation verification, global retrieval matching, and abnormal curve removal.

[0039] This invention includes an initial rapid exploration and refinement phase, and a final integrity verification phase.

[0040] S1. Initial rapid exploration and refinement stage, which includes multi-directional edge drawing mechanism, omnidirectional growth mechanism, rapid exploration and refinement mechanism, to complete the initial extraction and refinement of curve features.

[0041] S11, Multi-directional edge drawing mechanism: Multi-directional edge rendering mechanisms are used to flexibly and accurately extract and extend curve features under nonlinear distortion. Unlike traditional line segment detection, which often suffers from missed detections, offsets, and local instability in scenes with large distortions, multi-directional edge rendering mechanisms continuously track edge growth trends within a unified directional domain by mapping local gradients to multiple equidistant directions. Specifically, the mechanisms include the following steps: (1) Map the local gradient of the curve to multiple equidistant directions, see Figure 4 In this embodiment, the local gradient is mapped to eight equidistant directions; (2) Quantify the direction of the curve: (21) Taking the pixel P(x,y) in the image as the center, calculate the gradients dx and dy of the neighborhood gray level in the horizontal and vertical directions, and then calculate the azimuth angle θ and the positive guiding direction Dir of the pixel. pos The calculation method is as follows: , .

[0042] (22) The edge extension process needs to take into account the reverse guidance direction to ensure global consistency and achieve mirror balance and complementarity of the direction space. Therefore, it is necessary to calculate the reverse guidance direction Dir corresponding to the pixel. neg : .

[0043] Leveraging a multi-directional edge rendering mechanism, the algorithm efficiently extends curve edges in eight preset directions using only simple numerical calculations, eliminating the need for complex filtering or extensive statistical analysis. Combined with a unified geometric representation of an equivalent spherical model, the algorithm operates robustly under various distortion conditions, avoiding blind spots and error accumulation common in traditional methods in areas of strong distortion and high curvature.

[0044] S12, Omnidirectional growth mechanism.

[0045] See Figure 5 Existing edge rendering algorithms typically extend the curve in a straight line along its initial growth direction until it can no longer continue. In scenarios with nonlinear distortion or complex structures, such unidirectional strategies struggle to detect curve bifurcation and multidirectional extension potential. To overcome this bottleneck, this invention proposes an omnidirectional growth mechanism based on multidirectional edge rendering mechanisms, enabling curve extension to have dynamic switching and adaptive capabilities.

[0046] See Figure 6The omnidirectional growth mechanism tightly couples the direction quantization results of the multi-directional edge rendering mechanism with the curve extension process: when the curve reaches its extension limit in the current direction, it first saves its state, and then selects the new direction with the most potential for extension from the remaining seven directions to continue growing, without re-initializing the curve. Through multiple state saving and direction switching, the curve can change direction in time when encountering obstacles, achieving full coverage of potential edges.

[0047] This mechanism transforms curve detection from unidirectional linear expansion to multidirectional dynamic exploration, enhancing the ability to fully extract object-level curve structures and ensuring the continuity and robustness of detection results in high-noise and high-distortion scenarios.

[0048] S13, Rapid exploration and refinement of mechanisms.

[0049] Building upon the omnidirectional growth mechanism, this invention introduces a rapid exploration and refinement mechanism to balance growth speed and local accuracy. This mechanism first utilizes gradient and distance constraints to rapidly select candidate extension points for the initial expansion curve. If a candidate extension point passes verification, the curve extends rapidly; otherwise, the least squares method is used for fine fitting to ensure both efficiency and accuracy in curve extraction. Specifically, it includes the following steps: (1) Gradient constraints: see Figure 7 The pixel gradient sequence {G} is calculated using a sliding window. i If the median difference ΔG meets the threshold, the distance constraint process begins; the calculation method is as follows: ΔG=median(|G i -G d |), i∈[1,5],G d This represents the gradient of the current pixel.

[0050] (2) Distance constraints: See Figure 8 First, calculate the three-dimensional coordinates P of the current extended point pixel. 3D The three-dimensional planar distance d between the target curve and the target curve plane See also Figure 9 If the three-dimensional plane distance d plane If the threshold is met, then the 3D coordinates P of the current extended point pixel will be... 3D Project onto a two-dimensional plane and calculate the reprojection error d. reproj The calculation method is as follows: , , Where n is the direction vector of the curve, Π is the projection matrix of the equivalent sphere onto the pixel plane, and P 2D These are the pixel coordinates of the target in a two-dimensional plane.

[0051] (3) Rapid exploration and evaluation: combining the median difference ΔG of gradient constraints with the reprojection error d of distance constraints reproj Define the evaluation index η: , Where ρ is the weighting factor for gradient constraints and distance constraints, and T G and T d These are the normalized thresholds for the gradient and distance, respectively.

[0052] If the evaluation index η meets the threshold, the current extension point is verified through the rapid exploration phase, and the curve can be extended quickly without additional refinement; otherwise, it needs to enter the refinement phase.

[0053] (4) Refinement stage: The curve is finely fitted using the least squares method to ensure the overall shape of the curve, and the projection deviation e at each point is recalculated. i The calculation method is as follows: , Among them, P i For all three-dimensional points in the curve.

[0054] Find the small deviation e of all points on the quasi-curve segment. i The sum of these is used as the cumulative error, which serves as a global consistency consideration for the overall structure of the latest curve. If the cumulative error exceeds the threshold, growth in the current direction is stopped and the omnidirectional growth mechanism is used to make a new decision.

[0055] The initial rapid exploration and refinement mechanism not only achieves rapid extension of local structures with low computational cost in the early stage, but also performs global optimization when necessary, thus exhibiting high elasticity and robustness in high noise and high distortion scenarios.

[0056] S2, the final integrity verification stage, consists of geometric correlation verification, global search and matching, and abnormal curve removal, which are performed sequentially to complete the curve splicing and optimization, ensuring the integrity, continuity, and effectiveness of the detection curve. Specifically, it includes the following steps: S21, Geometric correlation verification.

[0057] The geometric correlation of curves reflects whether they belong to the same object contour. This correlation is jointly determined by the multi-directional edge drawing mechanism and the omnidirectional growth mechanism. When two curves are identified as having a geometric correlation, it is necessary to further determine whether they can be seamlessly spliced. Specifically, this includes the following steps: (1) See Figure 10 The direction vectors of the curves are normalized, and the angle Δθ between their direction vectors is calculated. If the angle Δθ meets a threshold, it indicates that the two curves tend to be consistent in direction. The calculation method is as follows: , Where n1 and n2 are the direction vectors of the two curves.

[0058] (2) See Figure 10 After ensuring directional consistency, it is also necessary to examine the coordinate continuity of the curves. This is verified by calculating the average Euclidean distance Δcoord between adjacent points of two geometrically related curves to ensure that there are no abrupt breaks or discrete jumps in the spatial distribution. The calculation method is as follows: , Where k is the number of adjacent pairs, dis i Let be the Euclidean distance between the i-th pair of adjacent points; If the average Euclidean distance Δcoord meets the threshold, it can be determined that the two curve segments are smoothly connected at the pixel level, and the two curves can be spliced ​​together on a macro scale to form a curve structure without interruption and with a coordinated geometric relationship.

[0059] S22, Global Search Matching.

[0060] If the curve fails the geometric correlation verification, the final integrity verification mechanism further proposes a global retrieval matching. The optimal splicing candidate is selected through global retrieval matching and the splicing is completed. The specific steps include: (1) Based on directional similarity S θ Similarity S by distance d The weighted average global similarity S is calculated as follows: S=ωS θ +(1-ω)S d , where ω is the weighting coefficient.

[0061] (2) Define the directional similarity S by the angle Δθ between the direction vectors of the two curves. θ The distance d between the two endpoints of the curve in three-dimensional space 3D Measuring distance similarity S d The calculation method is as follows: , , Where, when Δθ→0, S θ →1, the more similar the directions; d max d represents the maximum allowable spacing between endpoints. 3D →0, S d →1, the closer the distance.

[0062] (3) If the global similarity S exceeds the preset threshold, the curve is spliced ​​with the optimal matching curve.

[0063] The global similarity S takes into account both directional consistency and distance consistency, providing a unified and adjustable evaluation standard for curve matching, which facilitates the selection of the optimal splicing candidate from multi-dimensional information.

[0064] S23, Abnormal curve removal.

[0065] Even after ensuring that similar curves have been stitched together in the geometric correlation verification and global retrieval matching stages, the stitched curves may still contain abnormal curves due to noise interference, local feature distortion, or algorithm errors. Therefore, anomaly detection and removal are necessary for the stitched curves. See also Figure 11 To improve detection efficiency, the abnormal curve removal stage first performs equidistant downsampling on the stitched curves. Then, a sliding window is used to statistically analyze the variance of the local gradient features of the stitched curves. With gradient mean Abnormal fluctuation areas are detected and removed, and the calculation method is as follows: , , Where N is the number of pixels within the sliding window, and G is the gradient value of the pixel. i This represents the gradient value of the i-th pixel after downsampling.

[0066] When gradient variance With gradient mean If all values ​​meet the preset threshold, the local features of the curve segment are determined to be stable; otherwise, it is considered an abnormal curve and is removed.

[0067] After the final integrity verification stage, the merged curves achieved a high level of performance in terms of direction, coordinates, and local consistency, while maintaining an advantage in recalling fragmented curves. Thus, this invention completes the entire evolution from initial detection to global continuous representation in highly distorted and complex scenes, ensuring the accuracy and robustness of curve detection.

[0068] The final image detection curve of the ultra-large field-of-view camera is as follows: Figure 12 As shown.

[0069] II. Comparative experiment between the present invention and the prior art.

[0070] This experiment evaluates the performance of the present invention on various complex scenes and multi-source distortion datasets through systematic experimental design and comparative analysis. Based on a unified evaluation system, this experiment examines the algorithm's object-level description capability, accuracy, repeatability consistency, and cross-camera universality from four aspects: lightweight object-level curve detection, ground truth curve detection, rotation robustness, and evaluation on a self-built dataset. This design ensures that the method has high robustness and versatility in practical applications.

[0071] 1. Public datasets.

[0072] To obtain a rigorous and comprehensive evaluation, this experiment selected multiple datasets for three imaging modes: fisheye, panoramic, and rotating equivalent spherical.

[0073] 2. Lightweight object-level curve detection experiment.

[0074] In highly distorted panoramic and fisheye images, this invention not only detects a greater number and wider coverage of linear features, but also intuitively presents object-level correlations by using a unified color to label multiple line segments on the same object. In contrast, other algorithms only return a small number of discrete line segments, lacking semantic aggregation of those belonging to the same object, making it difficult to form an overall structure.

[0075] See Figure 13 and Figure 14 This invention displays clear color groupings on structures such as beams, door frames, and roof supports, allowing for easy identification of the line segment relationships within a building or interior object at a glance. In contrast, the results of other methods are monotonous, scattered sets of line segments, lacking visual aggregation. More importantly, this invention can stably extract long line segments in areas of extreme distortion (such as the top and bottom edges of panoramic images or the edge areas of fisheye images), maintaining the height correspondence between their endpoints and the edges of real objects. Other methods often result in broken or excessively short line segments.

[0076] This invention outperforms comparative methods in terms of detection completeness, object-level correlation, and robustness in extreme regions, providing a more reliable foundation for subsequent object-level analysis, 3D reconstruction, and multimodal SLAM.

[0077] 3. Truth curve detection experiment.

[0078] To rigorously evaluate the detection accuracy of various algorithms in nonlinear distortion scenarios, this experiment selected fisheye and panoramic datasets with ground truth annotations for comparative testing. The evaluated methods included this invention, as well as existing OCSD and ULSD methods for curve detection in ultra-large field-of-view distortion cameras. Two types of distance metrics were used for evaluation: when the distance between the detected curve and the ground truth was less than 5 pixels, the repetition rate Rep-5, localization error LE-5, and repetition efficiency REI-5 were calculated respectively.

[0079] This invention consistently ranks first in both orthogonal distance and structural distance evaluation systems for both fisheye and panoramic distortion scenarios. It also leads in metrics such as repetition rate (Rep), positioning error (LE), repetition efficiency (REI), detection efficiency (DEI), and frame rate (FPS), fully demonstrating that this invention achieves high-speed processing capabilities while maintaining high-precision detection.

[0080] Taking the fisheye dataset as an example, under the definition of orthogonal distance, the processing speed of this invention reaches 110 FPS, which is 2.9 times and 13.4 times that of OCSD and ULSD, respectively; Rep is improved to 75.9% (5.8 times and 2.5 times), LE is only 2.9px (approximately 29% of OCSD and 14% of ULSD); REI reaches 4.4 records / ms, which is 15.9 times and 26.8 times that of OCSD and ULSD, respectively, achieving the best balance between speed and accuracy. Under the definition of structural distance, Rep remains at 75% (5.6 times that of OCSD and 1.6 times that of ULSD), LE is only 0.47px (21% of OCSD and 27% of ULSD); REI is 4.39, and DEI is as high as 27.3 records / ms, showing that it can still quickly and accurately reproduce curve features under more stringent metrics.

[0081] 4. Rotational robustness test.

[0082] This experiment aims to evaluate the stable detection capability of the same algorithm on the original image and on images after distortion rotation using fisheye or panoramic models, thereby verifying its detection consistency under conditions of viewpoint change and dynamic distortion. Nine representative distortion datasets were selected, covering both fisheye and panoramic scenes, providing sufficient evidence for evaluating the algorithm's performance under multiple distortion models.

[0083] Experimental Procedure: First, OCSD, ULSD, and the present invention were applied to the original image to obtain an initial curve set segs1. Then, fisheye or panoramic rotation distortion was applied to the original image, and the same algorithm was run again to obtain segs2. Simultaneously, segs1 was rotated using the same distortion to obtain segs1_rotate. Finally, segs1_rotate and segs2 were compared, and the consistency of the algorithm's curves under different viewpoints was quantified based on nine evaluation metrics.

[0084] See Figure 15 The present invention detects a large number of curves (green) in the original image, which are continuously distributed and can still be reproduced with a high coverage (red) after rotation, which is significantly better than OCSD and ULSD. The latter two show obvious omissions and breaks under large-angle viewing angle changes, reflecting their insufficient adaptability to distortion rotation.

[0085] Quantitative results further confirm the above conclusions. Taking the fisheye dataset as an example, the Rep-5 accuracy of this invention reaches 37.5%, a 58.1% improvement over OCSD and far exceeding ULSD; LE-5 is only 1.026px, significantly better than OCSD's 1.142px and ULSD's 6.081px; REI-5 reaches 9.658 records / ms, 10.3 times that of OCSD and over a thousand times that of ULSD; DEI is 17.147 records / ms, 35.6 times that of OCSD; and the frame rate reaches 73.9 FPS, which is 2.0 times that of OCSD and 9.0 times that of ULSD, respectively. In summary, this invention combines high detection rate, low positioning error, and significant real-time performance in environments with complex distortion and changing viewpoints, demonstrating excellent robustness and practicality.

[0086] Furthermore, it should be noted that the specific embodiments described in this specification may differ in the shape and name of their components, etc. The above description is merely illustrative of the structure of the present invention. All equivalent or simple variations made based on the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art can make various modifications or additions to the described specific embodiments or use similar methods to substitute them, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, all of which should fall within the protection scope of this invention.

Claims

1. A curve detection method suitable for ultra-large field of view distortion camera model, characterized in that: This includes an initial rapid exploration and refinement phase and a final integrity verification phase; S1, Initial Rapid Exploration and Refinement Phase, this phase includes: S11, Multi-directional edge drawing mechanism: Maps the local gradient of the curve to multiple equidistant directions, and then quantizes the direction of the curve; S12, Omnidirectional growth mechanism: When the curve reaches the extension limit in the current direction, its state is first saved, and then the new direction with the most possible extension is selected in the remaining directions to continue growing. S13. Rapid exploration and refinement mechanism: Using gradient constraints and distance constraints, candidate extension points of the initial expansion curve are quickly selected. If the candidate extension point is verified, the curve is rapidly extended; otherwise, fine fitting is performed. S2, Final Integrity Verification Phase, this phase includes: S21. Geometric correlation verification: Verify the geometric correlation between two curves. When two curves are identified as having a geometric correlation, further determine whether they can be seamlessly spliced ​​together. S22. Global Search Matching: If the curve fails the geometric correlation verification, the optimal splicing candidate is selected through global search matching and the splicing is completed. S23. Abnormal Curve Removal: Perform abnormal detection and removal on the spliced ​​curves.

2. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 1, characterized in that: In S11, the direction quantization of the curve includes the following steps: (1) take the pixel point P(x, y) in the image as the center, calculate the gradient dx and dy of the pixel point neighborhood gray scale in the horizontal and vertical directions, and then calculate the azimuth angle θ and the positive guide direction Dir of the pixel point pos ; (2) calculate the corresponding pixel point of the reverse guide direction Dir neg .

3. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 2, characterized in that: In S11, the local gradient is mapped to eight equidistant directions; The azimuth angle θ of the pixel point and the positive guide direction Dir pos The calculation is as follows: , ； The reverse direction Dir neg The calculation is as follows: 。 4. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 1, characterized in that: S13 includes the following steps: (1) Gradient constraint: Calculate the pixel gradient sequence {G} through a sliding window. i If the median difference ΔG of the interval meets the threshold, then the distance constraint process begins. (2) Distance constraint: First, calculate the three-dimensional coordinates P of the current extension point pixel. 3D The three-dimensional planar distance d between the target curve and the target curve plane If the three-dimensional plane distance d plane If the threshold is met, then the 3D coordinates P of the current extended point pixel will be... 3D Project onto a two-dimensional plane and calculate the reprojection error d. reproj ; (3) Fast exploration evaluation: combine the gradient-constrained median difference AG with the distance-constrained re-projection error d reproj , define the evaluation index η; if the evaluation index η meets the threshold value, the current extension point passes the fast exploration phase verification, and the curve is extended quickly, otherwise it needs to enter the refinement stage; (4) Refinement stage: through the least square method to curve fine fitting, and again calculate each point projection deviation e i ; Find the projection deviation e of all points. i The sum of these values ​​is used as the cumulative error. If the cumulative error exceeds the threshold, growth in the current direction is stopped and the omnidirectional growth mechanism is given a new decision.

5. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 4, characterized in that: The median difference ΔG, three-dimensional plane distance d plane , reprojection error d reproj , evaluation index η, projection deviation e i The calculation is as follows: AG = median(|G i - G d |), i e [1,5], , , , , Among them, G d Let P be the current pixel gradient, n be the direction vector of the curve, Π be the projection matrix of the equivalent sphere onto the pixel plane, and P be the gradient of the current pixel. 2D Let T be the coordinates of the target pixel in the two-dimensional plane, ρ be the weight factor for the gradient constraint and the distance constraint, and T be the distance. G and T d These are the normalized thresholds for the gradient and distance, respectively, P. i For all three-dimensional points in the curve.

6. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 1, characterized in that: S21 includes the following steps: (1) Normalize the direction vector of the curve and calculate the angle Δθ between its direction vectors. If the angle Δθ between the direction vectors meets the threshold, it means that the two curves tend to be consistent in direction. (2) After ensuring the consistency of direction, verify the continuity of the curve coordinates.

7. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 6, characterized in that: The angle Δθ between the direction vectors is calculated as follows: , Where n1 and n2 are the direction vectors of the two curves.

8. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 6, characterized in that: In step (2) of S21, the coordinate continuity is verified by calculating the average Euclidean distance Δcoord between adjacent points of two geometrically related curves. The calculation method is as follows: ， where k is the number of adjacent point pairs, dis i is the Euclidean distance of the ith adjacent point pair.

9. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 6, characterized in that: S22 includes the following steps: (1) The global similarity S is weighted by the direction similarity S θ and the distance similarity S d , and is calculated as follows: S = ωS θ + (1 - ω)S d where ω is a weight coefficient. (2) Define the directional similarity S by the angle Δθ between the direction vectors of the two curves. θ The distance d between the two endpoints of the curve in three-dimensional space 3D Measuring distance similarity S d The calculation method is as follows: , , Where, when Δθ→0, S θ →1, the more similar the directions; d max d represents the maximum allowable spacing between endpoints. 3D →0, S d →1, the closer the distance; (3) If the global similarity S exceeds the preset threshold, the curve is spliced ​​with the optimal matching curve.

10. The method for curve detection suitable for ultra-large field of view distortion camera model according to claim 1, characterized in that: In the S23, the variance of the local gradient features of the spliced curve is calculated by using a sliding window and the gradient mean value The abnormal fluctuation region is detected and removed, and the calculation method is as follows: ， ， Where N is the number of pixels within the sliding window, G i This represents the gradient value of the i-th pixel after downsampling. When gradient variance With gradient mean If all values ​​meet the preset threshold, the local features of the curve segment are determined to be stable; otherwise, it is considered an abnormal curve and is removed.

Images