Marker point error identification and elimination method, device and equipment and storage medium

Abstract

The application discloses a kind of mark point error identification and elimination method, device, equipment and storage medium, method is by obtaining the observation ray set of target mark point under multiple camera visual angle, establish local three-dimensional coordinate system with target mark point as origin, and according to the included angle distribution of observation ray relative to the main shaft of local three-dimensional coordinate system, observation ray set is divided into at least two direction regions;According to the ray number distribution in each direction region divided out, the overall observation quality of target mark point is judged, when overall observation quality satisfies preset condition, the comprehensive score of each observation ray is carried out, and the observation ray that comprehensive quality score does not satisfy preset screening condition is eliminated, and the ray set after screening is obtained;According to ray set, the three-dimensional coordinates of target mark point are calculated by preset space front intersection algorithm;It can realize the deep evaluation of mark point overall observation geometric quality, and the precision, stability and algorithm robustness of mark point three-dimensional coordinate reconstruction are improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and photogrammetry, and in particular to a method, apparatus, device, and storage medium for identifying and eliminating marker point errors. Background Technology

[0002] In the fields of photogrammetry and 3D reconstruction, to achieve high-precision and automated measurement of the geometric structure of object surfaces, the method of setting artificial markers on the surface of the target object is widely adopted. Among them, circular markers (including solid circles, concentric rings, etc.) have become an industry standard due to their unique advantages, which mainly include: 1. Stable and high-precision detection: Circular markers are rotationally invariant in the image, and their clear edge features facilitate sub-pixel-level center positioning, thus providing extremely reliable high-precision two-dimensional observation data for 3D reconstruction.

[0003] 2. Decoding and matching are relatively simple: By designing specific coding patterns (such as circular coding points) or utilizing their geometric consistency in multi-view images, efficient and accurate automatic matching can be achieved, solving the problem of corresponding points of a large number of names.

[0004] 3. Applicable to various complex scenarios: In areas with missing surface textures, reflections, or repetitive patterns, the pre-set circular markers provide stable and unique visual features, ensuring the complete acquisition of 3D information.

[0005] While circular markers offer significant convenience, their measurement accuracy in practical engineering applications is influenced by numerous factors. Generally, the sources of error can be categorized as: errors caused by measuring instruments (cameras) and accessories, software algorithms, operators and the environment, and the geometric structure of the spatial measurement network. In close-range photogrammetry systems, the strength of the spatial network geometry configuration directly affects the final measurement accuracy, including factors such as the size of the object being measured, the photographic distance, the overlap of photographs, the number of photographs, the intersection angle of the cameras (camera station layout), and the angle of incidence of light.

[0006] In dense, regularly arranged, or similarly encoded groups of marker points, incorrect cross-view matching may occur, generating incorrect pairs of points with the same name. These error points are usually mixed in with the valid data. If they cannot be effectively removed, they will directly participate in subsequent core calculation processes such as camera calibration, bundle adjustment, or point cloud generation as "contaminated data." Since these adjustment algorithms are usually based on the least squares principle, they are extremely sensitive to gross errors. A few error points may cause the solution results to deviate significantly, resulting in local distortion of the 3D model, a decrease in overall accuracy, or even reconstruction failure.

[0007] Existing solutions include: Statistical filtering based on reprojection error: This is the most commonly used method. After completing the initial 3D reconstruction and camera pose estimation, the reprojection error of each marker point (i.e., the deviation between the 2D position of the 3D point back to each image and the actual detection position) is calculated, and out-of-limit points are removed with a statistical threshold (such as 2-3 times the mean error). This method is simple, but it is highly dependent on the accuracy of the initial solution. When there are many error points, the initial solution itself may be distorted, causing the filtering to fail.

[0008] A robust estimation framework based on RANSAC integrates the random sampling consensus algorithm into the motion recovery structure or calibration process to iteratively find the optimal set of inliers. This method is currently a relatively robust solution, but it involves huge computational costs and slow convergence speed and challenges in stability when the inlier rate is low or the data dimensionality is high.

[0009] Manual screening and removal: After the key steps, professionals visually inspect the 3D point cloud or residual report and manually delete outliers. Although this method is direct and effective, it is inefficient, lacks automation, and is highly subjective, failing to meet the needs of modern automated measurement systems and large-scale data processing.

[0010] Pre-filtering based on simple geometric or grayscale features: During the detection stage, obvious unqualified detection results are filtered out by judging primary features such as the roundness, contrast, and size range of the marker points. This type of method can remove some obvious anomalies, but it cannot identify cases of incorrect reconstruction caused by mismatch. Summary of the Invention

[0011] The main objective of this invention is to provide a method, apparatus, device, and storage medium for identifying and eliminating marker point errors. This aims to address the technical problems in the prior art where, during the 3D reconstruction of marker points in multi-view photogrammetry, the assumption that all detected marker points have the same quality or simple screening based solely on the number of visible cameras ignores the spatial distribution differences of the observed rays. This leads to insufficient geometric constraints, numerical instability, and excessive elimination, which in turn affects the accuracy and reliability of the 3D reconstruction of marker points, especially when the observation direction is highly concentrated, a few abnormal rays dominate, or observation conditions are poor.

[0012] In a first aspect, the present invention provides a method for identifying and removing marker point errors, the method comprising the following steps: Obtain the set of observation rays of the target marker point from multiple camera perspectives, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system. The overall observation quality of the target marker is determined based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet the preset screening conditions are removed to obtain the filtered ray set. The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm.

[0013] Optionally, the step of acquiring the set of observation rays of the target marker point from multiple camera perspectives, establishing a local three-dimensional coordinate system with the target marker point as the origin, and dividing the set of observation rays into at least two directional regions based on the angle distribution of the observation rays relative to the principal axes of the local three-dimensional coordinate system, includes: In the images acquired from each camera viewpoint, the target marker point is located and the two-dimensional center coordinates corresponding to the target marker point are obtained. The two-dimensional center coordinates are then projected back into three-dimensional space using the camera imaging model to form multiple spatial rays that converge near the target marker point. All spatial rays are then combined to form an observation ray set. Analyze the intersection of each observation ray in the set of observation rays in three-dimensional space, determine the estimated intersection point based on the intersection, and use the estimated intersection point as the initial three-dimensional estimated position of the target marker point; The initial three-dimensional estimated position is set as the origin of the local three-dimensional coordinate system, and the main camera optical axis direction, the average direction of all observed rays, or the principal direction obtained through principal component analysis are selected as the principal axis direction of the local three-dimensional coordinate system. Calculate the angle between each observation ray vector and the principal axis direction, analyze the distribution density of the angles according to the preset angle threshold range, classify observation rays with similar angle characteristics into the same directional region, and divide the set of observation rays into at least two directional regions.

[0014] Optionally, the step involves locating the target marker point in the images acquired from each camera's viewpoint and obtaining the corresponding two-dimensional center coordinates of the target marker point. The two-dimensional center coordinates are then projected back into three-dimensional space using a camera imaging model, forming multiple spatial rays converging near the target marker point. All spatial rays are then combined to form an observation ray set, including: Images containing target markers are acquired from various camera perspectives. The target markers are located using multi-view geometric constraints, and the two-dimensional center coordinates of the target markers are obtained. The fundamental matrix is ​​calculated using the calibrated camera parameters. The two-dimensional center point in the viewpoint is projected onto another viewpoint to form an epipolar line. Candidate points with a distance less than a preset threshold are filtered out, and a third viewpoint is introduced to verify and form an effective three-view matching pair. Using the camera imaging model and projection matrix of each viewpoint, each two-dimensional pixel in the effective three-view matching pair is back-projected into three-dimensional space to form multiple spatial rays that start from the camera optical center and pass through the corresponding positions of the two-dimensional center coordinates. Output all spatial rays corresponding to the same physical landmark point to form an observation ray set.

[0015] Optionally, the step of determining the overall observation quality of the target marker point based on the distribution of the number of rays in each divided directional region, and when the overall observation quality meets preset conditions, performing a comprehensive score on each observation ray, and removing observation rays whose comprehensive quality scores do not meet preset screening conditions, to obtain a filtered ray set, includes: A ray distribution vector is constructed based on the distribution of the number of rays in each directional region. The proportion of the maximum region and the cumulative proportion of the maximum region are calculated. The overall observation quality of the target marker is determined based on the ray distribution vector, the proportion of the maximum region and the cumulative proportion. When the overall observation quality meets the preset conditions, the geometric information contribution item, two-dimensional observation accuracy item, three-dimensional consistency evaluation item and redundancy suppression factor of each observation ray are calculated. Based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item and the redundancy suppression factor, each observation ray is comprehensively scored to obtain a comprehensive quality score. Within each directional region, all observed rays are sorted in descending order according to the comprehensive quality score, and the top K rays with the highest scores are selected as the representative observed rays of the current directional region to obtain the filtered ray set.

[0016] Optionally, the step of constructing a ray distribution vector based on the ray quantity distribution within the divided directional regions, calculating the maximum region proportion and the cumulative proportion of the maximum region, and determining the overall observation quality of the target marker point based on the ray distribution vector, the maximum region proportion, and the cumulative proportion includes: Based on the distribution of ray quantity within each defined directional region, the ray distribution vector is constructed using the following formula:

[0017] in, The ray distribution vector in the directional region. This represents the total number of directional regions. The maximum area proportion is calculated using the following formula:

[0018] in, For the largest regional proportion, This represents the maximum number of rays in all directional regions. Let be the number of rays in the k-th directional region. The total number of rays possessed by the physical markers whose quality is currently being evaluated; The cumulative percentage of the largest region is calculated using the following formula:

[0019] in, This represents the cumulative percentage of the top two regions. The number of rays in the direction region with the highest number of rays. The number of rays in the direction region with the second highest number of rays. The total number of rays possessed by the physical markers whose quality is currently being evaluated; When the number of effective directional areas in the statistics is lower than the preset minimum coverage requirement, or the proportion of the maximum area exceeds the preset dominant threshold, or the cumulative proportion exceeds the preset cumulative threshold, the degradation detection condition is triggered, and the overall observation quality of the target marker is determined to be the first quality level. If the degradation detection condition is not triggered, but the distribution balance index calculated based on the ray distribution vector exceeds the preset balance threshold or the structural consistency index exceeds the preset consistency threshold, the overall observation quality of the target marker is determined to be the second quality level, wherein the second quality level is higher than the first quality level. If the degradation detection condition is not triggered, the distribution balance index does not exceed the preset balance threshold, and the structural consistency index does not exceed the preset consistency threshold, the overall observation quality of the target marker is determined to be the third quality level, wherein the third quality level is higher than the second quality level.

[0020] Optionally, when the overall observation quality meets preset conditions, the geometric information contribution, two-dimensional observation accuracy, three-dimensional consistency evaluation, and redundancy suppression factor of each observation ray are calculated. A comprehensive quality score is obtained by comprehensively evaluating each observation ray based on the geometric information contribution, the two-dimensional observation accuracy, the three-dimensional consistency evaluation, and the redundancy suppression factor, including: The geometric information contribution of each observed ray is calculated using the following formula:

[0021] in, Contributing to geometric information, Let be the angle of incidence of the i-th ray; The two-dimensional observation accuracy term is calculated using the following formula:

[0022] in, For two-dimensional observation accuracy, This represents the standard deviation of the two-dimensional measurement error. The three-dimensional consistency evaluation item is calculated using the following formula:

[0023]

[0024] in, For three-dimensional consistency evaluation items, For orthogonal residuals, The ray residual scale parameter, It is the identity matrix. Let be the unit direction vector of the i-th ray. For vectors transpose, The initial 3D estimated position of the marked point, Let be the position of the camera optical center for the i-th ray; The redundancy suppression factor is calculated using the following formula.

[0025]

[0026] in, As a redundancy suppressor, The minimum directional angle, The preset similarity angle threshold, Let be the unit direction vector of the i-th ray. For vectors transpose, Let be the unit direction vector of the j-th ray within the same directional region; Each observation ray is comprehensively scored based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item, and the redundancy suppression factor, and the comprehensive quality score is obtained by the following formula:

[0027] in, For the overall quality score, The weights for the corresponding geometric information contribution terms, Contributing to geometric information, For the weights corresponding to the two-dimensional observation accuracy term, For two-dimensional observation accuracy, To correspond to the weights of the three-dimensional consistency evaluation items, For three-dimensional consistency evaluation items, The corresponding weights of the redundancy suppression factor It is a redundancy suppression factor.

[0028] Optionally, the step of calculating the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm includes: The ray set is spatially intersected by the least squares spatial forward intersection algorithm to calculate the final three-dimensional coordinates of the target marker point. Simultaneously, based on the ray set and the final three-dimensional coordinates, the quality assessment information of the target marker point is calculated and output.

[0029] Secondly, to achieve the above objectives, the present invention also proposes a marker point error identification and removal device, the marker point error identification and removal device comprising: The region division module is used to obtain the set of observation rays of the target marker point under multiple camera views, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system. The ray optimization module is used to determine the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, the module performs a comprehensive score on each observation ray and removes observation rays whose comprehensive quality scores do not meet the preset screening conditions to obtain a set of filtered rays. The coordinate calculation module is used to calculate the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm.

[0030] Thirdly, to achieve the above objectives, the present invention also proposes a marker error identification and removal device, the marker error identification and removal device comprising: a memory, a processor, and a marker error identification and removal program stored in the memory and executable on the processor, the marker error identification and removal program being configured to implement the steps of the marker error identification and removal method as described above.

[0031] Fourthly, to achieve the above objectives, the present invention also proposes a storage medium storing a marker error identification and removal program, wherein the marker error identification and removal program, when executed by a processor, implements the steps of the marker error identification and removal method described above.

[0032] The marker error identification and elimination method proposed in this invention obtains the set of observation rays of the target marker point from multiple camera perspectives, establishes a local three-dimensional coordinate system with the target marker point as the origin, and divides the set of observation rays into at least two directional regions according to the angle distribution of the observation rays with respect to the principal axis of the local three-dimensional coordinate system; determines the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region; when the overall observation quality meets the preset conditions, performs a comprehensive score on each observation ray, and eliminates observation rays whose comprehensive quality scores do not meet the preset screening conditions, thus obtaining a filtered ray set; The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm. By performing local spatial structuring and directional regional distribution quantification on the observed rays, a deep evaluation of the overall observation geometric quality of the marker point can be achieved, avoiding the problems of insufficient geometric constraints and numerical instability caused by relying solely on the number of rays for screening. Through overall quality judgment and ray comprehensive scoring optimization, abnormal observation rays and highly redundant rays are effectively eliminated, while the most representative observations with the greatest geometric contribution are retained. This significantly improves the accuracy, stability, and robustness of the three-dimensional coordinate reconstruction of the marker point, ensuring that reliable spatial location information can still be obtained under complex observation conditions. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the marker point error identification and removal method of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the marker point error identification and removal method of the present invention; Figure 4 This is a flowchart illustrating the third embodiment of the marker point error identification and removal method of the present invention; Figure 5 This is a flowchart illustrating the fourth embodiment of the marker point error identification and removal method of the present invention; Figure 6 This is a schematic diagram of the main process of the marker point error identification and removal method of the present invention; Figure 7 This is a functional block diagram of the first embodiment of the marker point error identification and elimination device of the present invention.

[0034] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0035] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0036] The solution of this invention mainly involves: acquiring a set of observation rays from multiple camera perspectives of a target marker point; establishing a local three-dimensional coordinate system with the target marker point as the origin; dividing the set of observation rays into at least two directional regions based on the angle distribution of the observation rays relative to the principal axes of the local three-dimensional coordinate system; determining the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region; when the overall observation quality meets preset conditions, comprehensively scoring each observation ray and removing observation rays whose comprehensive quality scores do not meet preset screening conditions to obtain a filtered ray set; calculating the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm; and achieving a deep understanding of the overall observation geometric quality of the marker point by performing local spatial structured division and directional region distribution quantification on the observation rays. This method employs a layered evaluation approach to avoid insufficient geometric constraints and numerical instability caused by relying solely on the number of rays for screening. Through overall quality assessment and comprehensive ray scoring, it effectively eliminates anomalous and highly redundant rays, retaining only the most geometrically representative observations. This significantly improves the accuracy, stability, and robustness of the 3D coordinate reconstruction of marker points, ensuring reliable spatial location information even under complex observation conditions. It addresses the technical problem in existing technologies where, during 3D reconstruction of marker points in multi-view photogrammetry, the assumption that all detected marker points have the same quality or simple screening based solely on the number of visible cameras ignores the spatial distribution differences of observation rays. This leads to insufficient geometric constraints, numerical instability, and excessive rejection, affecting the accuracy and reliability of 3D reconstruction, especially under conditions of highly concentrated observation directions, a few anomalous rays dominating, or poor observation conditions.

[0037] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0038] like Figure 1As shown, the device may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0039] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0040] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating device, a network communication module, a user interface module, and a marker error identification and removal program.

[0041] The device of the present invention calls the marker point error identification and rejection program stored in the memory 1005 through the processor 1001, and performs the following operations: Obtain the set of observation rays of the target marker point from multiple camera perspectives, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system. The overall observation quality of the target marker is determined based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet the preset screening conditions are removed to obtain the filtered ray set. The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm.

[0042] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: In the images acquired from each camera viewpoint, the target marker point is located and the two-dimensional center coordinates corresponding to the target marker point are obtained. The two-dimensional center coordinates are then projected back into three-dimensional space using the camera imaging model to form multiple spatial rays that converge near the target marker point. All spatial rays are then combined to form an observation ray set. Analyze the intersection of each observation ray in the set of observation rays in three-dimensional space, determine the estimated intersection point based on the intersection, and use the estimated intersection point as the initial three-dimensional estimated position of the target marker point; The initial three-dimensional estimated position is set as the origin of the local three-dimensional coordinate system, and the main camera optical axis direction, the average direction of all observed rays, or the principal direction obtained through principal component analysis are selected as the principal axis direction of the local three-dimensional coordinate system. Calculate the angle between each observation ray vector and the principal axis direction, analyze the distribution density of the angles according to the preset angle threshold range, classify observation rays with similar angle characteristics into the same directional region, and divide the set of observation rays into at least two directional regions.

[0043] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: Images containing target markers are acquired from various camera perspectives. The target markers are located using multi-view geometric constraints, and the two-dimensional center coordinates of the target markers are obtained. The fundamental matrix is ​​calculated using the calibrated camera parameters. The two-dimensional center point in the viewpoint is projected onto another viewpoint to form an epipolar line. Candidate points with a distance less than a preset threshold are filtered out, and a third viewpoint is introduced to verify and form an effective three-view matching pair. Using the camera imaging model and projection matrix of each viewpoint, each two-dimensional pixel in the effective three-view matching pair is back-projected into three-dimensional space to form multiple spatial rays that start from the camera optical center and pass through the corresponding positions of the two-dimensional center coordinates. Output all spatial rays corresponding to the same physical landmark point to form an observation ray set.

[0044] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: A ray distribution vector is constructed based on the distribution of the number of rays in each directional region. The proportion of the maximum region and the cumulative proportion of the maximum region are calculated. The overall observation quality of the target marker is determined based on the ray distribution vector, the proportion of the maximum region and the cumulative proportion. When the overall observation quality meets the preset conditions, the geometric information contribution item, two-dimensional observation accuracy item, three-dimensional consistency evaluation item and redundancy suppression factor of each observation ray are calculated. Based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item and the redundancy suppression factor, each observation ray is comprehensively scored to obtain a comprehensive quality score. Within each directional region, all observed rays are sorted in descending order according to the comprehensive quality score, and the top K rays with the highest scores are selected as the representative observed rays of the current directional region to obtain the filtered ray set.

[0045] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: Based on the distribution of ray quantity within each defined directional region, the ray distribution vector is constructed using the following formula:

[0046] in, The ray distribution vector in the directional region. This represents the total number of directional regions. The maximum area proportion is calculated using the following formula:

[0047] in, For the largest regional proportion, This represents the maximum number of rays in all directional regions. Let be the number of rays in the k-th directional region. The total number of rays possessed by the physical markers whose quality is currently being evaluated; The cumulative percentage of the largest region is calculated using the following formula:

[0048] in, This represents the cumulative percentage of the top two regions. The number of rays in the direction region with the highest number of rays. The number of rays in the direction region with the second highest number of rays. The total number of rays possessed by the physical markers whose quality is currently being evaluated; When the number of effective directional areas in the statistics is lower than the preset minimum coverage requirement, or the proportion of the maximum area exceeds the preset dominant threshold, or the cumulative proportion exceeds the preset cumulative threshold, the degradation detection condition is triggered, and the overall observation quality of the target marker is determined to be the first quality level. If the degradation detection condition is not triggered, but the distribution balance index calculated based on the ray distribution vector exceeds the preset balance threshold or the structural consistency index exceeds the preset consistency threshold, the overall observation quality of the target marker is determined to be the second quality level, wherein the second quality level is higher than the first quality level. If the degradation detection condition is not triggered, the distribution balance index does not exceed the preset balance threshold, and the structural consistency index does not exceed the preset consistency threshold, the overall observation quality of the target marker is determined to be the third quality level, wherein the third quality level is higher than the second quality level.

[0049] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: The geometric information contribution of each observed ray is calculated using the following formula:

[0050] in, Contributing to geometric information, Let be the angle of incidence of the i-th ray; The two-dimensional observation accuracy term is calculated using the following formula:

[0051] in, For two-dimensional observation accuracy, This represents the standard deviation of the two-dimensional measurement error. The three-dimensional consistency evaluation item is calculated using the following formula:

[0052]

[0053] in, For three-dimensional consistency evaluation items, For orthogonal residuals, The ray residual scale parameter, It is the identity matrix. Let be the unit direction vector of the i-th ray. For vectors transpose, The initial 3D estimated position of the marked point, Let be the position of the camera optical center for the i-th ray; The redundancy suppression factor is calculated using the following formula.

[0054]

[0055] in, As a redundancy suppressor, The minimum directional angle, The preset similarity angle threshold, Let be the unit direction vector of the i-th ray. For vectors transpose, Let be the unit direction vector of the j-th ray within the same directional region; Each observation ray is comprehensively scored based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item, and the redundancy suppression factor, and the comprehensive quality score is obtained by the following formula:

[0056] in, For the overall quality score, The weights for the corresponding geometric information contribution terms, Contributing to geometric information, For the weights corresponding to the two-dimensional observation accuracy term, For two-dimensional observation accuracy, To correspond to the weights of the three-dimensional consistency evaluation items, For three-dimensional consistency evaluation items, The corresponding weights of the redundancy suppression factor It is a redundancy suppression factor.

[0057] The device of the present invention, through processor 1001 calling the marker point error identification and rejection program stored in memory 1005, also performs the following operations: The ray set is spatially intersected by the least squares spatial forward intersection algorithm to calculate the final three-dimensional coordinates of the target marker point. Simultaneously, based on the ray set and the final three-dimensional coordinates, the quality assessment information of the target marker point is calculated and output.

[0058] This embodiment, through the above-described scheme, obtains a set of observation rays from multiple camera perspectives of the target marker point, establishes a local three-dimensional coordinate system with the target marker point as the origin, and divides the set of observation rays into at least two directional regions based on the angle distribution of the observation rays relative to the principal axes of the local three-dimensional coordinate system. The overall observation quality of the target marker point is determined based on the distribution of the number of rays within each directional region. When the overall observation quality meets preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet preset screening conditions are eliminated, resulting in a filtered ray set. The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm. This approach enables a deep evaluation of the overall geometric quality of the marker point's observations by performing local spatial structured division and directional region distribution quantification, avoiding insufficient geometric constraints and numerical instability caused by relying solely on ray quantity screening. Through overall quality determination and ray comprehensive scoring optimization, abnormal and highly redundant observation rays are effectively eliminated, while the most geometrically representative observations are retained, thereby significantly improving the accuracy, stability, and robustness of the marker point's three-dimensional coordinate reconstruction, ensuring reliable spatial location information can still be obtained under complex observation conditions.

[0059] Based on the above hardware structure, an embodiment of the marker point error identification and elimination method of the present invention is proposed.

[0060] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the marker error identification and removal method of the present invention.

[0061] In the first embodiment, the marker error identification and removal method includes the following steps: Step S10: Obtain the set of observation rays of the target marker point from multiple camera perspectives, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system.

[0062] It should be noted that the observation rays are obtained from all camera views, that is, the set of observation rays of the target marker point from multiple camera views. Then, a local three-dimensional reference frame centered on the target marker point is constructed. By quantifying the angle distribution of each ray relative to the local principal axis, the set of observation rays is divided into at least two directional regions, thereby realizing the spatial structured division of the set of observation rays.

[0063] Step S20: Determine the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, perform a comprehensive score on each observation ray and remove the observation rays whose comprehensive quality scores do not meet the preset screening conditions to obtain the filtered ray set.

[0064] It should be understood that the reliability of the overall observation geometry of the marker points is evaluated based on the distribution of the number of rays within the directional region to avoid invalid calculations for marker points with poor overall quality. Only when the overall quality meets the standard is each observation ray further evaluated for comprehensive quality, and abnormal observations and redundant data are eliminated by removing rays that do not meet the scoring conditions. Finally, a set of filtered rays is obtained, which provides a reliable data foundation for subsequent high-precision 3D reconstruction.

[0065] Step S30: Calculate the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm.

[0066] It is understandable that the optimized ray set after error elimination is used as input data, and the intersection position of multiple spatial rays in three-dimensional space is solved by a preset spatial forward intersection algorithm, thereby calculating the final three-dimensional coordinates of the target marker point and completing the reconstruction process from optimized observation data to high-precision spatial position information.

[0067] This embodiment, through the above-described scheme, obtains a set of observation rays from multiple camera perspectives of the target marker point, establishes a local three-dimensional coordinate system with the target marker point as the origin, and divides the set of observation rays into at least two directional regions based on the angle distribution of the observation rays relative to the principal axes of the local three-dimensional coordinate system. The overall observation quality of the target marker point is determined based on the distribution of the number of rays within each directional region. When the overall observation quality meets preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet preset screening conditions are eliminated, resulting in a filtered ray set. The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm. This approach enables a deep evaluation of the overall geometric quality of the marker point's observations by performing local spatial structured division and directional region distribution quantification, avoiding insufficient geometric constraints and numerical instability caused by relying solely on ray quantity screening. Through overall quality determination and ray comprehensive scoring optimization, abnormal and highly redundant observation rays are effectively eliminated, while the most geometrically representative observations are retained, thereby significantly improving the accuracy, stability, and robustness of the marker point's three-dimensional coordinate reconstruction, ensuring reliable spatial location information can still be obtained under complex observation conditions.

[0068] Furthermore, Figure 3This is a flowchart illustrating the second embodiment of the marker error identification and removal method of the present invention, as shown below. Figure 3 As shown, based on the first embodiment, a second embodiment of the marker point error identification and removal method of the present invention is proposed. In this embodiment, step S10 specifically includes the following steps: Step S11: Locate the target marker point in the images acquired from each camera viewpoint and obtain the two-dimensional center coordinates corresponding to the target marker point. Use the camera imaging model to project the two-dimensional center coordinates back to three-dimensional space to form multiple spatial rays that converge near the target marker point. Combine all spatial rays to form an observation ray set.

[0069] It should be noted that the center coordinates of the marker points in the multi-view 2D images are converted into geometric rays in 3D space through the camera imaging model. Due to the existence of measurement errors, these rays do not strictly intersect at a single point but converge near the target marker point. Finally, the spatial rays generated by all perspectives are integrated into a set of observation rays, which serves as the input for subsequent error identification and removal processing.

[0070] Furthermore, step S11 specifically includes the following steps: Images containing target markers are acquired from various camera perspectives. The target markers are located using multi-view geometric constraints, and the two-dimensional center coordinates of the target markers are obtained. The fundamental matrix is ​​calculated using the calibrated camera parameters. The two-dimensional center point in the viewpoint is projected onto another viewpoint to form an epipolar line. Candidate points with a distance less than a preset threshold are filtered out, and a third viewpoint is introduced to verify and form an effective three-view matching pair. Using the camera imaging model and projection matrix of each viewpoint, each two-dimensional pixel in the effective three-view matching pair is back-projected into three-dimensional space to form multiple spatial rays that start from the camera optical center and pass through the corresponding positions of the two-dimensional center coordinates. Output all spatial rays corresponding to the same physical landmark point to form an observation ray set.

[0071] It should be understood that by using multi-view geometric constraints and epipolar geometric verification mechanisms, the two-dimensional center coordinates used to generate space rays are ensured to have correct matching relationships, thereby constructing a highly reliable initial set of observation rays. First, the fundamental matrix and three-view verification are used to eliminate mismatched points and form effective matching pairs. Then, the verified two-dimensional points are back-projected into space rays. Finally, the rays corresponding to the same physical marker point are integrated into the set of observation rays, providing accurate geometric input data for subsequent error identification and elimination.

[0072] Step S12: Analyze the intersection of each observation ray in the observation ray set in three-dimensional space, determine the estimated intersection point based on the intersection, and use the estimated intersection point as the initial three-dimensional estimated position of the target marker point.

[0073] Understandably, by analyzing the intersection state of each ray in the observed ray set in three-dimensional space, an estimated intersection point is calculated, and this point is used as the initial three-dimensional estimated position of the target marker point, ensuring that the error identification and elimination process has a stable geometric basis.

[0074] Step S13: Set the initial three-dimensional estimated position as the origin of the local three-dimensional coordinate system, and select the main camera optical axis direction, the average direction of all observed rays, or the principal direction obtained through principal component analysis as the principal axis direction of the local three-dimensional coordinate system.

[0075] It should be understood that constructing a local three-dimensional reference frame centered on the target marker point, by setting the initial three-dimensional estimated position as the origin and selecting a specific direction as the principal axis, establishes the spatial position and orientation of the coordinate system; this enables the spatial structured division and directional region definition of the observed ray set.

[0076] Step S14: Calculate the angle between each observation ray vector and the principal axis direction, analyze the distribution density of the angle according to the preset angle threshold range, classify observation rays with similar angle characteristics into the same directional region, and divide the observation ray set into at least two directional regions.

[0077] It is understandable that by calculating the angle between the observed ray and the local principal axis, the spatial pointing characteristics of the ray are quantified, and the angle distribution density is analyzed based on the preset angle threshold range. Rays with similar spatial pointing are aggregated into the same directional region, thereby dividing the originally disordered set of observed rays into at least two directional regions with clear spatial structures.

[0078] This embodiment, through the above scheme, locates the target marker point in the images acquired from each camera's perspective and obtains the corresponding two-dimensional center coordinates of the target marker point. Using the camera imaging model, the two-dimensional center coordinates are projected back into three-dimensional space, forming multiple spatial rays converging near the target marker point. All spatial rays are combined into an observation ray set. The intersection of each observation ray in the observation ray set in three-dimensional space is analyzed, and an estimated intersection point is determined based on the intersection. This estimated intersection point is used as the initial three-dimensional estimated position of the target marker point. The initial three-dimensional estimated position is set as the origin of the local three-dimensional coordinate system. Simultaneously, the optical axis direction of the main camera, the average direction of all observation rays, or the principal direction obtained through principal component analysis are selected as the origin of the local three-dimensional coordinate system. The principal axis direction of the local three-dimensional coordinate system is calculated; the angle between each observation ray vector and the principal axis direction is calculated, and the distribution density of the angle is analyzed according to the preset angle threshold range. Observation rays with similar angle characteristics are classified into the same directional region, and the set of observation rays is divided into at least two directional regions. A local three-dimensional reference frame centered on the target marker point can be constructed through back projection and intersection analysis. Based on the angle distribution characteristics between the rays and the principal axis, disordered observation rays are divided into directional regions with clear spatial structures, thereby realizing the spatial structured expression of observation data. This provides a unified geometric benchmark and quantitative basis for subsequent overall quality judgment of marker points and ray selection based on directional region distribution, ensuring that the quality assessment can accurately reflect the observation geometric constraint characteristics and is not affected by the global coordinate system.

[0079] Furthermore, Figure 4 This is a flowchart illustrating the third embodiment of the marker error identification and removal method of the present invention, as shown below. Figure 4 As shown, based on the first embodiment, a third embodiment of the marker point error identification and removal method of the present invention is proposed. In this embodiment, step S20 specifically includes the following steps: Step S21: Construct a ray distribution vector based on the ray quantity distribution in each directional region, calculate the maximum region proportion and the cumulative proportion of the maximum region, and determine the overall observation quality of the target marker point based on the ray distribution vector, the maximum region proportion and the cumulative proportion.

[0080] It should be noted that constructing ray distribution vectors and calculating key indicators such as the maximum area proportion and cumulative proportion are used to identify whether there is insufficient directional coverage or degradation dominated by a few directions. This allows for a stratified determination of the overall reliability of the marker points before entering individual ray scoring, avoiding invalid calculations for marker points with poor geometric structures.

[0081] Furthermore, step S21 specifically includes the following steps: Based on the distribution of ray quantity within each defined directional region, the ray distribution vector is constructed using the following formula:

[0082] in, The ray distribution vector in the directional region. This represents the total number of directional regions. The maximum area proportion is calculated using the following formula:

[0083] in, For the largest regional proportion, This represents the maximum number of rays in all directional regions. Let be the number of rays in the k-th directional region. The total number of rays possessed by the physical markers whose quality is currently being evaluated; The cumulative percentage of the largest region is calculated using the following formula:

[0084] in, This represents the cumulative percentage of the top two regions. The number of rays in the direction region with the highest number of rays. The number of rays in the direction region with the second highest number of rays. The total number of rays possessed by the physical markers whose quality is currently being evaluated; When the number of effective directional areas in the statistics is lower than the preset minimum coverage requirement, or the proportion of the maximum area exceeds the preset dominant threshold, or the cumulative proportion exceeds the preset cumulative threshold, the degradation detection condition is triggered, and the overall observation quality of the target marker is determined to be the first quality level. If the degradation detection condition is not triggered, but the distribution balance index calculated based on the ray distribution vector exceeds the preset balance threshold or the structural consistency index exceeds the preset consistency threshold, the overall observation quality of the target marker is determined to be the second quality level, wherein the second quality level is higher than the first quality level. If the degradation detection condition is not triggered, the distribution balance index does not exceed the preset balance threshold, and the structural consistency index does not exceed the preset consistency threshold, the overall observation quality of the target marker is determined to be the third quality level, wherein the third quality level is higher than the second quality level.

[0085] It should be understood that by constructing ray distribution vectors and calculating the maximum area proportion and cumulative proportion, the distribution structure of observed rays in directional regions is quantified. Then, a progressive judgment strategy is adopted to divide the overall observation quality of marker points into three levels. This can prioritize the identification of degradation cases with insufficient directional coverage or a few directions dominating, and conduct hierarchical evaluation of distribution balance and structural consistency. This allows for rapid screening of the overall reliability of marker points before entering individual ray scoring, ensuring that only marker points with qualified geometric structures undergo subsequent fine processing.

[0086] Step S22: When the overall observation quality meets the preset conditions, calculate the geometric information contribution item, two-dimensional observation accuracy item, three-dimensional consistency evaluation item, and redundancy suppression factor for each observation ray. Based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item, and the redundancy suppression factor, a comprehensive score is obtained for each observation ray to obtain a comprehensive quality score.

[0087] Understandably, provided that the overall observation quality of the marker points meets the standards, each individual observation ray is evaluated in detail and quantitatively from four dimensions: geometric information, two-dimensional accuracy, three-dimensional consistency, and directional redundancy. A comprehensive quality score is obtained through the fusion calculation of multi-dimensional indicators. This achieves hierarchical screening from the whole to the individual, distinguishing the contribution value of different rays under the same marker point.

[0088] Furthermore, step S22 specifically includes the following steps: When the overall observation quality meets the preset conditions, the geometric information contribution, two-dimensional observation accuracy, three-dimensional consistency evaluation, and redundancy suppression factor of each observation ray are calculated. A comprehensive score is then obtained for each observation ray based on the geometric information contribution, the two-dimensional observation accuracy, the three-dimensional consistency evaluation, and the redundancy suppression factor, including: The geometric information contribution of each observed ray is calculated using the following formula:

[0089] in, Contributing to geometric information, Let be the angle of incidence of the i-th ray; The two-dimensional observation accuracy term is calculated using the following formula:

[0090] in, For two-dimensional observation accuracy, This represents the standard deviation of the two-dimensional measurement error. The three-dimensional consistency evaluation item is calculated using the following formula:

[0091]

[0092] in, For three-dimensional consistency evaluation items, For orthogonal residuals, The ray residual scale parameter, It is the identity matrix. Let be the unit direction vector of the i-th ray. For vectors transpose, The initial 3D estimated position of the marked point, Let be the position of the camera optical center for the i-th ray; The redundancy suppression factor is calculated using the following formula.

[0093]

[0094] in, As a redundancy suppressor, The minimum directional angle, The preset similarity angle threshold, Let be the unit direction vector of the i-th ray. For vectors transpose, Let be the unit direction vector of the j-th ray within the same directional region; Each observation ray is comprehensively scored based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item, and the redundancy suppression factor, and the comprehensive quality score is obtained by the following formula:

[0095] in, For the overall quality score, The weights for the corresponding geometric information contribution terms, Contributing to geometric information, For the weights corresponding to the two-dimensional observation accuracy term, For two-dimensional observation accuracy, To correspond to the weights of the three-dimensional consistency evaluation items, For three-dimensional consistency evaluation items, The corresponding weights of the redundancy suppression factor It is a redundancy suppression factor.

[0096] It should be understood that by constructing ray distribution vectors and calculating the maximum area proportion and cumulative proportion, the distribution structure of observed rays in directional regions is quantified. Then, a progressive judgment strategy is adopted to divide the overall observation quality of marker points into three levels. This can prioritize the identification of degradation cases with insufficient directional coverage or a few directions dominating, and conduct hierarchical evaluation of distribution balance and structural consistency. This allows for rapid screening of the overall reliability of marker points before entering individual ray scoring, ensuring that only marker points with qualified geometric structures undergo subsequent fine processing.

[0097] Step S23: Within each directional region, sort all observed rays in descending order according to the comprehensive quality score, select the top K rays with the highest scores as the representative observed rays of the current directional region, and obtain the filtered ray set.

[0098] It should be understood that, within each directional region, the observed rays are selected based on the comprehensive quality score. The top K rays with the highest scores are selected in descending order as representative observations for that region, thus obtaining the selected ray set. While preserving the diversity of directional coverage, low-quality and highly redundant rays are eliminated to ensure that the observation data used for the final 3D reconstruction is both high-quality and representative.

[0099] This embodiment, through the above scheme, constructs a ray distribution vector based on the ray quantity distribution within the divided directional regions, calculates the maximum region proportion and the cumulative proportion of the maximum region, and determines the overall observation quality of the target marker point based on the ray distribution vector, the maximum region proportion, and the cumulative proportion. When the overall observation quality meets preset conditions, it calculates the geometric information contribution, two-dimensional observation accuracy, three-dimensional consistency evaluation, and redundancy suppression factor for each observation ray, and comprehensively scores each observation ray based on the geometric information contribution, the two-dimensional observation accuracy, the three-dimensional consistency evaluation, and the redundancy suppression factor to obtain a comprehensive quality score. Within each directional region, all observed rays are sorted in descending order according to the comprehensive quality score. The top K rays with the highest scores are selected as the representative observed rays of the current directional region, resulting in a filtered ray set. This allows for rapid hierarchical determination of the overall observation quality of marker points by constructing ray distribution vectors and calculating the region proportions, avoiding invalid calculations for geometrically degraded data. Furthermore, by combining multi-dimensional comprehensive scores of geometry, accuracy, consistency, and redundancy with optimal selection within directional regions, abnormal observations and highly redundant rays are effectively eliminated. This maximizes the quality of observation data while ensuring the diversity of directional coverage, thereby significantly improving the accuracy, stability, and robustness of the 3D reconstruction of marker points.

[0100] Furthermore, Figure 5 This is a flowchart illustrating the fourth embodiment of the marker error identification and removal method of the present invention, as shown below. Figure 5As shown, based on the first embodiment, a fourth embodiment of the marker point error identification and removal method of the present invention is proposed. In this embodiment, step S30 specifically includes the following steps: Step S31: Perform spatial intersection calculation on the ray set using the least squares spatial forward intersection algorithm to calculate the final three-dimensional coordinates of the target marker point.

[0101] It should be noted that by using the ray set after error identification and elimination as input data, the least squares spatial forward intersection algorithm is used to solve for the optimal intersection position of multiple spatial rays in three-dimensional space, thereby calculating the final three-dimensional coordinates of the target marker point. This can complete the reconstruction process from optimized observation data to high-precision spatial position information. The least squares optimization mechanism ensures that the final output three-dimensional coordinates have the highest geometric consistency and measurement accuracy, achieving highly reliable three-dimensional reconstruction of the marker point.

[0102] Step S32: Simultaneously, based on the ray set and the final three-dimensional coordinates, calculate and output the quality assessment information of the target marker point.

[0103] Understandably, after completing the 3D coordinate reconstruction, various quantitative indicators are calculated and output based on the selected ray set and the final 3D coordinates as quality assessment information for the marker point reconstruction results. This can provide reliability and accuracy evaluation for the reconstructed 3D data, enabling downstream applications to judge the credibility of the data based on information such as the number of directions, the number of rays, reconstruction error, and back projection error, thereby achieving closed-loop monitoring of the reconstruction results and subsequent analysis applications.

[0104] In the specific implementation, see Figure 6 , Figure 6 This is a schematic diagram of the main process of the marker error identification and removal method of the present invention, as shown below. Figure 6 As shown, the main processing flow is divided into preprocessing, marker matching, marker error identification and removal, and marker reconstruction modules. The specific functions and processing flow of each module are as follows: 1. Preprocessing module This module receives raw images from multiple camera viewpoints, in which the elliptical outlines of suspected landmarks have been preliminarily extracted using an ellipse detection algorithm.

[0105] The module filters the elliptical data based on preset physical prior knowledge, removing noisy data that clearly does not conform to the characteristics of the marker points. The specific filtering conditions are as follows: Ellipse area threshold: retain ellipses with a major axis length greater than the threshold to exclude excessively small noise.

[0106] Major-minor axis ratio threshold: retain ellipses with a major-minor axis ratio close to the threshold to exclude excessively distorted ellipses.

[0107] Contour fitting error threshold: Ellipses with fitting errors less than the threshold are retained to ensure contour quality.

[0108] Ellipses that meet all the conditions will be retained, and their center pixel coordinates, ellipse parameters, and other information will be output as valid data to the next module.

[0109] 2. Marker point matching module 2.1 Pole line matching: a. For any two viewpoints (such as viewpoint A and viewpoint B), calculate the fundamental matrix F using the calibrated camera parameters (intrinsic and extrinsic parameters).

[0110] b. The center point p of an ellipse in viewpoint A a According to the principle of epipolar geometry, when projected onto the image plane at viewpoint B, an epipolar line l is formed. b .

[0111] c. In viewpoint B, find all lines leading to that pair of polar lines. b Ellipse center points whose distance is less than a preset threshold are selected as candidate matching points.

[0112] d. To further verify, a third perspective C is introduced to view p. a Its best candidate matching point p in viewpoint B b Projection matching is performed with viewpoint C respectively.

[0113] If p a and p b A consistent corresponding point p can be found in viewpoint C. c Then it is considered that (p a , p b , p c This constitutes a valid three-view matching pair.

[0114] 2.2 Ray generation: a. For each confirmed set of matching points (e.g., {p a ,p b ,p c Using the camera projection matrix of each viewpoint, each two-dimensional pixel is projected backward into three-dimensional space, forming a three-dimensional ray that originates from the optical center of the camera and passes through that pixel.

[0115] b. Set of all rays corresponding to the same physical landmark point {Ray i The output, as the initial observation data for the marked point, is sent to the error identification and removal module.

[0116] 3. Marker error identification and rejection module 3.1 Marker Quality Judgment Model In multi-view photogrammetry, the same marker point is usually observed from multiple viewpoints, generating multiple spatial rays. However, due to factors such as shooting posture distribution, occlusion, reflection, and imaging noise, the observation conditions between different marker points vary significantly.

[0117] In existing technologies, it is typically assumed that all detected markers have the same quality, or that markers are simply filtered based on the number of visible cameras. This approach is prone to problems in the following situations: 1. The observation directions of the marker points are highly concentrated, resulting in insufficient spatial geometric constraints; 2. A few anomalous rays dominate the three-dimensional estimation results; 3. Continuing to finely screen markers with poor observation conditions can lead to excessive rejection or numerical instability.

[0118] Therefore, it is necessary to assess the overall observation quality of the marker points before proceeding to a detailed quality assessment at the ray level. The quality of a marker point is essentially determined by the spatial distribution structure of its observed rays, not just the number of rays.

[0119] Specifically, a high-quality marker should meet the following conditions simultaneously: 1. Sufficient directional coverage: The rays are distributed across multiple directional regions, avoiding concentration in a single direction; 2. The number of rays is moderate: it has sufficient redundancy without being overly concentrated; 3. Relatively balanced across regions: There is no situation in a single region where rays occupy an absolute majority.

[0120] Based on the above understanding, this invention quantitatively evaluates the overall observation quality of the marker points from three aspects: directional area coverage, total amount of rays, and regional distribution balance.

[0121] Step 1: Constructing the Local Coordinate System Step 1.1 Based on the set of multiple view rays corresponding to the marker point, and combined with the spatial estimation results of the marker point, determine the reference position and reference direction of the marker point, and establish a local three-dimensional coordinate system accordingly, where the reference position is used as the origin of the coordinate system and the reference direction is used as the principal axis direction of the local coordinate system.

[0122] Step 1.2 In the local coordinate system, calculate the angle between each ray and the principal axis direction, and based on the statistical distribution of the angles of all rays, adaptively determine the boundary conditions for distinguishing between "near-principal axis rays" and "off-principal axis rays". Under the premise of satisfying the separation of ray directions, make the boundary point so that the number of the two groups of rays is relatively balanced, thereby dividing the rays into at least two types of directional regions.

[0123] Step 1.3 For rays that deviate from the principal axis, project their direction onto the local reference plane to obtain the planar direction feature quantity; then, based on the relative closeness between the ray direction feature quantities, merge rays with similar direction features and assign corresponding direction region numbers.

[0124] Step 1.4 Output the ray partitioning results so that each ray uniquely belongs to a directional region. The directional region serves as the input for subsequent marker point quality judgment, ray quality scoring, and optimization.

[0125] Step 2 Overall Quality Assessment Step 2.1 Based on the directional region division results obtained in Step 1, count the number of rays in each directional region and construct the directional region ray distribution vector:

[0126] in, The ray distribution vector in the directional region. This represents the total number of directional regions. Step 2.2 Calculate the effective directional coverage and count the number of effective directional regions N of the marked points. valid It is then compared with a preset threshold to determine whether the ray forms a valid observation in multiple directions. If N valid If the coverage is below the minimum requirement, it is considered that the coverage in the ray direction is insufficient and there is a significant risk of geometric degradation.

[0127] Step 2.3 Detect directional dominance to distinguish between two types of degradation: "single-directional dominance" and "minority-directional dominance," and calculate the maximum region proportion:

[0128] in, For the largest regional proportion, This represents the maximum number of rays in all directional regions. Let be the number of rays in the k-th directional region. The total number of rays possessed by the physical markers whose quality is currently being evaluated; And further calculate the cumulative proportion of the top two regions.

[0129] in, This represents the cumulative percentage of the top two regions. The number of rays in the direction region with the highest number of rays. The number of rays in the direction region with the second highest number of rays. The total number of rays possessed by the physical markers whose quality is currently being evaluated; Step 2.4 Determine the redundancy of the number of rays and the structural density, and count the total number of rays (Nray) of the marker point to determine whether the marker point has the minimum number of observations required for subsequent screening and three-dimensional estimation.

[0130] And combine directional coverage to make a joint judgment: If N ray Smaller and N valid If the value is low, it is considered that the observation redundancy is insufficient; If N ray Larger but r max or r top2 If the value is too high, it is considered that the redundancy is mainly concentrated in local directions and the information correlation is relatively strong.

[0131] Step 2.5 Analyze the regional distribution uniformity and dispersion, and calculate the mean and dispersion of the number of rays in the directional region.

[0132] in, The dispersion of the number of rays in the directional region, This represents the total number of directional regions. Let be the number of rays in the k-th directional region. This represents the average number of rays in the directional region. The index of the directional region; Larger This indicates that the distribution of rays is uneven across directional regions. Furthermore, to avoid scale issues caused by these differences, a normalized dispersion is introduced:

[0133] in, This represents the average number of rays in the directional region. This represents the total number of directional regions. The total number of rays possessed by the physical markers whose quality is currently being evaluated; Used for unified evaluation under different directional regions.

[0134] Step 2.6 Local structural consistency judgment: In order to further analyze whether there is abnormal concentration or breakage in the ray distribution structure, a regional continuity judgment is introduced.

[0135] Analyze the difference in the number of rays between adjacent regions according to the direction region numbering order:

[0136] in, This represents the difference in the number of rays between adjacent regions. Let be the number of rays in the k-th directional region. This represents the number of rays in the (k+1)th directional region.

[0137] If multiple continuous regions satisfy the condition of being significantly larger, then the ray distribution is considered to have a significant break or discontinuity in the directional space.

[0138] Step 2.7 Multi-layer joint quality judgment logic, combining the judgment results from Step 2.2 to Step 2.5, adopts a progressive judgment strategy: If the degradation detection conditions (insufficient directional coverage, single area or a few areas dominating) are considered, the overall quality of the marker points is judged to be low. If the directional degradation detection condition is not triggered, but the distribution balance or structural consistency index does not meet the requirements, the overall quality of the marker point is judged to be medium. If all requirements are met, the overall quality of the marker point is judged to be high, and it is allowed to enter the ray quality scoring and optimization process.

[0139] 3.2 X-ray quality scoring and optimization model For the same marker point, the contribution of different rays to 3D reconstruction is mainly affected by the following factors: the relative incident geometry between the ray and the local plane of the marker point, the measurement accuracy of the ray in the 2D image, whether the ray is consistent with the overall 3D estimation, and whether there is height and directional redundancy among multiple rays. Based on the above analysis, each ray is scored, and representative selection is performed within the directional region. The specific process is as follows: Step 1: Quantification of Incident Geometric Conditions: Let the i-th ray originate from the camera center Cam. i and unit direction vector d i describe.

[0140] Based on the normal vector n of the marked point obtained in the previous section, we first calculate the incident angle of the ray. A geometric information contribution term for the ray is constructed to characterize the effective spatial constraint strength provided by the ray under the current observation geometry.

[0141]

[0142] in, Contributing to geometric information, Let be the angle of incidence of the i-th ray; This form is used to suppress rays that are approximately normally or approximately tangentially incident, giving higher weight to rays with moderate incident angles. Step 2: For each ray, the two-dimensional localization or ellipse fitting process of the corresponding marked points in the image will introduce a certain measurement error. Let the variance of this error be... Define the two-dimensional observation accuracy term as:

[0143] in, For two-dimensional observation accuracy, This represents the standard deviation of the two-dimensional measurement error. Step 3: Based on the initial 3D estimated position X0, calculate the orthogonal residual from the ray to that point: Construct three-dimensional consistency evaluation items: in, For three-dimensional consistency evaluation items, For orthogonal residuals, The ray residual scale parameter, It is the identity matrix. Let be the unit direction vector of the i-th ray. For vectors transpose, The initial 3D estimated position of the marked point, Let be the position of the camera optical center for the i-th ray; This is used to determine whether the ray is consistent with the overall spatial position estimate of the current marker point, thereby reducing the interference of anomalous observations on subsequent optimization. Step 4 involves explicit quantification of ray direction redundancy. Within the same directional region, calculate the minimum directional angle between ray i and other rays:

[0144] And construct a redundancy suppression factor

[0145] in, As a redundancy suppressor, The minimum directional angle, The preset similarity angle threshold, Let be the unit direction vector of the i-th ray. For vectors transpose, Let be the unit direction vector of the j-th ray within the same directional region; This redundancy suppression factor is used to reduce the weight of rays with highly similar orientations in the overall score, so as to avoid the adverse effects of orientation redundancy on subsequent optimization.

[0146] Step 5: The overall quality score of a ray is defined by considering its geometric incidence conditions, two-dimensional observation accuracy, three-dimensional consistency, and directional redundancy, and assigning corresponding weights ω.

[0147] in, For the overall quality score, The weights for the corresponding geometric information contribution terms, Contributing to geometric information, For the weights corresponding to the two-dimensional observation accuracy term, For two-dimensional observation accuracy, To correspond to the weights of the three-dimensional consistency evaluation items, For three-dimensional consistency evaluation items, The corresponding weights of the redundancy suppression factor It is a redundancy suppression factor.

[0148] Within each directional region, all rays are sorted in descending order of Si, and the top K rays with the highest scores are selected as representative observations for that region and added to the ray set {Ray}. 4. Marker point reconstruction module This module receives the filtered ray set {Ray} and calculates the final three-dimensional coordinates P3d(x,y,z) of the marked points using the least squares spatial forward intersection algorithm.

[0149] Simultaneously, the quality assessment information of this marker point is calculated and output for subsequent analysis and application: Number of directions: The number of directional regions covered by the optimized ray set.

[0150] Ray count: The total number of rays after optimization.

[0151] Reconstruction error: The residual in the forward intersection calculation, reflecting the consistency of the ray intersection.

[0152] Back projection error: The reconstructed 3D point P3d is reprojected back to each original image, the average pixel distance from the center point of the original ellipse is calculated, and the 3D accuracy is comprehensively evaluated.

[0153] It should be noted that this embodiment has the following technical effects: 1. Effectively reduces redundant observations, improving computational efficiency and stability. By partitioning rays according to spatial direction and eliminating similar rays, redundant observation data in the same direction is significantly reduced. In subsequent adjustment optimization or pose solving, the size and condition number of the normal equation matrix can be reduced, improving the solution speed and numerical stability, which is especially suitable for large-scale marker point scenarios.

[0154] 2. Improve the accuracy of 3D point reconstruction and positioning. Preferring rays with an incident angle close to 45° helps balance the intensity of the observation geometry. Such rays typically have better error propagation characteristics in depth estimation, reducing reconstruction sensitivity issues caused by overly parallel or perpendicular observation angles, thereby improving the estimation accuracy of 3D point coordinates and camera attitude.

[0155] 3. Enhance the robustness of the algorithm to noise and anomaly observations. Retaining representative rays by direction can, to some extent, suppress the impact of abnormal observations caused by poor imaging quality, occlusion, or mismatch from individual perspectives; the retained observation data after screening is of higher quality and more reasonably distributed, which is conducive to improving the reliability and repeatability of the system in real complex environments.

[0156] 3. Adaptively handle different observation configurations to maintain the algorithm's generalization ability. By setting thresholds for the number of directions and rays, the algorithm can dynamically adjust its selection strategy based on the actual observations of 3D points. When observations are sufficient, it performs optimal selection to improve quality; when observations are insufficient, it retains all data to ensure solution conditions, adapting to different shooting layouts and variations in marker visibility.

[0157] 4. Improve directional coverage uniformity in multi-view geometry The partitioning retention mechanism ensures that the filtered rays are representative observations in different directions in space, avoiding observations concentrated in a narrow angular range; this helps to establish more uniform multi-view geometric constraints and improve the overall consistency between 3D reconstruction and camera calibration.

[0158] This embodiment transforms the abstract error problem into a quantitative analysis of the directional distribution of observed rays by establishing a local coordinate system for each marker point and dividing its entire space into multiple fine-dimensional directional regions. By synergistically considering multiple dimensions such as the uniformity of ray directional distribution, back-projection error, consistency of angles between rays, and contour fitting accuracy, it can accurately identify and eliminate various explicit and implicit error points caused by matching errors, occlusion, poor imaging, etc. This technical process constitutes a complete closed-loop system from data depth preprocessing to geometric modeling and multi-attribute intelligent decision-making, significantly improving the accuracy, robustness, and automation level of error identification, and providing a highly pure data foundation for core 3D reconstruction.

[0159] This embodiment, through the above-described scheme, uses the least squares spatial forward intersection algorithm to perform spatial intersection calculation on the ray set, calculating the final three-dimensional coordinates of the target marker point; simultaneously, based on the ray set and the final three-dimensional coordinates, it calculates and outputs the quality assessment information of the target marker point; it can achieve optimal intersection calculation of multiple spatial rays through the least squares spatial forward intersection algorithm, obtaining high-precision final three-dimensional coordinates of the target marker point, ensuring the geometric consistency of the reconstruction results; at the same time, by calculating and outputting quality assessment information, it provides reliability and accuracy evaluation for the reconstructed three-dimensional data, enabling downstream applications to judge the credibility of the data based on the assessment information, thereby realizing closed-loop monitoring of the reconstruction results and subsequent analysis applications.

[0160] Accordingly, the present invention further provides a device for identifying and eliminating marker point errors.

[0161] Reference Figure 7 , Figure 7 This is a functional block diagram of the first embodiment of the marker point error identification and elimination device of the present invention.

[0162] In a first embodiment of the marker error identification and rejection device of the present invention, the marker error identification and rejection device includes: The region division module 10 is used to obtain the set of observation rays of the target marker point under multiple camera views, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system.

[0163] The ray optimization module 20 is used to determine the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, it performs a comprehensive score on each observation ray and removes the observation rays whose comprehensive quality scores do not meet the preset screening conditions to obtain the filtered ray set.

[0164] The coordinate calculation module 30 is used to calculate the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm.

[0165] The steps for implementing each functional module of the marker error identification and removal device can be referred to in the various embodiments of the marker error identification and removal method of the present invention, and will not be repeated here.

[0166] Furthermore, this embodiment of the invention also proposes a storage medium storing a marker error identification and removal program, wherein when the marker error identification and removal program is executed by a processor, the operation described in the above-described marker error identification and removal method embodiment is implemented.

[0167] Those skilled in the art will understand that all or part of the steps in the methods described above can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium is a computer-readable storage medium, including: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program code.

[0168] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0169] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0170] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for identifying and eliminating marker point errors, characterized in that, The method for identifying and eliminating marker point errors includes: Obtain the set of observation rays of the target marker point from multiple camera perspectives, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system. The overall observation quality of the target marker is determined based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet the preset screening conditions are removed to obtain the filtered ray set. The three-dimensional coordinates of the target marker point are calculated based on the ray set using a preset spatial forward intersection algorithm.

2. The marker point error identification and removal method as described in claim 1, characterized in that, The process involves acquiring a set of observation rays from multiple camera perspectives of the target marker point, establishing a local three-dimensional coordinate system with the target marker point as the origin, and dividing the set of observation rays into at least two directional regions based on the angle distribution of the observation rays relative to the principal axes of the local three-dimensional coordinate system, including: In the images acquired from each camera viewpoint, the target marker point is located and the two-dimensional center coordinates corresponding to the target marker point are obtained. The two-dimensional center coordinates are then projected back into three-dimensional space using the camera imaging model to form multiple spatial rays that converge near the target marker point. All spatial rays are then combined to form an observation ray set. Analyze the intersection of each observation ray in the set of observation rays in three-dimensional space, determine the estimated intersection point based on the intersection, and use the estimated intersection point as the initial three-dimensional estimated position of the target marker point; The initial three-dimensional estimated position is set as the origin of the local three-dimensional coordinate system, and the main camera optical axis direction, the average direction of all observed rays, or the principal direction obtained through principal component analysis are selected as the principal axis direction of the local three-dimensional coordinate system. Calculate the angle between each observation ray vector and the principal axis direction, analyze the distribution density of the angles according to the preset angle threshold range, classify observation rays with similar angle characteristics into the same directional region, and divide the set of observation rays into at least two directional regions.

3. The marker point error identification and removal method as described in claim 2, characterized in that, The process involves locating target markers in images acquired from various camera viewpoints and obtaining their corresponding two-dimensional center coordinates. These coordinates are then projected back into three-dimensional space using a camera imaging model, forming multiple spatial rays converging near the target marker. All these spatial rays are then combined to form an observation ray set, including: Images containing target markers are acquired from various camera perspectives. The target markers are located using multi-view geometric constraints, and the two-dimensional center coordinates of the target markers are obtained. The fundamental matrix is ​​calculated using the calibrated camera parameters. The two-dimensional center point in the viewpoint is projected onto another viewpoint to form an epipolar line. Candidate points with a distance less than a preset threshold are filtered out, and a third viewpoint is introduced to verify and form an effective three-view matching pair. Using the camera imaging model and projection matrix of each viewpoint, each two-dimensional pixel in the effective three-view matching pair is back-projected into three-dimensional space to form multiple spatial rays that start from the camera optical center and pass through the corresponding positions of the two-dimensional center coordinates. Output all spatial rays corresponding to the same physical landmark point to form an observation ray set.

4. The marker point error identification and removal method as described in claim 1, characterized in that, The overall observation quality of the target marker point is determined based on the distribution of the number of rays within the divided directional regions. When the overall observation quality meets preset conditions, each observation ray is comprehensively scored, and observation rays whose comprehensive quality scores do not meet preset screening conditions are removed, resulting in a filtered ray set, including: A ray distribution vector is constructed based on the distribution of the number of rays in each directional region. The proportion of the maximum region and the cumulative proportion of the maximum region are calculated. The overall observation quality of the target marker is determined based on the ray distribution vector, the proportion of the maximum region and the cumulative proportion. When the overall observation quality meets the preset conditions, the geometric information contribution item, two-dimensional observation accuracy item, three-dimensional consistency evaluation item and redundancy suppression factor of each observation ray are calculated. Based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item and the redundancy suppression factor, each observation ray is comprehensively scored to obtain a comprehensive quality score. Within each directional region, all observed rays are sorted in descending order according to the comprehensive quality score, and the top K rays with the highest scores are selected as the representative observed rays of the current directional region to obtain the filtered ray set.

5. The marker point error identification and removal method as described in claim 4, characterized in that, The process of constructing a ray distribution vector based on the ray quantity distribution within each divided directional region, calculating the maximum region proportion and the cumulative proportion of the maximum region, and determining the overall observation quality of the target marker point based on the ray distribution vector, the maximum region proportion, and the cumulative proportion includes: Based on the distribution of ray quantity within each defined directional region, the ray distribution vector is constructed using the following formula: in, The ray distribution vector in the directional region. This represents the total number of directional regions. The maximum area proportion is calculated using the following formula: in, For the largest regional proportion, This represents the maximum number of rays in all directional regions. Let be the number of rays in the k-th directional region. The total number of rays possessed by the physical markers whose quality is currently being evaluated; The cumulative percentage of the largest region is calculated using the following formula: in, This represents the cumulative percentage of the top two regions. The number of rays in the direction region with the highest number of rays. The number of rays in the direction region with the second highest number of rays. The total number of rays possessed by the physical markers whose quality is currently being evaluated; When the number of effective directional areas in the statistics is lower than the preset minimum coverage requirement, or the proportion of the maximum area exceeds the preset dominant threshold, or the cumulative proportion exceeds the preset cumulative threshold, the degradation detection condition is triggered, and the overall observation quality of the target marker is determined to be the first quality level. If the degradation detection condition is not triggered, but the distribution balance index calculated based on the ray distribution vector exceeds the preset balance threshold or the structural consistency index exceeds the preset consistency threshold, the overall observation quality of the target marker is determined to be the second quality level, wherein the second quality level is higher than the first quality level. If the degradation detection condition is not triggered, the distribution balance index does not exceed the preset balance threshold, and the structural consistency index does not exceed the preset consistency threshold, the overall observation quality of the target marker is determined to be the third quality level, wherein the third quality level is higher than the second quality level.

6. The marker point error identification and removal method as described in claim 4, characterized in that, When the overall observation quality meets the preset conditions, the geometric information contribution, two-dimensional observation accuracy, three-dimensional consistency evaluation, and redundancy suppression factor of each observation ray are calculated. A comprehensive score is then obtained for each observation ray based on the geometric information contribution, the two-dimensional observation accuracy, the three-dimensional consistency evaluation, and the redundancy suppression factor, including: The geometric information contribution of each observed ray is calculated using the following formula: in, Contributing to geometric information, Let be the angle of incidence of the i-th ray; The two-dimensional observation accuracy term is calculated using the following formula: in, For two-dimensional observation accuracy, This represents the standard deviation of the two-dimensional measurement error. The three-dimensional consistency evaluation item is calculated using the following formula: in, For three-dimensional consistency evaluation items, For orthogonal residuals, The ray residual scale parameter, It is the identity matrix. Let be the unit direction vector of the i-th ray. For vectors transpose, The initial 3D estimated position of the marked point. Let be the position of the camera optical center for the i-th ray; The redundancy suppression factor is calculated using the following formula. in, As a redundancy suppressor, The minimum directional angle, The preset similarity angle threshold, Let be the unit direction vector of the i-th ray. For vectors transpose, Let be the unit direction vector of the j-th ray within the same directional region; Each observation ray is comprehensively scored based on the geometric information contribution item, the two-dimensional observation accuracy item, the three-dimensional consistency evaluation item, and the redundancy suppression factor, and the comprehensive quality score is obtained by the following formula: in, For the overall quality score, The weights for the corresponding geometric information contribution terms, Contributing to geometric information, For the weights corresponding to the two-dimensional observation accuracy term, For two-dimensional observation accuracy, To correspond to the weights of the three-dimensional consistency evaluation items, For three-dimensional consistency evaluation items, The corresponding weights of the redundancy suppression factor It is a redundancy suppression factor.

7. The marker point error identification and removal method as described in claim 1, characterized in that, The step of calculating the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm includes: The ray set is spatially intersected by the least squares spatial forward intersection algorithm to calculate the final three-dimensional coordinates of the target marker point. Simultaneously, based on the ray set and the final three-dimensional coordinates, the quality assessment information of the target marker point is calculated and output.

8. A device for identifying and eliminating marker point errors, characterized in that, The marker point error identification and elimination device includes: The region division module is used to obtain the set of observation rays of the target marker point under multiple camera views, establish a local three-dimensional coordinate system with the target marker point as the origin, and divide the set of observation rays into at least two directional regions according to the angle distribution of the observation rays relative to the principal axis of the local three-dimensional coordinate system. The ray optimization module is used to determine the overall observation quality of the target marker point based on the distribution of the number of rays in each directional region. When the overall observation quality meets the preset conditions, the module performs a comprehensive score on each observation ray and removes observation rays whose comprehensive quality scores do not meet the preset screening conditions to obtain a set of filtered rays. The coordinate calculation module is used to calculate the three-dimensional coordinates of the target marker point based on the ray set using a preset spatial forward intersection algorithm.

9. A marker point error identification and rejection device, characterized in that, The marker error identification and rejection device includes: a memory, a processor, and a marker error identification and rejection program stored in the memory and executable on the processor, wherein the marker error identification and rejection program is configured to implement the steps of the marker error identification and rejection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a marker error identification and removal program, which, when executed by a processor, implements the steps of the marker error identification and removal method as described in any one of claims 1 to 7.

Images