Target point positioning method and device and scanning equipment

Abstract

The invention discloses a target point positioning method and device and scanning equipment, and belongs to the field of three-dimensional scanning. The method comprises the following steps: acquiring a target point set; the target point set comprises target points measured in a plurality of observation frames and coordinate information, and the coordinate information comprises global coordinates and image coordinates in the binocular camera; determining an error function based on the image coordinate of the target point in the binocular camera, and optimizing the error function to update the global coordinate of the target point; identifying the same target points in the target point set based on the updated global coordinates, and performing duplicate removal processing on the target point set based on the same target points to obtain a new target point set; and returning to the first step to continue the execution until an iteration termination condition is met, and obtaining a final target point set. According to the method, binocular measurement data fusion is combined, the three-dimensional coordinates of the target points are iteratively optimized, and repeated target points are eliminated, so that the measurement precision is improved, and the layering phenomenon is reduced.

Description

Technical Field

[0001] This application belongs to the field of 3D scanning, and particularly relates to a method, apparatus, and scanning device for target point positioning. Background Art

[0002] In the field of 3D reconstruction and measurement, a vision-based method for target point measurement and positioning is often adopted. The target points on the target object are captured by a monocular camera, and their 3D coordinates are calculated using the projection relationship. For example, using multiple frames of data from a single camera, the 3D coordinates are estimated by methods such as structured light or simultaneous localization and mapping.

[0003] However, in the related art, due to camera pose estimation errors or optical distortions, in a large-scale scene, due to the existence of cumulative errors, layering often occurs at the loopback, seriously affecting the accuracy of 3D scanning and reconstruction. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems existing in the related art. For this purpose, this application proposes a method, apparatus, and scanning device for target point positioning to improve the measurement accuracy.

[0005] In a first aspect, this application provides a method for target point positioning, the method including:

[0006] Obtain a target point set; the target point set includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in a binocular camera;

[0007] Determine an error function based on the image coordinates of the target points in the binocular camera, and optimize the error function to update the global coordinates of the target points;

[0008] Identify the same target points in the target point set based on the updated global coordinates, and perform deduplication processing on the target point set based on the same target points to obtain a new target point set;

[0009] Return to the first step and continue to execute until the iteration termination condition is met to obtain the final target point set.

[0010] In the above technical solution, by recording the target points and coordinate information measured in multiple observation frames during the scanning process and forming a target point set, it is possible to optimize based on multi-frame observation data, improve the consistency of the target point data, and make the measurement result not affected by the error of a single frame; by determining the error function based on the image coordinates of the target point in the binocular camera and using the error function as the optimization target for constrained solution to optimize the target point coordinates, it is possible to reduce the measurement error and make the coordinates of the target point more accurate; and after iterative optimization, by identifying the same target points in the target point set based on the updated global coordinates, and performing duplicate removal on the target point set based on the same target points, and repeating multiple times until the iterative termination condition is met to obtain the final target point set, it is possible to effectively merge multiple observation points of the same physical point, eliminate the layering phenomenon caused by cumulative error, avoid incorrect matching of target points, and greatly improve the positioning accuracy of target points.

[0011] According to an embodiment of the present application, the determining the error function based on the image coordinates of the target point in the binocular camera includes:

[0012] Determining the projection coordinates of the target point projected onto the binocular camera in multiple observation frames;

[0013] Based on the difference between the projection coordinates and the actual image coordinates, determining the binocular reprojection error;

[0014] Constrained by the reference error, constructing an error function based on the binocular reprojection error.

[0015] In the above embodiment, through the optimization of the binocular reprojection error, the reprojection deviation of the target point coordinates is reduced, and the three-dimensional coordinate accuracy is improved; by introducing the reference error constraint, it is possible to help constrain and optimize the scale, error, and coordinate accuracy in three-dimensional measurement, and avoid deformation or distortion caused by error accumulation.

[0016] In some embodiments, the determining the projection coordinates of the target point projected onto the binocular camera in multiple observation frames and determining the binocular reprojection error based on the difference between the projection coordinates and the actual image coordinates includes:

[0017] Determining the first projection coordinates of the target point in the first camera and calculating the difference between the first projection coordinates and the actual image coordinates in the first camera to obtain the first reprojection error;

[0018] Determining the second projection coordinates of the target point in the second camera and calculating the difference between the second projection coordinates and the actual image coordinates in the second camera to obtain the second reprojection error;

[0019] Based on the first reprojection error and the second reprojection error, determining the binocular reprojection error.

[0020] In the above embodiments, by optimizing the binocular reprojection error, the deviation caused by the projection error in the 3D reconstruction process can be reduced, so that the global coordinates of the target points are closer to the real positions. And compared with a monocular camera, by fusing the information of the binocular camera, the measurement accuracy can be further reduced.

[0021] In a second aspect, the present application provides a target point positioning device, which includes:

[0022] An acquisition module, configured to acquire a target point set; the target point set includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in the binocular camera;

[0023] An optimization module, configured to determine an error function based on the image coordinates of the target points in the binocular camera, and optimize the error function to update the global coordinates of the target points;

[0024] A duplicate removal module, configured to identify the same target points in the target point set based on the updated global coordinates, and perform duplicate removal processing on the target point set based on the same target points to obtain a new target point set;

[0025] An iteration module, configured to return to the first step and continue to execute until an iteration termination condition is met, to obtain a final target point set.

[0026] In the above technical solution, by recording the target points and coordinate information measured in multiple observation frames during the scanning process and forming a target point set, the optimization based on multi-frame observation data can be performed to improve the consistency of the target point data, so that the measurement result is not affected by the error of a single frame; by determining an error function based on the image coordinates of the target points in the binocular camera and using the error function as an optimization target for constrained solution to optimize the target point coordinates, the measurement error can be reduced and the coordinates of the target points can be made more accurate; and after iterative optimization, by identifying the same target points in the target point set based on the updated global coordinates, performing duplicate removal processing on the target point set based on the same target points, and repeating multiple times until the iteration termination condition is met to obtain a final target point set, the multiple observation points of the same physical point can be effectively merged, the layering phenomenon caused by cumulative error can be eliminated, the mis-matching of target points can be avoided, and the target point positioning accuracy can be greatly improved.

[0027] In a third aspect, the present application provides a scanning device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, it implements the target point positioning method as described in the first aspect above.

[0028] Fourthly, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the target point positioning method described in the first aspect above is implemented.

[0029] Fifthly, the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the target point positioning method described in the first aspect above.

[0030] Sixthly, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the target point positioning method described in the first aspect above is implemented.

[0031] The additional aspects and advantages of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present application. Description of the Drawings

[0032] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, where:

[0033] Figure 1 is a schematic flowchart of the target point positioning method provided by the present application in some embodiments;

[0034] Figure 2 is a schematic diagram of the principle of binocular projection of the target point provided by the present application in some embodiments;

[0035] Figure 3 is a schematic flowchart of constructing an error function provided by the present application in some embodiments;

[0036] Figure 4 is a schematic flowchart of determining the binocular reprojection error provided by the present application in some embodiments;

[0037] Figure 5 is a schematic structural diagram of the target point positioning device provided by the present application in some embodiments;

[0038] Figure 6 is a schematic structural diagram of the scanning device provided by the present application in some embodiments. Detailed Embodiments

[0039] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the present application.

[0040] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application pertains; the terms used in the description of this application in the specification are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and not to describe a specific order or primary-secondary relationship.

[0041] Referring to "embodiment" in this application means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appearing at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0042] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "coupled", "attached" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0043] The term "and / or" in this application is merely a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally represents an "or" relationship between the associated objects before and after.

[0044] The term "a plurality of" appearing in this application refers to two or more (including two). Similarly, "a plurality of groups" refers to two or more groups (including two groups), and "a plurality of pieces" refers to two or more pieces (including two pieces).

[0045] Due to the influence of camera calibration errors, perspective changes, and cumulative errors, the target points may have position deviations in different frames, and even cause the same physical point to be recognized as multiple different target points, namely the so-called "stratification" problem. If directly performing stitching and positioning based on the results of the target points reconstructed by monocular or binocular vision, it is very easy to generate false matches, which greatly reduces the accuracy of the three-dimensional scanning and reconstruction results.

[0046] In view of this, an embodiment of the present application proposes a target point positioning method, aiming to remove stratification through binocular joint iterative precision optimization in target point positioning; that is, through bundle adjustment optimization combined with binocular measurement data fusion, iteratively optimize the three-dimensional coordinates of the target point and eliminate duplicate target points, thereby improving the measurement accuracy, reducing the stratification phenomenon, and ensuring the stable convergence of the target point data, which is applicable to high-precision three-dimensional reconstruction.

[0047] The three-dimensional scanning method provided by the embodiment of the present application can improve the overall accuracy of three-dimensional reconstruction, and is applicable to high-precision measurement scenarios such as industrial part detection, especially applicable to medium or large measurement scenarios.

[0048] The following will combine the accompanying drawings and specifically illustrate the target point positioning method and the like provided by the embodiment of the present application through specific embodiments and their application scenarios.

[0049] The target point positioning method provided by the embodiment of the present application, and the execution subject of the target point positioning method can be a scanning device or a functional module or functional entity in the scanning device that can implement the data processing method.

[0050] It should be noted that the scanning device mentioned in the embodiment of the present application includes, but is not limited to, a handheld scanning device (such as a handheld laser scanner or a tracking scanner, etc.) and a non-handheld scanning device. The non-handheld scanning device can be, for example, an intelligent robot equipped with a binocular camera, a drone, or a surround scanning device, etc. It is easy to understand that when the scanning device is handheld, the phenomenon of target point stratification is more likely to occur, but this does not limit the scanning device. The scanning device can also be non-handheld. For example, in the case of an irregularly shaped target object, the non-handheld scanning device may also face the phenomenon of target point stratification. Therefore, the present application does not make a specific limitation on the scanning device.

[0051] The following takes the scanning device as the execution subject as an example to illustrate the target point positioning method provided by the embodiment of the present application.

[0052] As Figure 1 shown, the target point positioning method includes: Step 110 to Step 140.

[0053] Step 110, obtain a target point set; the target point set includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in the binocular camera.

[0054] The target object can be an object, a living body or a part of a living body, etc., such as a vehicle, a building, an industrial component, a human body or a part of a human body, etc. The target points in the embodiments of the present application may include non-coded marker points, coded points or marker points on marker balls provided on the target object. The target points are usually markers with special reflective materials covering their surfaces. The target points can be circular target points or a combination of circular target points and other patterns.

[0055] Among them, a binocular camera is an imaging system composed of two cameras. Generally speaking, the two cameras of the binocular camera are arranged at a fixed distance interval in the horizontal direction, so they can be called the left-eye camera or the right-eye camera. However, it is not limited to this. For example, the two cameras can also be arranged in the vertical direction, or can be arranged in central symmetry, etc. Therefore, for the sake of description convenience, one of the cameras in the binocular camera is called the first camera, and the other camera is called the second camera. The first camera can be the right-eye camera or the left-eye camera.

[0056] Specifically, the scanning device collects multiple frames of observation data in the scene, and extracts target points from the binocular images of each observation frame. For example, the user can hold the scanning device and scan around the target object to collect multiple frames of observation data. During the scanning process, the scanning device identifies the target points set on the target object and records the position information of each target point to form a target point set.

[0057] The target point set includes target points from different observation frames and their coordinate information. The coordinate information includes the global coordinates of each target point and its image coordinates in the binocular camera. The global coordinates are three-dimensional coordinates in the world coordinate system, and the image coordinates are two-dimensional coordinates in the binocular camera coordinate system. Exemplarily, for each detected target point, the scanning device uses the internal and external parameters of the binocular camera for triangulation to obtain the initially calculated three-dimensional coordinates as the global coordinates and stores them in the target point set. Another example is that after the scanning device captures the target object through the binocular camera, it determines the position of the target point in the image in each observation frame as the image coordinates and stores them in the target point set.

[0058] At this time, there may be a layering phenomenon in the target points identified by the scanning device, that is, the same physical point may be identified as multiple different target points, affecting the measurement consistency. The key to solving the layering problem lies in eliminating the cumulative error. If this process is processed in real time during the scanning process, it will consume a large amount of computing power and affect the three-dimensional reconstruction efficiency. Therefore, in the present application, the scanning device first scans multiple frames of data, and then processes the target points to eliminate layering and improve the accuracy.

[0059] Step 120: Determine an error function based on the image coordinates of the target points in the binocular camera, and optimize the error function to update the global coordinates of the target points.

[0060] Among them, the layering phenomenon is caused by the fact that the same physical point is recognized as multiple different target points during the scanning process, and this phenomenon is particularly prominent in large scenes. In the embodiments of the present application, this problem is transformed into a constraint solving problem, and an error function is constructed as the optimization objective for iterative optimization. During this process, the coordinates of the target points are continuously adjusted to make the function converge, that is, the error is minimized, and thus the accurate coordinates of the target points can be obtained.

[0061] Among them, the error function includes the error caused by the first camera and the error caused by the second camera. Due to camera calibration errors or lens distortion, when the three-dimensional coordinates of the target point are projected onto the image coordinates of the first camera, there will be a deviation from the actually observed coordinates, which is called the reprojection error. In addition, there may also be measurement errors in the internal parameters of the first camera, such as the focal length, principal point offset, pixel scaling factor, etc., resulting in inaccurate positions of the three-dimensional points projected onto the image, thereby introducing internal parameter errors. Moreover, if the external parameters of the first camera, such as the rotation matrix and translation matrix, are estimated inaccurately, it may introduce external parameter errors, resulting in an offset of the overall position of the point cloud and affecting the correctness of the target point matching.

[0062] Therefore, the error function includes one or more of, but is not limited to, reprojection error, camera internal parameter error, camera external parameter error, and distortion error, etc. For example, the error function can be obtained by weighted summation based on reprojection error, camera internal parameter error, camera external parameter error, and distortion error.

[0063] Since the projection relationship is a non-linear function, the embodiments of the present application optimize the error function through a non-linear optimization method, and at the same time optimize the global coordinates of the target points and the camera parameters to minimize the overall error. Among them, the non-linear optimization method includes, but is not limited to, bundle adjustment method, etc.

[0064] In some embodiments, the scanning device can introduce a dynamic iteration step size to determine whether the iteration termination condition is met, that is, if the error change in consecutive multiple iterations is less than the set threshold, the optimization is terminated in advance to reduce unnecessary iterations and improve real-time performance.

[0065] In other embodiments, the scanning device can introduce a local area adaptive mechanism, and different convergence criteria can be adopted for different regions. For example, a stricter convergence criterion is required for the target points in the key parts, while a looser criterion is adopted for the non-key parts, which can further reduce the computational overhead.

[0066] Step 130: Identify the same target points in the target point set based on the updated global coordinates, and perform duplicate removal processing on the target point set based on the same target points to obtain a new target point set.

[0067] After optimizing the global coordinates of the target points, the cumulative error tends to be eliminated. At this time, the observation data of all frames tend to be consistent when the camera projection error is minimized. That is, the coordinates estimated for the same target point in different frames are closer to the true value. Even if multiple target points are identified for the same physical point, the coordinates of these target points should overlap or show a concentrated trend.

[0068] Among them, the same target points refer to the target points measured in different observation frames and corresponding to the same physical space point. That is to say, although these target points are independently measured and recorded in different frames, they actually correspond to the same physical point.

[0069] Therefore, based on the updated global coordinates and combined with geometric constraints, etc., the scanning device can identify the same target points in the set of target points and fuse the same target points, such as merging their global coordinate and image coordinate information, to remove duplicate target points, thereby updating the set of target points to obtain a new set of target points.

[0070] Exemplarily, the scanning device finds target points that are relatively close through neighborhood search, determines which target points belong to the same physical point based on the distance, and merges them. For example, by assigning numbers to the target points and assigning the same number to the same target points to achieve duplicate removal processing.

[0071] Step 140: Return to the first step and continue to execute until the iteration termination condition is met to obtain the final set of target points.

[0072] The scanning device can preset the iteration termination condition, which includes but is not limited to one or more of function convergence, the iteration number reaching the upper limit, or no change in the numbers of the target points, etc. If the termination condition is not met, return to step 110 to continue processing, gradually improving the accuracy and consistency of the target points until the iteration termination condition is met to obtain the final set of target points.

[0073] With the final set of target points, the scanning device can accurately reconstruct the three-dimensional model of the target object for measurement, simulation, or visual display. Or, the scanning device can also perform high-precision environmental mapping and positioning based on this final set of target points, such as for environmental modeling of unmanned aerial vehicles, etc. Or, the scanning device can also perform human body model modeling, motion trajectory tracking, or pose recognition, etc., based on this final set of target points. Or, the scanning device can perform scene positioning and virtual-real fusion based on this final set of target points for use in AR / VR / MR scenarios.

[0074] The target point positioning method provided by the embodiments of the present application can form a target point set by recording the measured target points and coordinate information in multiple observation frames during the scanning process. It can optimize based on multi-frame observation data, improve the consistency of target point data, and make the measurement result not affected by the error of a single frame. By determining the error function based on the image coordinates of the target point in the binocular camera and using the error function as the optimization target for constrained solution to optimize the target point coordinates, it can reduce the measurement error and make the coordinates of the target point more accurate. And after iterative optimization, by identifying the same target points in the target point set based on the updated global coordinates, de-duplicating the target point set based on the same target points, and repeating multiple times until the iterative termination condition is met to obtain the final target point set, it can effectively merge multiple observation points of the same physical point, eliminate the stratification phenomenon caused by cumulative errors, avoid mis-matching of target points, and greatly improve the target point positioning accuracy.

[0075] In three-dimensional reconstruction based on a binocular camera, the accuracy of the target point is affected by the observation error. Therefore, it is necessary to construct an error function and optimize it. The imaging principle of the binocular camera determines that the reprojection error can be used to measure the accuracy of the three-dimensional point. By comparing the image coordinates and projection coordinates of the target point in the binocular camera, an error function can be constructed, and based on this, the three-dimensional coordinates of the target point can be optimized. In addition, in order to further improve the accuracy, a reference error constraint can be introduced to ensure the consistency of the global scale during the optimization process.

[0076] Among them, reprojection refers to the secondary projection of a spatial point. During the first projection, the camera captures the target object, and the target points on it are mapped onto the image, that is, the global coordinates of the target point are converted from the world coordinate system to the camera coordinate system to obtain the image coordinates. Since it is a binocular camera, based on the calculated camera pose, etc., the image coordinates in one camera coordinate system can be converted to the image coordinates in another camera coordinate system again, and this process is the reprojection. As Figure 2 shown, the spatial point P is mapped to the image coordinate system of the first camera to obtain the pixel point p1, and the coordinates of the pixel point p1 are the image coordinates of the spatial point P in the first camera. And when the spatial point P is converted to the second camera coordinate system, the pixel point p'2 can be obtained, and there is a certain difference between the pixel point p'2 and the detected pixel point p2 in the image, and this difference is the reprojection error.

[0077] For this reason, in some embodiments, as Figure 3 shown, determining the error function based on the image coordinates of the target point in the binocular camera includes steps 310 to 320:

[0078] Step 310: Determine the projection coordinates of the target point projected onto the binocular camera in multiple observation frames, and determine the binocular reprojection error based on the difference between the projection coordinates and the actual image coordinates;

[0079] Step 320: Under the constraint of the reference error, construct an error function based on the binocular reprojection error.

[0080] For each target point in multiple observation frames, the scanning device calculates its coordinates projected onto the binocular camera based on the camera pose and camera parameters. The camera parameters include but are not limited to rotation matrix, translation matrix, etc. Exemplarily, the scanning device can perform triangulation on the target points in the image, use geometric information to construct triangles to determine the positional relationship between the target points, and further obtain the coordinate conversion relationship from the spatial points to the pixel points. Also, for example, the scanning device can be calibrated based on a reference object to obtain the internal and external parameters of the camera, and then combine the global coordinates of the target points to calculate the projected coordinates of the target points in the camera coordinate system.

[0081] The actual image coordinates are the pixel coordinates of the target points in the binocular camera image detected through image processing algorithms, etc. Due to possible camera calibration errors or lens distortions, etc., they are noisy and have errors, which leads to a difference between them and the projected coordinates.

[0082] Furthermore, the scanning device calculates the binocular reprojection error based on the difference between the projected coordinates and the actual image coordinates, and under the constraint of the reference error, uses the binocular reprojection error as the optimization target to construct an error function, providing a basis for subsequent optimization.

[0083] Among them, in the scanning scene, a reference object can be placed to provide known geometric information, help optimize the calculation accuracy, and prevent scale drift. Exemplarily, the reference object can be one or more of a scale / ruler / gage with a known length or scale, a calibration board, or other objects with known dimensions (such as a standard sphere, a standard cube, etc.). The number of reference objects is not limited. Thus, the reference error can be, for example, the difference between the spatial distance between any two or more reference points on the reference object and the corresponding true value.

[0084] In the above embodiments, through the optimization of the binocular reprojection error, the reprojection deviation of the target point coordinates is reduced, and the three-dimensional coordinate accuracy is improved; by introducing the reference error constraint, it can help constrain and optimize the scale, error, and coordinate accuracy in three-dimensional measurement, and avoid deformation or distortion caused by error accumulation.

[0085] To improve the measurement accuracy, the measurement results can be evaluated and compensated for errors using reference points with known true spatial distances. For this purpose, in some embodiments, the above method further includes: determining at least two reference points measured in all observation frames; determining the reference error based on the difference between the spatial distance between the reference points and the corresponding true value.

[0086] Specifically, the scanning device can measure and record the three-dimensional coordinates of at least two reference points in multiple observation frames. Thus, based on the measured three-dimensional coordinates of the reference points, the scanning device calculates the Euclidean distance between them to obtain the spatial distance between the two. The spatial distance can be, for example, one or more of Euclidean distance, Manhattan distance, Chebyshev distance, or cosine distance, etc.

[0087] Through physical measurement, a scale, or known calibration data, the scanning device can obtain the true spatial distance between the two reference points as the ground truth. Thus, by comparing the difference between the measured spatial distance and the true spatial distance, the scanning device can calculate the reference error for subsequent optimization objective constraints.

[0088] In the above embodiments, by introducing reference points and calculating the reference error, the measurement accuracy can be effectively evaluated, and systematic deviations caused by camera calibration errors, perspective transformation errors, etc. can be identified and compensated, thereby improving the global accuracy of the final target point set.

[0089] Since a binocular camera consists of two cameras, and each camera will image the same target point, calculating the reprojection errors of the first camera and the second camera respectively and combining them can more accurately describe the projection error, thereby optimizing the accuracy of three-dimensional coordinate calculation. For this purpose, in some embodiments, as Figure 4 shown, determine the projection coordinates of the target point projected into the binocular camera in multiple observation frames, and based on the difference between the projection coordinates and the actual image coordinates, determine the binocular reprojection error, including steps 410 to 430:

[0090] Step 410: Determine the first projection coordinates of the target point in the first camera, and calculate the difference between the first projection coordinates and the actual image coordinates in the first camera to obtain the first reprojection error;

[0091] Step 420: Determine the second projection coordinates of the target point in the second camera, and calculate the difference between the second projection coordinates and the actual image coordinates in the second camera to obtain the second reprojection error;

[0092] Step 430: Based on the first reprojection error and the second reprojection error, determine the binocular reprojection error.

[0093] Specifically, the scanning device acquires multiple observation frames, extracts the image coordinates of the target point in the binocular camera, and calculates the initial three-dimensional coordinates through the parameters of the binocular camera. Exemplarily, the actual image coordinates can be detected by image detection methods such as morphological processing.

[0094] For the first camera, the scanning device uses the internal and external parameters of the first camera to project the three-dimensional coordinates of the target point onto the imaging plane of the first camera, obtaining the first projection coordinates. Furthermore, the scanning device calculates the difference between the first projection coordinates and the actual image coordinates in the first camera, obtaining the first reprojection error.

[0095] Exemplarily, taking the first camera as the right-eye camera as an example, the first reprojection error can be expressed as:

[0096]

[0097] where CostL is the first reprojection error, that is, the right-camera reprojection error, pifL is the actual image coordinate of the target point M i in the right-eye image of the f-th frame, pifL′ is the target point M i according to the pose t of the f-th frame f and the camera parameters c of the right-eye camera f calculated projection coordinates, that is, the first projection coordinates. N is the number of target points.

[0098] For the second camera, the scanning device projects the three-dimensional coordinates of the target point onto the imaging plane of the second camera in the same way as the first camera, obtaining the second projection coordinates, and calculates the difference between the second projection coordinates and the actual image coordinates in the second camera, obtaining the second reprojection error.

[0099] Exemplarily, taking the first camera as the right-eye camera as an example, the first reprojection error can be expressed as:

[0100]

[0101] where CostR is the second reprojection error, that is, the left-camera reprojection error, pifR is the actual image coordinate of the target point M i in the left-eye image of the f-th frame, pifR′ is the target point M i according to the pose t of the f-th frame f and the camera parameters c of the left-eye camera f calculated projection coordinates, that is, the second projection coordinates.

[0102] Exemplarily, taking the first camera as the right-eye camera and the second camera as the left-eye camera as an example, the spatial coordinate transformation relationship between the left-eye camera and the right-eye camera can be expressed as follows:

[0103] P roght =R right ×P global +T right (3)

[0104] P left =R extrinsic ×Pright +T extrinsic (4)

[0105] Among them, R right is the rotation matrix for converting the world coordinate system to the right-eye camera coordinate system, and P global is the three-dimensional coordinate of the target point P in the world coordinate system, and T right is the translation matrix for converting the world coordinate system to the right-eye camera coordinate system, and R extrinsic is the rotation matrix for converting the right-eye camera coordinate system to the left-eye camera coordinate system, and T extrinsic is the translation matrix for converting the right-eye camera coordinate system to the left-eye camera coordinate system.

[0106] Thus, based on the first reprojection error and the second reprojection error, the stereo reprojection error is obtained by the scanning device to characterize the overall projection error of the target point. Exemplarily, the stereo reprojection error is the sum or weighted sum of the first reprojection error and the second reprojection error, etc. For example, the optimization objective of the stereo reprojection error can be expressed as:

[0107] min∑(‖pi - p′i‖+‖spi - sp′i‖) (5)

[0108] Among them, pi represents the projection coordinate of the coordinate of the spatial point p in the right camera coordinate system projected onto the right camera, and p′i represents the actual image coordinate of the spatial point p in the right-eye image; spi represents the projection coordinate of the coordinate of the spatial point p in the left camera coordinate system projected onto the left camera, and sp′i represents the actual image coordinate of the spatial point p in the left-eye image.

[0109] In the above embodiments, by optimizing the stereo reprojection error, the deviation caused by the projection error in the three-dimensional reconstruction process can be reduced, so that the global coordinates of the target point are closer to the real position, and compared with the monocular camera, by fusing the information of the binocular cameras, the measurement accuracy can be further reduced.

[0110] In the process of three-dimensional measurement and reconstruction based on binocular cameras, it is necessary to project the three-dimensional coordinates of the target point onto the camera imaging plane for error calculation and optimization. However, the three-dimensional coordinates of the target point are usually defined in the global coordinate system, while the imaging coordinate system of the camera is a different local coordinate system. Therefore, in order to correctly calculate the projection coordinates, the conversion parameters of the camera need to be used to convert the global coordinates into the coordinate systems of each camera, and further obtain the projection coordinates. Since the actual image coordinates are detected from the real image, and the projection coordinates are calculated by projection transformation based on the three-dimensional points, the difference between the two is the source of the reprojection error.

[0111] To this end, in some embodiments, determining the projection coordinates of a target point projected into a binocular camera in multiple observation frames includes: based on a first conversion parameter between a global coordinate system and a first camera coordinate system, converting the global coordinates of the target point in the multiple observation frames into first projection coordinates projected into the first camera; based on a second conversion parameter between the first camera coordinate system and a second camera coordinate system, converting the first projection coordinates into second projection coordinates projected into the second camera.

[0112] Specifically, the scanning device, based on the first conversion parameter between the global coordinate system and the first camera coordinate system, where the first conversion parameter includes a rotation matrix and a translation matrix for converting the global coordinate system to the first camera coordinate system, converts the global coordinates of the target point into image coordinates in the first camera coordinate system based on this first conversion parameter. Similarly, the scanning device, based on the second conversion parameter between the global coordinate system and the second camera coordinate system, where the second conversion parameter includes a rotation matrix and a translation matrix for converting the first camera coordinate system to the second camera coordinate system, converts the global coordinates of the target point into image coordinates in the second camera coordinate system based on this second conversion parameter.

[0113] In the above embodiments, through the conversion between global coordinates and camera coordinates, the deviation that may be brought about by directly calculating errors in image coordinates is avoided, the accuracy of calculating projection coordinates is improved, and it is applicable to both horizontally arranged binocular cameras and vertically arranged binocular cameras, improving the flexibility of 3D reconstruction.

[0114] After obtaining the error function, in some embodiments, optimizing the error function to update the global coordinates of the target point includes: taking the minimization of the error output by the error function as the optimization goal, and iteratively optimizing by adjusting the global coordinates of the target point and the camera parameters of the binocular camera until convergence to obtain the updated global coordinates. In other words, the input of the error function is the global coordinates of the target point, camera parameters (such as extrinsic and intrinsic parameters), and the calculated binocular reprojection error, and the output is the calculated total error value. Taking the minimization of the error as the optimization goal, using a non - linear least - squares optimization algorithm, such as bundle adjustment, etc., continuously adjust the global coordinates of the target point and the camera parameters, and through repeated iteration, gradually reduce the error until the iteration termination condition is met. Finally, the scanning device uses the optimized global coordinates of the target point as the final output to provide more accurate spatial position information.

[0115] Thus, by optimizing the error function, the influence brought about by camera error, projection error, and measurement error can be effectively reduced, making the finally calculated global coordinates of the target point more accurate and improving the overall accuracy of binocular measurement.

[0116] As described above, in order to reduce redundancy and improve reconstruction accuracy, it is necessary to match and identify target points based on the optimized global coordinates to ensure that the same physical target point has a unique correspondence in the target point set. To this end, in some embodiments, identifying the same target points in the target point set based on the updated global coordinates includes: in the current match, for any target point in the target point set, determining one or more target target points that match the targeted target point; assigning the same number to the targeted target point and the one or more target target points that match it to obtain other target points that are the same as the targeted target point; traversing the target point set again based on the result of the current match until all the same target points in the target point set are determined.

[0117] Specifically, the scanning device determines all the same target points in the target point set through multiple iterative matches. For example, in the current match, the scanning device sequentially selects each target point in the target point set as the current target point, and determines one or more target target points that match the targeted target point through neighborhood search. The matching of a target point and a target target point includes, but is not limited to, the spatial positions of the target point and the target target point being the same or close to each other, etc.

[0118] The scanning device can assign a number to each identified target point for identification. For the one or more target target points that match the current target point, the scanning device unifies the numbers of these target points to indicate that these target points are essentially the same target point.

[0119] After that, based on the result of the current match, the scanning device traverses the target point set again to determine the numbers of other target points using the same number that has been marked. Through multiple iterative matches, the scanning device can find all the same target points, assign the same number to the same target points, and further, the scanning device can identify all the same target points to eliminate the stratification phenomenon.

[0120] In the above embodiments, by matching target points based on global coordinates and assigning a unified number, it is possible to effectively eliminate the repetition of target points caused by measurement errors, improve the accuracy and stability of three-dimensional reconstruction, reduce data redundancy at the same time, and improve the calculation efficiency.

[0121] Among them, in some embodiments, for any target point in the target point set, determining one or more target target points that match the targeted target point includes: for any target point in the target point set, determining the spatial distance between the targeted target point and other target points; screening out one or more other target points with a spatial distance less than a preset distance threshold as the one or more target target points that match the targeted target point.

[0122] Specifically, the scanning device sequentially selects each target point in the target point set as the current target point and performs matching calculations. For example, the scanning device calculates the spatial distances between the current target point and all other target points in the target point set. By setting a preset distance threshold, the scanning device can filter out the target points whose spatial distances from the current target point are less than the threshold as the target target points. Thus, the scanning device uses the filtered target target points as the target point set that matches the current target point for subsequent target point merging or number assignment.

[0123] For another example, the scanning device can perform a neighborhood search to find other target points near the current target point and calculate the spatial distance between the current target point and the other target point; that is, search in the target point set for other target points whose distances are within the preset threshold range as candidate target target points. The scanning device calculates the spatial distance between the current target point and the candidate target target points. If the spatial distance meets the matching condition, it is considered that these target points belong to the same physical point. Thus, compared with calculating all other target points in the target point set, the amount of calculation is further reduced.

[0124] In the above embodiments, by calculating the spatial distances between target points and filtering out the target target points, it is possible to effectively reduce the duplication of target points caused by measurement errors, improve the matching accuracy, provide reliable data support for subsequent target point merging and optimization, and thus enhance the accuracy and stability of 3D reconstruction.

[0125] In some other embodiments, the scanning device can also calculate the topological structure (such as the relative angles, distances, etc. of adjacent points) of the target point in its local neighborhood, and improve the matching method from single-point matching to neighborhood matching. If the neighborhood topological structures of two target points are similar, they are more likely to be the same physical point. Specifically, the scanning device traverses each target point P in the target point set and calculates the spatial distance between the target point P and all other target points. By setting a neighborhood radius R, the scanning device filters out all neighboring target points whose distances from P are less than r, denoted as the neighborhood point set {P}, and calculates the topological relationships of all neighborhood points in the neighborhood point set {P} with respect to the target point P. The topological relationships include, but are not limited to, one or more of relative angles, relative distances, and topological arrangements (such as triangular or nearest neighbor relationships), etc. Thus, the scanning device obtains the local topology of the target point P and each neighborhood point, and stores the global coordinates, image coordinates, neighborhood point information, etc. of P together for subsequent matching use.

[0126] After that, the scanning device compares the local topology of the target point P with the local topologies of each neighboring point. If the topological structures of the target point P and the neighboring point Q are similar enough (e.g., exceeding a set topological matching threshold), it is determined that the target point P and the neighboring point Q are the same target point. Thus, false matching and false deletion can be effectively reduced, the reliability of target point matching can be improved, and further the accuracy and stability of the final target point data can be enhanced.

[0127] To improve the accuracy and stability of the target points, it is necessary to deduplicate the target point set based on the matching results and fuse the coordinate information of the same physical target point to obtain a more stable target point set. For this purpose, in some embodiments, the target point set is deduplicated based on the same target points to obtain a new target point set, including: merging the coordinate information corresponding to the target points with the same number to fuse the global coordinates and / or the image coordinates of the binocular cameras of the same target point in multiple observation frames to obtain a new target point set.

[0128] The scanning device reads the matching information of all target points in the target point set, identifies the target points with the same number, and determines that they belong to the same physical point. Then, the scanning device merges the coordinate information corresponding to the target points with the same number to remove the duplicate target points corresponding to the same physical point.

[0129] The merging process includes but is not limited to: fusing the global coordinates of the same target point in multiple observation frames, and fusing the image coordinates of the binocular cameras of the same target point in multiple observation frames, etc. For example, the scanning device can calculate the mean value of the global coordinates of the target points with the same number in the world coordinate system and replace the original global coordinates with this mean value; another example is that the scanning device can calculate the mean value of the image coordinates of the target points with the same number in the camera coordinate system and replace the original image coordinates with this mean value. Thus, redundant data can be removed to generate a new target point set.

[0130] In the above embodiments, merging the coordinate information of the matched target points can reduce the coordinate deviation caused by measurement errors and improve the positioning accuracy of the target points. At the same time, by fusing the information of multiple observation frames, the stability of the target points can be enhanced, making them more reliable in subsequent applications such as 3D reconstruction.

[0131] In some embodiments, the scanning device can adopt coordinate merging based on confidence weighting to improve the ability to suppress error points. For example, the scanning device can calculate the confidence through the reprojection error, and the reprojection error is inversely proportional to the confidence, that is, the smaller the reprojection error, the higher the confidence.

[0132] For another example, the scanning device can calculate the confidence level based on the number of times a target point is observed in multiple observation frames. The more times it is observed, the more stable it is, and the higher the confidence level should be. If a target point appears in only a small number of observation frames, its weight is reduced to avoid the influence of outliers on the global calculation.

[0133] For yet another example, if a target point has a small change in position (exists stably) in the time dimension, the confidence level is high. For example, calculate the variance of the positions of the target points in multiple frames. The smaller the variance, the higher the confidence level.

[0134] Thus, the scanning device can perform a merging process on the coordinate information corresponding to the target points with the same number based on the confidence level to remove duplicate target points corresponding to the same physical point. Assume that during the deduplication process, multiple target points P1, P2,..., Pn correspond to the same physical point, and the corresponding three-dimensional coordinates are (X i , Y i , Z i ). Exemplarily, the final target point coordinates (X i ′, Y i ′, Z i ′) can be expressed as:

[0135] Thus, by introducing the confidence level to merge the same target points, the data points with high confidence levels contribute more, reducing the influence of error points and improving the stability of the three-dimensional target points.

[0136] The following is illustrated with a specific example.

[0137] Several target points are placed in the scene to be measured. For example, they can be set on the surface of the target object, and there are reference objects placed in the environment. Taking a scale with known true values as an example, the scale has special markings that can be distinguished from ordinary target points. When the scanning device scans the target object, it uses the binocular scanning method to collect scanning data and obtains a scanning data set. The scanning data set includes, but is not limited to, the following data: the initial three-dimensional target points M = {m1, m2, m3....mN} (i.e., the global coordinates of the target points, N is the number of target points), the camera poses T = {t1, t2, t3....tM} for each frame (M is the number of frames), the two-dimensional image points P = {p1, p2, p3....pN} (i.e., the image coordinates, N is the number of target points), the camera parameters C = {c1, c2, c3.....cM} (M is the number of frames), and the scale recognition data R.

[0138] Among them, in the initial three-dimensional target points M, there may be duplicate target points, that is, target points with different coordinates correspond to the same physical space point, but are recognized as multiple points due to the existence of cumulative errors. Each target point in the two-dimensional image point P includes its left image coordinates and right image coordinates in all observation frames. The camera parameters C for each frame include the left camera internal parameters intrinsL, the right camera internal parameters intrinsR, and the left and right camera external parameters extrinsic. The scale identification data R indicates which points in M belong to the scale points (i.e., reference points).

[0139] After the scan data is acquired, the scanning device can post-process the scan data to eliminate target point errors. Exemplarily, the scanning device uses the bundle adjustment method with additional scale constraints to optimize the coordinates of the target points. Specifically, the scanning device respectively determines the left image reprojection error and the right image reprojection error (corresponding to the first reprojection error and the second reprojection error respectively), thereby obtaining the binocular reprojection error. The binocular reprojection error is, for example, the sum or weighted sum of the left image reprojection error and the right image reprojection error. In addition, the scanning device also determines the scale length error (i.e., the reference error) as a proportional reference. Thus, the scanning device constructs an error function, and the specific steps and processes can refer to the previous content and will not be elaborated here. Exemplarily, the scanning device can reduce the error by using the gradient descent method until the error function iteratively converges to the minimum error, and obtain the optimized target point coordinates M’ = {m’1, m’2, m’3....m’N}.

[0140] After that, the scanning device can search for target points close to each target point in a neighborhood search manner, form target point pairs, assign the same number to each target point within a target point pair, then traverse the target point scan frame data from the beginning, use the already marked same number to determine the numbers of other target points, and assign the same number after finding all the same points. Thus, the scanning device then fuses the coordinate information of the same points. For example, the scanning device can fuse the three-dimensional global coordinates and / or the two-dimensional image coordinates, such as taking the average of the two coordinates. The scanning device can continuously repeat the steps from calculating the reprojection error to fusing the coordinate information until no target point numbers change or exceed a certain number of iterations, etc., and determine that the iteration termination condition is met to obtain the final target point set.

[0141] Thus, in the final set of target points, the cumulative error has been eliminated. Subsequently, operations such as 3D reconstruction can be performed based on this final set of target points to achieve high-precision measurement and reconstruction tasks. In the above method, there is no need to set coded target points, and it can eliminate the stratification phenomenon caused by cumulative error, avoid incorrect matching of target points, and greatly improve the positioning accuracy of target points. Compared with the method of using threshold judgment to determine whether there is a stratification phenomenon, the present application can be applied to multiple scenarios. Especially in large scenarios, it can well maintain the accuracy, while the former cannot be applied in large scenarios because no matter how the threshold is set, there will always be scenarios where the threshold is exceeded, resulting in the inability to eliminate the cumulative error.

[0142] In the target point positioning method provided by the embodiments of the present application, the execution subject may be a target point positioning device. In the embodiments of the present application, taking the target point positioning device executing the target point positioning method as an example, the target point positioning device provided by the embodiments of the present application is described.

[0143] The embodiments of the present application further provide a target point positioning device, which is applied to a scanning device. As Figure 5 shown, the target point positioning device includes an acquisition module 501, an optimization module 502, a duplicate removal module 503, and an iteration module 504. Among them:

[0144] The acquisition module 501 is configured to acquire a set of target points; the set of target points includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in a binocular camera;

[0145] The optimization module 502 is configured to determine an error function based on the image coordinates of the target points in the binocular camera, and optimize the error function to update the global coordinates of the target points;

[0146] The duplicate removal module 503 is configured to identify the same target points in the set of target points based on the updated global coordinates, and perform duplicate removal processing on the set of target points based on the same target points to obtain a new set of target points;

[0147] The iteration module 504 is configured to return to the first step and continue to execute until the iteration termination condition is met to obtain the final set of target points.

[0148] According to the target point positioning device provided by the embodiments of the present application, by recording the target points and coordinate information measured in multiple observation frames during the scanning process and forming a target point set, it is possible to optimize based on multi-frame observation data, improve the consistency of target point data, and make the measurement result not affected by the error of a single frame; by determining an error function based on the image coordinates of the target point in the binocular camera and using the error function as the optimization target for constrained solution to optimize the target point coordinates, it is possible to reduce the measurement error and make the coordinates of the target point more accurate; and after iterative optimization, by identifying the same target points in the target point set based on the updated global coordinates, and performing duplicate removal processing on the target point set based on the same target points, and repeating multiple times until the iterative termination condition is met to obtain the final target point set, it is possible to effectively merge multiple observation points of the same physical point, eliminate the stratification phenomenon caused by cumulative errors, avoid incorrect matching of target points, and greatly improve the target point positioning accuracy.

[0149] In some embodiments, the optimization module is further configured to determine the projection coordinates of the target point projected onto the binocular camera in multiple observation frames, and determine the binocular reprojection error based on the difference between the projection coordinates and the actual image coordinates; under the constraint of the reference error, construct an error function based on the binocular reprojection error.

[0150] In some embodiments, the binocular camera includes a first camera and a second camera; the optimization module is further configured to determine the first projection coordinates of the target point in the first camera, and calculate the difference between the first projection coordinates and the actual image coordinates in the first camera to obtain the first reprojection error; determine the second projection coordinates of the target point in the second camera, and calculate the difference between the second projection coordinates and the actual image coordinates in the second camera to obtain the second reprojection error; determine the binocular reprojection error based on the first reprojection error and the second reprojection error.

[0151] In some embodiments, the optimization module is further configured to convert the global coordinates of the target point in multiple observation frames into the first projection coordinates projected onto the first camera based on the first conversion parameter between the global coordinate system and the first camera coordinate system; convert the first projection coordinates into the second projection coordinates projected onto the second camera based on the second conversion parameter between the first camera coordinate system and the second camera coordinate system.

[0152] In some embodiments, the optimization module is further configured to determine at least two reference points measured in all observation frames; determine the reference error based on the difference between the spatial distance between the reference points and the corresponding true value.

[0153] In some embodiments, the optimization module is further configured to take the minimization of the error output by the error function as the optimization target, and perform iterative optimization by adjusting the global coordinates of the target point and the camera parameters of the binocular camera until convergence to obtain the updated global coordinates of the target point.

[0154] In some embodiments, the deduplication module is further configured to, in the current matching, for any target point in the target point set, determine one or more target target points that match the targeted target point; assign the same number to the targeted target point and the one or more target target points that match it to obtain other target points that are the same as the targeted target point; and traverse the target point set again based on the current matching result until all identical target points in the target point set are determined.

[0155] In some embodiments, the deduplication module is further configured to, for any target point in the target point set, determine the spatial distance between the targeted target point and other target points; and screen out one or more other target points whose spatial distance is less than a preset distance threshold as the one or more target target points that match the targeted target point.

[0156] In some embodiments, the coordinate information corresponding to the target points with the same number is merged to fuse the global coordinates and / or the image coordinates of a binocular camera of the same target point in multiple observation frames, thereby obtaining a new target point set.

[0157] The target point positioning device in the embodiments of the present application may be a scanning device or a component in a scanning device, such as an integrated circuit or a chip.

[0158] The target point positioning device in the embodiments of the present application may be a device with an operating system. The operating system may be the Microsoft (Windows) operating system, the Android operating system, the IOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0159] The target point positioning device provided in the embodiments of the present application can implement each process implemented in the above method embodiments. To avoid repetition, it will not be elaborated here.

[0160] In some embodiments, as Figure 6 shown, the embodiments of the present application further provide a scanning device 600, including a processor 601, a memory 602, and a computer program stored on the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements each process in the above method embodiments and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0161] It should be noted that the scanning device in the embodiments of the present application includes the above-mentioned mobile scanning device and non-mobile scanning device, and may also be other devices with a binocular camera and a scanning function, such as intelligent robots, medical scanning devices, or drones.

[0162] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above-mentioned embodiment of the target point positioning method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0163] Among them, the processor is the processor in the scanning device described in the above embodiment. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0164] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above-mentioned target point positioning method.

[0165] Among them, the processor is the processor in the scanning device described in the above embodiment. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0166] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned embodiment of the target point positioning method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0167] It should be understood that the chip mentioned in the embodiment of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0168] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0169] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0170] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

[0171] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0172] If there is no special instruction, all embodiments and optional embodiments of the present application can be combined with each other to form a new technical solution.

[0173] If there is no special instruction, all technical features and optional technical features of the present application can be combined with each other to form a new technical solution.

[0174] If there is no special instruction, all steps of the present application can be carried out in sequence or randomly, and preferably in sequence. For example, the method includes steps (a) and (b), which means that the method can include steps (a) and (b) carried out in sequence, or can also include steps (b) and (a) carried out in sequence. For example, it is mentioned that the method may further include step (c), which means that step (c) can be added to the method in any order. For example, the method can include steps (a), (b), and (c), or can also include steps (a), (c), and (b), or can also include steps (c), (a), and (b), etc.

[0175] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for target point positioning, characterized in that, The method includes: Obtaining a target point set; the target point set includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in a binocular camera; Determining an error function based on the image coordinates of the target points in the binocular camera, and optimizing the error function to update the global coordinates of the target points; Identifying identical target points in the target point set based on the updated global coordinates, and performing duplicate removal processing on the target point set based on the identical target points to obtain a new target point set; Returning to the first step to continue execution until an iteration termination condition is satisfied to obtain a final target point set.

2. The method according to claim 1, wherein The determining an error function based on the image coordinates of the target points in the binocular camera includes: Determining the projection coordinates of the target points projected onto the binocular camera in multiple observation frames, and determining a binocular reprojection error based on the difference between the projection coordinates and the actual image coordinates; Constructing an error function based on the binocular reprojection error under the constraint of a reference error.

3. The method according to claim 2, wherein The binocular camera includes a first camera and a second camera; the determining the projection coordinates of the target points projected onto the binocular camera in multiple observation frames, and determining a binocular reprojection error based on the difference between the projection coordinates and the actual image coordinates includes: Determining the first projection coordinates of the target points in the first camera, and calculating the difference between the first projection coordinates and the actual image coordinates in the first camera to obtain a first reprojection error; Determining the second projection coordinates of the target points in the second camera, and calculating the difference between the second projection coordinates and the actual image coordinates in the second camera to obtain a second reprojection error; Determining a binocular reprojection error based on the first reprojection error and the second reprojection error.

4. The method according to claim 3, wherein The determining the projection coordinates of the target points projected onto the binocular camera in multiple observation frames includes: Based on a first conversion parameter between the global coordinate system and the first camera coordinate system, converting the global coordinates of the target points in multiple observation frames into first projection coordinates projected onto the first camera; Based on a second conversion parameter between the first camera coordinate system and the second camera coordinate system, converting the first projection coordinates into second projection coordinates projected onto the second camera.

5. The method according to claim 2, characterized in that The method further includes: Determining at least two reference points measured in all observation frames; Determining a reference error based on the difference between the spatial distance between the reference points and the corresponding true value.

6. The method according to any one of claims 1 to 5, characterized in that, The optimizing the error function to update the global coordinates of the target points includes: Taking the minimization of the error output by the error function as the optimization objective, and performing iterative optimization until convergence by adjusting the global coordinates of the target points and the camera parameters of the binocular camera to obtain the updated global coordinates of the target points.

7. The method according to claim 1, characterized in that, The identifying identical target points in the target point set based on the updated global coordinates includes: In the current match, for any target point in the target point set, determining one or more target target points that match the targeted target point; Assigning the same number to the targeted target point and the one or more target target points that match it to obtain other target points that are the same as the targeted target point; Traverse the target point set again based on the current matching result until all the same target points in the target point set are determined.

8. The method according to claim 7, characterized in that, Determining one or more target target points that match the target point being targeted for any target point in the target point set includes: For any target point in the target point set, determine the spatial distance between the target point being targeted and other target points; Filter out one or more other target points whose spatial distance is less than a preset distance threshold as one or more target target points that match the target point being targeted.

9. The method according to claim 7 or 8, characterized in that, The deduplication process of the target point set based on the same target points to obtain a new target point set includes: Merge the coordinate information corresponding to the target points with the same number to fuse the global coordinates and / or the image coordinates in the binocular camera of the same target point in multiple observation frames to obtain a new target point set.

10. A target point positioning device, characterized in that, The device includes: An acquisition module for acquiring a target point set; the target point set includes target points and coordinate information measured in multiple observation frames, and the coordinate information includes global coordinates and image coordinates in a binocular camera; An optimization module for determining an error function based on the image coordinates of the target point in the binocular camera and optimizing the error function to update the global coordinates of the target point; A deduplication module for identifying the same target points in the target point set based on the updated global coordinates and performing a deduplication process on the target point set based on the same target points to obtain a new target point set; An iteration module for returning to the first step and continuing to execute until an iteration termination condition is met to obtain a final target point set.

11. A scanning device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the target point positioning method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the target point positioning method according to any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the target point positioning method according to any one of claims 1-9.

Images