Three-dimensional reconstruction method and related apparatus
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-11
- Publication Date
- 2026-08-11
Smart Images
Figure CN122550798A_ABST
Abstract
Description
Technical Field
[0001] The embodiments described in this application relate to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method and related apparatus. Background Technology
[0002] In the field of 3D reconstruction, a 3D measuring device is typically used to scan the object being measured to generate 3D point cloud data, and the 3D shape of the object is then reconstructed based on this point cloud data. In some methods, a tracker can be used to track the 3D measuring device to help determine its spatial state during the scanning process. To ensure the accuracy of the 3D reconstruction results, related technologies typically require methods such as standard instrument testing, shutdown calibration, or re-inspection after obtaining abnormal measurement results to assess the tracker's accuracy.
[0003] However, there is still a problem in the related technologies of identifying whether the tracker has an accuracy abnormality during the process of generating 3D point cloud data of the object to be measured by 3D measuring equipment. Summary of the Invention
[0004] In view of this, multiple embodiments of this application aim to provide a three-dimensional reconstruction method and related apparatus, which can identify whether the tracker has an accuracy abnormality during the process of generating three-dimensional point cloud data of the measured object by a three-dimensional measuring device.
[0005] In a first aspect, one embodiment of this application provides a three-dimensional reconstruction method, comprising: generating first positional relationship characterization data based on a plurality of first marker point data; wherein the plurality of first marker point data is generated by a tracker measuring a plurality of marker points; each first marker point data corresponds to a marker point; the first positional relationship characterization data is used to characterize the relative positional relationship of at least two marker points among the plurality of marker points; during the process of the three-dimensional measuring device measuring and generating three-dimensional point cloud data of the object under test, the tracker generates second marker point data corresponding to the plurality of marker points respectively; generating second positional relationship characterization data based on the second marker point data of the plurality of marker points; wherein the second positional relationship characterization data is used to characterize the relative positional relationship of at least two marker points among the plurality of marker points; and determining that the tracker has an accuracy anomaly when the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points meet a specified anomaly identification condition; wherein the specified anomaly identification condition includes: the deviation value between the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points is greater than a specified deviation threshold.
[0006] Optionally, generating first positional relationship representation data based on multiple first marker point data includes: selecting at least two first marker point data from the first marker point data of the multiple marker points according to specified filtering conditions; wherein the specified filtering conditions include at least one of the following: the marker point corresponding to the first marker point data cannot be located on the edge of the multiple marker points; the distance between the marker points corresponding to the selected first marker point data is greater than a specified distance threshold; the marker points corresponding to the selected first marker point data are arranged along the horizontal direction or the direction of gravity.
[0007] Optionally, the relative positional relationship includes the relative distance between the marker points.
[0008] Optionally, the specified anomaly identification condition further includes: the duration for which the deviation value is greater than a specified deviation threshold, exceeding the anomaly duration threshold.
[0009] Optionally, the multiple marker points form multiple marker point pairs, and each marker point pair corresponds to the first positional relationship characterization data and the second positional relationship characterization data; the specified anomaly identification conditions include: the ratio of the number of marker point pairs with deviation values greater than a specified deviation threshold to the total number of the multiple marker point pairs, which exceeds a specified ratio threshold.
[0010] Optionally, the specified ratio threshold value is not less than 45%.
[0011] Optionally, the specified deviation threshold is not higher than 0.05 mm.
[0012] Secondly, one embodiment of this application provides a three-dimensional reconstruction apparatus, the three-dimensional reconstruction apparatus comprising: a first generation module, configured to generate first positional relationship characterization data based on a plurality of first marker point data; wherein the plurality of first marker point data is generated by a tracker measuring a plurality of marker points; each first marker point data corresponds to a marker point; the first positional relationship characterization data is used to characterize the relative positional relationship of at least two marker points among the plurality of marker points; a second generation module, configured to generate second marker point data corresponding to the plurality of marker points respectively through the tracker during the process of the three-dimensional measuring device measuring and generating three-dimensional point cloud data of the measured object; a third generation module, configured to generate second positional relationship characterization data based on the second marker point data of the plurality of marker points; wherein the second positional relationship characterization data is used to characterize the relative positional relationship of at least two marker points among the plurality of marker points; and an identification module, configured to identify that the tracker has an accuracy abnormality when the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points meet a specified abnormality identification condition; wherein the specified abnormality identification condition includes: the deviation value of the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points is greater than a specified deviation threshold.
[0013] Thirdly, one embodiment of this application also provides an electronic device, the electronic device including a memory and a processor, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the method as described above.
[0014] Fourthly, one embodiment of this application also provides a computer-readable storage medium storing at least one computer program that, when executed by a processor, can implement the method described above.
[0015] Fifthly, one embodiment of this application also provides a computer program product for implementing the method as described above.
[0016] In the various embodiments provided in this application, first positional relationship characterization data is generated by generating multiple first marker point data based on multiple marker points measured by a tracker. During the process of generating three-dimensional point cloud data of the object under test by a three-dimensional measuring device, second marker point data corresponding to multiple marker points are generated by the tracker. Second positional relationship characterization data is further generated based on the second marker point data of multiple marker points. When the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points meet the specified anomaly identification conditions, the tracker is identified as having an accuracy anomaly. This realizes the anomaly identification of the tracker's accuracy status during the process of measuring the object under test by a three-dimensional measuring device, which facilitates the operator to detect the accuracy anomaly of the tracker in a timely manner. Attached Figure Description
[0017] Figure 1 A schematic diagram of a three-dimensional reconstruction system provided in one embodiment of this application.
[0018] Figure 2 A schematic diagram of the operation of a three-dimensional reconstruction system provided in one embodiment of this application.
[0019] Figure 3 A flowchart of a three-dimensional reconstruction method provided for one embodiment of this application.
[0020] Figure 4 This is a schematic diagram of the object under test and the marker points provided as an example of an application scenario of this application.
[0021] Figure 5 A schematic diagram of a three-dimensional reconstruction system provided in one embodiment of this application.
[0022] Figure 6 A schematic diagram of an electronic device provided according to one embodiment of this application. Detailed Implementation
[0023] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0024] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0025] In related technologies, in the field of 3D reconstruction, a 3D measuring device is typically used to measure the object being measured to generate 3D point cloud data of the object, and the 3D shape of the object is reconstructed based on the 3D point cloud data. In some methods, a tracker can also be used to track the 3D measuring device to help determine the spatial state of the 3D measuring device during the scanning process.
[0026] To ensure the accuracy of 3D reconstruction results, it is usually necessary to monitor the tracker's accuracy during actual operation. Specifically, related technologies often involve methods such as standard instrument testing, shutdown calibration, or re-inspection after obtaining abnormal measurement results to determine if there are any accuracy deviations in the tracker. While these methods can detect tracker accuracy issues to some extent, they typically rely on additional testing processes, leading to a decrease in overall work efficiency.
[0027] In summary, the relevant technologies still have the problem of difficulty in timely identifying the accuracy abnormalities of the tracker during the measurement of the object by the 3D measuring equipment, which requires further improvement.
[0028] Please see Figure 1 and Figure 2 In various embodiments of this application, the 3D reconstruction system may include a variety of electronic devices. Specifically, the 3D reconstruction system may include a tracker and a 3D measurement device. Both the tracker and the 3D measurement device may integrate computing chips and memory, enabling them to possess certain data processing capabilities. In some embodiments, the electronic devices of the 3D reconstruction system may further include a host computer, which can receive data provided by the tracker and the 3D measurement device and perform data processing. The host computer may be a desktop computer, laptop computer, tablet computer, workstation, or server, etc.
[0029] 3D measurement equipment can be either optical scanning equipment or contact measurement equipment. Optical scanning equipment can include, but is not limited to, visible light scanners, structured light scanners, and laser scanners. Contact measurement equipment includes light pens. For example, a laser scanner can measure the distance to an object's surface by emitting a laser beam and detecting changes in the laser's reflection time or phase, generating high-precision scan data. Specifically, this scan data can be 3D point cloud data. A structured light scanner can project a structured light pattern (such as stripes or a dot matrix) onto an object's surface and generate scan data by detecting the light reflected from the object's surface.
[0030] This application does not specify the particular type and principle of the three-dimensional measurement equipment.
[0031] The 3D measuring device possesses tracking features, which can serve as positioning features for the device. Multiple tracking features can be deployed at various locations on the 3D measuring device, allowing for tracking of the device's spatial pose. In some embodiments, the tracking features may include, but are not limited to, marker points, coded points, 3D targets, geometric features of objects, and other features that can be acquired and recognized by the tracker. Marker points can be reflective, in which case the tracker emits light and receives the reflected light. In some embodiments, marker points can be luminescent, in which case the tracker can directly receive the light emitted by the marker points.
[0032] A tracker, based on stereo vision tracking principles, outputs tracking information corresponding to the position and orientation of a 3D measurement device in space. This tracking information can be used to determine pose information representing the spatial orientation of the 3D measurement device. Each tracker includes a camera. The tracker can have one or more cameras. Preferably, the tracker is a binocular tracker or a multi-view tracker. The number of cameras in different trackers can be the same or different. The tracker can form tracking information from images captured by the cameras. Specifically, the camera can continuously capture multiple image frames, with small time intervals between the multiple image frames, resulting in corresponding differences between the multiple image frames as the 3D measurement device moves in space. The tracking information can include frame information. Each frame information can include image frames captured by multiple cameras of the corresponding tracker at the same time. It can be understood that each frame information can include at least one image frame.
[0033] The 3D reconstruction method can be applied to the aforementioned 3D reconstruction system. Specifically, it can be applied to one or more electronic devices within the system. Those skilled in the art can deploy the electronic devices that execute the 3D reconstruction method according to the specific circumstances.
[0034] Please see Figure 3 This application provides a three-dimensional reconstruction method. This three-dimensional reconstruction method is applied to an electronic device within a three-dimensional reconstruction system. The three-dimensional reconstruction method may include the following steps.
[0035] Step S110: Generate first positional relationship representation data based on multiple first marker point data; wherein, the multiple first marker point data is generated by the tracker measuring multiple marker points; each first marker point data corresponds to one marker point; the first positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the multiple marker points.
[0036] Step S120: During the process of generating three-dimensional point cloud data of the object under test by the three-dimensional measuring device, the tracker generates second marker point data corresponding to the multiple marker points respectively.
[0037] Step S130: Based on the second marker point data of the plurality of marker points, generate second positional relationship representation data; wherein, the second positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the plurality of marker points.
[0038] Step S140: If the first positional relationship representation data and the second positional relationship representation data between at least two marker points meet the specified anomaly identification conditions, the tracker is identified as having an accuracy anomaly; wherein the specified anomaly identification conditions include: the deviation value between the first positional relationship representation data and the second positional relationship representation data between at least two marker points is greater than a specified deviation threshold.
[0039] In this embodiment, the 3D reconstruction method can be applied to electronic devices in a 3D reconstruction system. The electronic device can be any one or more of a tracker, a measuring device, or a host computer. The electronic device can acquire marker point data generated by the tracker at different stages and determine the accuracy status of the tracker based on the marker point data, thereby identifying whether the tracker has experienced accuracy anomalies during the process of the 3D measuring device measuring and generating the 3D point cloud data of the object being measured.
[0040] In this embodiment, the electronic device can generate first positional relationship characterization data based on multiple first marker point data. The multiple first marker point data are data generated by the tracker measuring multiple marker points, with each first marker point data corresponding to one marker point. The multiple marker points can be multiple reference feature points arranged in the environment outside the object under test. For example, the multiple marker points can be arranged in a background area to provide positional relationship reference during the scanning of the object under test. The first marker point data can be used to characterize the spatial position of each marker point in a reference state. Specifically, the first marker point data can include spatial coordinate data determined by the electronic device based on the measurement results of the tracker. The first positional relationship characterization data can be used to characterize the spatial relative positional relationship of at least two of the multiple marker points. Specifically, the first positional relationship characterization data can include geometric relationship data between at least two marker points, such as relative distance or other data capable of characterizing relative positional relationships. In some embodiments, the multiple first marker point data can be data generated by the tracker measuring multiple marker points after calibration and in a normal reference accuracy state. Thus, the electronic device can establish a reference positional relationship between multiple marker points based on the multiple first marker point data.
[0041] In this embodiment, the electronic device can also generate second marker point data corresponding to multiple marker points during the process of the 3D measuring device measuring and generating the 3D point cloud data of the object under test. This can be understood as the second marker point data representing the measurement results of the tracker on the spatial positions corresponding to multiple marker points during the actual scanning of the object under test by the 3D measuring device. Specifically, while tracking the 3D measuring device and cooperating with it to acquire the scan data of the object under test and generate the 3D point cloud data of the object under test, the tracker can also continuously or intermittently measure multiple marker points to obtain the second marker point data corresponding to each marker point.
[0042] In this embodiment, the electronic device can generate second positional relationship representation data based on second marker point data from multiple marker points. This second positional relationship representation data can be used to characterize the spatial relative positional relationship between at least two marker points among the multiple marker points during the scanning of the object by the 3D measuring device. Specifically, the electronic device can parse the multiple second marker point data to determine the spatial relative positional relationship between at least two marker points and generate the second positional relationship representation data accordingly. Thus, the first positional relationship representation data can characterize the spatial relative positional relationship of the marker points in a reference state, and the second positional relationship representation data can characterize the spatial relative positional relationship of the marker points during the scanning of the object. The electronic device can further identify the accuracy status of the tracker based on the difference between the two.
[0043] In this embodiment, the electronic device can determine that the tracker has experienced an accuracy anomaly if the first positional relationship representation data and the second positional relationship representation data between at least two marker points meet a specified anomaly identification condition. The specified anomaly identification condition can be a set of criteria used to determine whether the tracker has experienced an accuracy anomaly. Specifically, the specified anomaly identification condition includes: the deviation value between the first positional relationship representation data and the second positional relationship representation data between at least two marker points is greater than a specified deviation threshold. The deviation value can be understood as the difference between the relative positional relationship in the reference state and the relative positional relationship during the scanning of the object under test. The electronic device can determine the corresponding deviation value based on the first positional relationship representation data and the second positional relationship representation data, and determine whether the deviation value is greater than the specified deviation threshold; if the deviation value is greater than the specified deviation threshold, the electronic device can determine that the tracker has experienced an accuracy anomaly. In this way, the electronic device can utilize the changes in the relative positional relationship between multiple marker points to identify anomalies in the accuracy status of the tracker during actual operation, facilitating timely detection of accuracy anomalies by operators.
[0044] In several embodiments provided in this application, first positional relationship characterization data is generated by generating multiple first marker point data based on multiple marker points measured by a tracker. During the process of generating three-dimensional point cloud data of the object under test by a three-dimensional measuring device, second marker point data corresponding to multiple marker points are generated by the tracker. Second positional relationship characterization data is further generated based on the second marker point data of multiple marker points. When the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points meet the specified anomaly identification conditions, the tracker is identified as having an accuracy anomaly. This enables the anomaly identification of the tracker's accuracy status during the scanning of the object under test by the three-dimensional measuring device, making it easier for operators to promptly detect accuracy anomalies in the tracker.
[0045] In some implementations, the electronic device can filter at least two first marker point data from the first marker point data of the plurality of marker points according to specified filtering conditions; wherein the specified filtering conditions include at least one of the following: the marker point corresponding to the first marker point data cannot be located at the edge of the plurality of marker points; the distance between the marker points corresponding to the selected first marker point data is greater than a specified distance threshold; the marker points corresponding to the selected first marker point data are arranged along the horizontal direction or the direction of gravity.
[0046] In this embodiment, when the electronic device generates first positional relationship representation data based on multiple first marker point data, it can first filter at least two first marker point data from the multiple marker point data according to specified filtering conditions, and then generate the first positional relationship representation data based on the filtered at least two first marker point data. The specified filtering conditions can be understood as the filtering conditions set by the electronic device for the spatial distribution characteristics satisfied by the marker points corresponding to the first marker point data in order to select marker points more suitable for representing the reference positional relationship from multiple marker points. In this way, the electronic device does not need to directly generate the first positional relationship representation data based on all the first marker point data, thereby reducing the interference of unsuitable marker points on subsequent accuracy anomaly identification. Of course, in other embodiments, the first positional relationship representation data can also be generated based on all the first marker point data.
[0047] In this embodiment, the specified filtering conditions may include at least one of the following: the marker point corresponding to the first marker point data cannot be located at the edge of the plurality of marker points; the distance between the marker points corresponding to the selected first marker point data is greater than a specified distance threshold; the marker points corresponding to the selected first marker point data are arranged along the horizontal direction or the direction of gravity. Here, a marker point located at the edge of the plurality of marker points can be understood as being located outside the overall distribution area of the plurality of marker points. Marker points located at the edge are more susceptible to the effects of decreased imaging quality at the edge of the field of view, local occlusion, or attitude changes during actual measurement, resulting in poor stability of the corresponding first marker point data. Therefore, the electronic device can preferentially exclude first marker point data corresponding to edge marker points and filter out at least two first marker point data from the remaining first marker point data.
[0048] In this embodiment, the electronic device can also filter data based on the distance between the selected first marker point data. The specified distance threshold can be understood as a threshold used to constrain the minimum interval between the selected marker points. If the distance between two marker points is too small, the corresponding change in relative positional relationship may not be numerically significant, which is detrimental to subsequent identification of whether the tracker has experienced accuracy anomalies. Therefore, the electronic device can select first marker point data corresponding to marker points with a distance greater than the specified distance threshold, so as to form the first positional relationship representation data based on the larger intervals between marker points. This allows for better distinguishability of the deviation values subsequently determined by the first and second positional relationship representation data.
[0049] In this embodiment, the electronic device can also filter based on the arrangement direction of the marker points corresponding to the selected first marker point data. The horizontal direction or the gravity direction can be understood as the preferred reference direction in the space where the 3D reconstruction system is located. Specifically, marker points arranged along the horizontal direction can be used to characterize the lateral positional relationship; marker points arranged along the gravity direction can be used to characterize the vertical positional relationship. This helps the electronic device to accurately determine the accuracy status of the tracker in combination with the actual application scenario.
[0050] In some implementations, the electronic device can filter and obtain at least two first marker point data based on only one of the specified filtering conditions. For example, when multiple marker points are arranged on a background board, the electronic device can first exclude marker points located in the perimeter area of the background board, and then select the corresponding first marker point data from the remaining marker points to generate the first positional relationship representation data. In other implementations, the electronic device can also filter and obtain at least two first marker point data based on multiple specified filtering conditions simultaneously. For example, when multiple marker points are distributed in the background area around the object being measured, the electronic device can first exclude marker points located at the edges of the multiple marker points, and then filter the first marker point data corresponding to at least two marker points that are more than a specified distance threshold apart and arranged horizontally from the remaining marker points, and further generate the first positional relationship representation data.
[0051] In some implementations, the relative positional relationship includes the relative distance between marker points.
[0052] In this embodiment, the first positional relationship characterization data and the second positional relationship characterization data can be used to characterize the relative positional relationship using the relative distance between marker points. The relative distance can be understood as the spatial interval data between at least two marker points. Specifically, the electronic device can determine the relative distance between the at least two marker points in a reference state based on the first marker point data corresponding to the at least two marker points, and use this relative distance as the first positional relationship characterization data; the electronic device can also determine the relative distance between the at least two marker points during the scanning of the object under test based on the second marker point data corresponding to the at least two marker points, and use this relative distance as the second positional relationship characterization data. Thus, the relative distance calculation method is relatively direct and facilitates the electronic device in comparing the relative positional relationship between the reference state and the actual working state.
[0053] Furthermore, since multiple marker points remain fixed in the environment, if the tracker is in a normal accuracy state, the relative distance between the same marker points should remain basically consistent at different stages; if the tracker experiences an accuracy anomaly, the measured relative distance between the corresponding marker points may show a significant deviation. In this way, the electronic device can identify whether the tracker has experienced an accuracy anomaly based on the change in the relative distance between the marker points.
[0054] In some implementations, the specified anomaly identification condition further includes: the duration for which the deviation value is greater than a specified deviation threshold, exceeding the anomaly duration threshold.
[0055] In this embodiment, when the electronic device identifies whether the tracker has experienced an accuracy anomaly based on first positional relationship characterization data and second positional relationship characterization data between at least two marker points, it can further combine this with the duration for which the deviation value is greater than a specified deviation threshold for judgment. Specifically, the specified anomaly identification condition may further include: the duration for which the deviation value is greater than the specified deviation threshold exceeds an anomaly duration threshold. The duration can be understood as the length of time during which the electronic device detects the deviation value as continuously greater than the specified deviation threshold. The anomaly duration threshold can be understood as a time threshold used to distinguish between instantaneous fluctuations and true accuracy anomalies. In this way, the electronic device can avoid directly identifying an accuracy anomaly in the tracker due to short-term fluctuations, thereby improving the stability of accuracy anomaly identification.
[0056] Specifically, during the process of generating 3D point cloud data of the measured object using a 3D measuring device, the electronic device can continuously or intermittently generate second positional relationship representation data, and determine the corresponding deviation value based on the first and second positional relationship representation data. When the electronic device determines that the deviation value is greater than a specified deviation threshold, it can start timing; in subsequent moments, if the deviation value is still greater than the specified deviation threshold, the corresponding duration continues to accumulate; if the deviation value falls below the specified deviation threshold, the current duration statistics can be terminated, or the already counted duration can be cleared. The electronic device only considers the tracker to have an accuracy abnormality when it determines that the duration exceeds an abnormal duration threshold. In this way, short-term deviations caused by partial occlusion, instantaneous vibration, short-term environmental interference, or single measurement noise can be avoided from being misjudged as accuracy abnormalities.
[0057] In some implementations, the electronic device can continuously determine the duration based on the deviation values corresponding to multiple sampling times. For example, the electronic device generates second positional relationship representation data every preset time interval and determines a corresponding deviation value. If the deviation values corresponding to multiple consecutive sampling times are all greater than a specified deviation threshold, the electronic device can accumulate the corresponding times between adjacent sampling times as the duration. In other implementations, the electronic device can also perform real-time statistics on the duration based on a continuous data stream. Those skilled in the art can flexibly set the statistical method for the duration according to the tracker's data sampling frequency and accuracy anomaly identification requirements.
[0058] In some implementations, the plurality of marker points form a plurality of marker point pairs, each marker point pair corresponding to the first positional relationship characterization data and the second positional relationship characterization data; the specified anomaly identification condition includes: the ratio of the number of marker point pairs with deviation values greater than a specified deviation threshold to the total number of the plurality of marker point pairs, which exceeds a specified ratio threshold.
[0059] In this embodiment, when the electronic device identifies whether the tracker has experienced an accuracy anomaly based on the first positional relationship representation data and the second positional relationship representation data between at least two marker points, it can further perform statistical judgment on the deviations corresponding to multiple marker point pairs. Specifically, the multiple marker points can form multiple marker point pairs, and each marker point pair corresponds to the first positional relationship representation data and the second positional relationship representation data. The specified anomaly identification condition may include: the ratio of the number of marker point pairs with deviation values greater than a specified deviation threshold to the total number of the multiple marker point pairs, exceeding a specified ratio threshold. A marker point pair can be a point pair unit composed of any two marker points from the multiple marker points. In this way, the electronic device can not only determine whether the tracker has experienced an accuracy anomaly based on the deviation situation corresponding to a single marker point pair, but also make a judgment based on the overall deviation distribution of multiple marker point pairs, thereby improving the representativeness and reliability of the accuracy anomaly identification result.
[0060] In this embodiment, the electronic device can first form multiple marker point pairs based on the multiple marker points. For each marker point pair, the electronic device can determine the first positional relationship characterization data corresponding to the marker point pair in the reference state, and the second positional relationship characterization data corresponding to the marker point pair during the scanning of the object under test. In this way, the electronic device can determine the corresponding deviation values for multiple marker point pairs respectively.
[0061] After determining the deviation values corresponding to multiple marker point pairs, the electronic device can count the number of marker point pairs whose deviation values exceed a specified deviation threshold, and further determine the ratio between this number and the total number of marker point pairs. The specified ratio threshold can be a threshold used to determine whether the proportion of abnormal marker point pairs in all marker point pairs has reached the requirement for accuracy anomaly judgment. If the proportion exceeds the specified ratio threshold, the electronic device can determine that the tracker has an accuracy anomaly; if the proportion does not exceed the specified ratio threshold, the electronic device can temporarily not determine that the tracker has an accuracy anomaly. In this way, the electronic device can use the deviation distribution corresponding to multiple marker point pairs to judge the overall accuracy status of the tracker, thereby reducing the interference of local marker point anomalies, partial occlusion, or individual marker point recognition errors on the accuracy anomaly recognition results.
[0062] In some implementations, the specified percentage threshold is not less than 45%.
[0063] In this embodiment, when the electronic device counts the percentage of marker pairs with deviation values exceeding a specified deviation threshold, it requires at least 45% of these pairs to determine that the tracker has experienced an accuracy anomaly. This setting ensures that the determination of accuracy anomaly is based on a large range of abnormal changes across multiple marker pairs, thus avoiding the assumption of an accuracy anomaly when only a small number of marker pairs show deviations. Furthermore, the specified percentage threshold can be 45%, 50%, 55%, 60%, or higher. For example, when multiple markers form 20 marker pairs, if the specified percentage threshold is 45%, at least 9 marker pairs must have deviation values exceeding the specified deviation threshold for the electronic device to determine that the tracker has experienced an accuracy anomaly; if the specified percentage threshold is 50%, at least 10 marker pairs must meet the aforementioned condition; and if the specified percentage threshold is 60%, at least 12 marker pairs must meet the aforementioned condition. Therefore, as the specified percentage threshold increases, the electronic device's requirements for the overall degree of anomaly distribution also increase, thereby adapting to different requirements for false alarm rate and sensitivity in different accuracy monitoring scenarios.
[0064] In some implementations, the specified deviation threshold is no higher than 0.05 mm.
[0065] In this embodiment, the electronic device can control the difference between the first positional relationship characterization data and the second positional relationship characterization data between at least two marker points to be within a small order of magnitude for judgment. For example, the specified deviation threshold can be 0.05mm, 0.04mm, 0.03mm, 0.02mm, or even smaller values. For instance, when the specified deviation threshold is 0.05mm, the electronic device can identify marker point pairs with a deviation value greater than 0.05mm as abnormal marker point pairs; when the specified deviation threshold is 0.03mm, the electronic device can identify marker point pairs with a deviation value greater than 0.03mm as abnormal marker point pairs; and when the specified deviation threshold is 0.02mm, it can identify even smaller changes in relative positional relationships. Thus, the electronic device can flexibly set the specified deviation threshold according to the tracker's measurement accuracy, application scenario, and requirements for the timing of accuracy anomaly warnings.
[0066] One embodiment of this application provides an example of an application scenario for a 3D reconstruction method. An electronic device can be applied to a 3D reconstruction system. During the process of generating 3D point cloud data of the object being measured by a 3D measuring device, the accuracy status of the tracker is identified based on changes in the positional relationships of multiple marker points in the background area. If specified anomaly identification conditions are met, the tracker is identified as having an accuracy anomaly. In this scenario example, taking a box as the object being measured, specifically, the 3D measuring device can be a handheld scanner, the tracker can be a binocular tracker used to track the spatial pose of the 3D measuring device, and the electronic device can be a host computer communicatively connected to the tracker and the 3D measuring device. Thus, the electronic device can perform online accuracy monitoring of the tracker without relying on a standard and without significantly increasing additional measurement time.
[0067] For example, referring to Figure 4, before the formal scanning of the enclosure, multiple marker points, designated A1 to A10, are pre-arranged in the background area. These marker points provide a positional reference when the tracker is in a reference state. After calibration, the tracker can measure these marker points in the background area to generate multiple first marker point data. Each first marker point data corresponds to a marker point and represents the position data of that marker point in the reference state. The electronic device can receive the multiple first marker point data and generate first positional relationship representation data based on it. The first positional relationship representation data represents the relative positional relationship between at least two marker points. In this scenario example, the relative positional relationship includes the relative distance between the marker points.
[0068] In this scenario example, the electronic device can first determine that marker points A1, A2, A3, A4, A5, and A6 are located on the outer edge or edge of the overall distribution of multiple marker points. Therefore, the first marker point data corresponding to these marker points is not used as the basis for generating the first positional relationship representation data subsequently. Then, the electronic device can select the first marker point data for accuracy state recognition from the first marker point data corresponding to the remaining marker points A7, A8, A9, and A10. Marker point A7 is located to the left of marker point A8, marker point A10 is located below marker point A8, and marker point A9 is located to the right of marker point A8. Therefore, marker points A7 and A8 are roughly arranged horizontally, marker points A8 and A10 are roughly arranged along the direction of gravity, and marker points A8 and A9 are roughly arranged horizontally, and the distance between these marker points is greater than a specified distance threshold. For example, the specified distance threshold can be 50 mm. After filtering and obtaining the first marker point data corresponding to marker points A7, A8, A9 and A10, the electronic device can select three marker point pairs as marker point pairs for generating the first positional relationship representation data.
[0069] In this scenario example, the three marker point pairs are marker point pair A7-A8, marker point pair A8-A10, and marker point pair A8-A9. The electronic device can determine the relative distances corresponding to the above three marker point pairs and use these relative distances as part of the first positional relationship characterization data. For example, the electronic device can determine that the relative distance corresponding to marker point pair A7-A8 is 62.00 mm, the relative distance corresponding to marker point pair A8-A10 is 74.00 mm, and the relative distance corresponding to marker point pair A8-A9 is 158.00 mm. In this way, the electronic device can establish the relative positional relationship of the three marker point pairs in the reference state, and subsequently, without the need for separate measurement of the standard, the first positional relationship characterization data can be used as a reference for online accuracy monitoring.
[0070] Subsequently, the 3D measuring device begins a formal scan of the enclosure to generate 3D point cloud data of the object under test. During the process of generating the 3D point cloud data, the electronic device can also continuously or intermittently generate second marker point data corresponding to multiple marker points via the tracker. This second marker point data can be understood as representing the measurement results of the tracker on the current positions of multiple marker points during the actual scanning of the enclosure. Furthermore, the electronic device can generate second positional relationship representation data based on the second marker point data of the multiple marker points. This second positional relationship representation data is also used to represent the relative positional relationship between at least two of the multiple marker points.
[0071] In this scenario example, while the 3D measuring device continuously outputs scanning data to the box, the electronic device updates the second marker data corresponding to marker points A7, A8, A9, and A10 every 0.2 seconds, and determines the relative distances of marker pair A7-A8, marker pair A8-A10, and marker pair A8-A9 at the current moment to form second positional relationship representation data. In this way, the electronic device can track the accuracy status of the tracker in real time without pausing the scanning task.
[0072] For example, in the first 0.8 seconds after the scan begins, the recognition results of some marker points experience momentary disturbances due to the operator briefly passing through the background area. At this time, the electronic device can determine that the second positional relationship representation data for marker points A7-A8 is 62.06 mm, for marker points A8-A10 it is 74.03 mm, and for marker points A8-A9 it is 158.02 mm. Correspondingly, the electronic device can determine that the deviation value for marker points A7-A8 is 0.06 mm, for marker points A8-A10 it is 0.03 mm, and for marker points A8-A9 it is 0.02 mm. Since the specified deviation threshold can be set to 0.05 mm in this scenario example, only the deviation value for marker points A7-A8 is greater than the specified deviation threshold. However, within this time period, the deviation value exceeding the specified deviation threshold only lasted for 0.8 seconds, and the number of marker pairs with a deviation value greater than 0.05 mm was only 1, accounting for 33.3% of the total three marker pairs. In this scenario example, the abnormal duration threshold can be set to 1.5 seconds, and the specified proportion threshold can be set to 45%. Since 0.8 seconds does not exceed the abnormal duration threshold of 1.5 seconds, and 33.3% does not exceed the specified proportion threshold of 45%, the electronic device can temporarily not determine that the tracker has experienced an accuracy anomaly. In this way, the electronic device can avoid misjudging the tracker's accuracy anomaly due to short-term occlusion or instantaneous disturbances.
[0073] As the scanning continued, the tracker's mounting structure loosened due to continuous vibration, causing the tracker's measurement results for multiple marker points to continuously change. Even so, the electronic device could still continuously update the second positional relationship representation data while the 3D measuring equipment was generating the 3D point cloud data of the object under test. For example, at several consecutive sampling times, the electronic device could determine that the second positional relationship representation data for marker point pair A7-A8 was 62.07 mm, for pair A8-A10 it was 74.08 mm, and for pair A8-A9 it was 158.03 mm. The electronic device further compared the corresponding second positional relationship representation data with the first positional relationship representation data, determining that the deviation value for marker point pair A7-A8 was 0.07 mm, for pair A8-A10 it was 0.08 mm, and for pair A8-A9 it was 0.03 mm.
[0074] Furthermore, in this scenario example, the electronic device can continuously count the duration of the state where the deviation value corresponding to the above three marker point pairs is greater than 0.05mm. If the electronic device determines that the abnormal state corresponding to marker point pairs A7-A8 and A8-A10 lasts continuously for 2.3 seconds, then since 2.3 seconds exceeds the abnormal duration threshold of 1.5 seconds, the duration condition is satisfied. Simultaneously, the electronic device can also count the ratio of the number of marker point pairs with deviation values greater than a specified deviation threshold to the total number of multiple marker point pairs. In this scenario example, the number of marker point pairs with deviation values greater than the specified deviation threshold is 2, and the total number of the three marker point pairs is 3, therefore, the corresponding ratio is 66.7%. Since 66.7% exceeds the specified ratio threshold of 45%, the ratio condition is also satisfied.
[0075] In the above scenario, the electronic device can determine that the tracker has experienced an accuracy anomaly by identifying that the second positional relationship representation data generated based on the second marker point data of multiple marker points meets the specified anomaly identification conditions with the first positional relationship representation data generated based on the first marker point data. Specifically, the electronic device can output a prompt message "The tracker has experienced an accuracy anomaly; recalibration is recommended" on the display interface to indicate to the operator that the current accuracy status of the tracker has changed significantly compared to the reference state. Furthermore, the electronic device can also indicate that the currently generated 3D point cloud data may be affected by the tracker's accuracy anomaly, thereby prompting the operator to pause the current scanning task and continue the 3D reconstruction of the box after recalibrating the tracker.
[0076] For example, in another scanning scenario, if the deviation of only one of the three marker point pairs is consistently greater than 0.05 mm, the corresponding proportion is 33.3%. Even if this 33.3% abnormal state lasts for more than 1.5 seconds, the electronic device can temporarily not consider the tracker to have an accuracy anomaly because 33.3% still does not reach the specified proportion threshold of 45%. As another example, in a high-precision detection scenario, the electronic device can still set the specified proportion threshold to be no less than 45% and the specified deviation threshold to 0.03 mm or 0.02 mm to identify changes in the tracker's accuracy state earlier. In this way, the electronic device can flexibly perform accuracy anomaly identification according to the tracking accuracy requirements of different 3D reconstruction tasks.
[0077] In this scenario example, the electronic device first generates first positional relationship representation data based on multiple first marker point data. Then, during the process of generating 3D point cloud data of the measured object using a 3D measuring device, the tracker continuously generates second marker point data corresponding to multiple marker points. Based on the second marker point data of these multiple marker points, second positional relationship representation data is generated. Subsequently, by combining the deviation value, duration, and the proportion of marker point pairs with deviation values exceeding a specified deviation threshold, the device identifies whether the tracker has experienced accuracy anomalies. In this way, the electronic device can utilize multiple marker points in the background area to perform online accuracy monitoring of the tracker in a low-cost manner. Without relying on a standard instrument or significantly increasing additional measurement time, it can promptly detect accuracy anomalies in the tracker during the scanning of the measured object and prompt the operator to recalibrate.
[0078] Please see Figure 5 This application provides a three-dimensional reconstruction device. The three-dimensional reconstruction device includes: a first generation module, a second generation module, a third generation module, and an identification module.
[0079] The first generation module is used to generate first positional relationship representation data based on multiple first marker point data; wherein, the multiple first marker point data is generated by the tracker measuring multiple marker points; each first marker point data corresponds to one marker point; the first positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the multiple marker points.
[0080] The second generation module is used to generate second marker point data corresponding to the multiple marker points respectively through the tracker during the process of generating three-dimensional point cloud data of the measured object by the three-dimensional measuring equipment.
[0081] The third generation module is used to generate second positional relationship representation data based on the second marker point data of the plurality of marker points; wherein the second positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the plurality of marker points.
[0082] The identification module is used to identify that the tracker has an accuracy abnormality when the first positional relationship representation data and the second positional relationship representation data between at least two marker points meet a specified anomaly identification condition; wherein the specified anomaly identification condition includes: the deviation value between the first positional relationship representation data and the second positional relationship representation data between at least two marker points is greater than a specified deviation threshold.
[0083] In this embodiment, the specific functions and effects achieved by the three-dimensional reconstruction device can be explained by referring to other embodiments of this application, and will not be repeated here.
[0084] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method as described above.
[0085] This application also provides a computer program product containing instructions that, when executed by a processor, implement the method as described above.
[0086] Please see Figure 6 Embodiments of this application may provide an electronic device, the electronic device comprising: a memory, and one or more processors communicatively connected to the memory; the memory storing instructions executable by the one or more processors, the instructions being executed by the one or more processors to cause the one or more processors to implement the method as described above.
[0087] In some embodiments, the electronic device may include a processor, a storage medium, and a communication interface connected to a system bus. The storage medium may store related computer programs.
[0088] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments of this application, and are not intended to limit the scope of the invention.
[0089] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0090] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.
[0091] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.
[0092] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0093] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.
[0096] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0098] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0100] The above description is merely a specific embodiment of this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this invention should be determined by the scope of the claims.
Claims
1. A three-dimensional reconstruction method, characterized by, include: First positional relationship representation data is generated based on multiple first marker point data; wherein, the multiple first marker point data is generated by the tracker measuring multiple marker points; each first marker point data corresponds to one marker point; the first positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the multiple marker points; During the process of generating three-dimensional point cloud data of the object under test by a three-dimensional measuring device, the tracker generates second marker point data corresponding to the multiple marker points respectively; Based on the second marker point data of the plurality of marker points, second positional relationship representation data is generated; wherein, the second positional relationship representation data is used to represent the relative positional relationship of at least two marker points among the plurality of marker points; If the first positional relationship representation data and the second positional relationship representation data between at least two marker points meet a specified anomaly identification condition, the tracker is identified as having an accuracy anomaly; wherein the specified anomaly identification condition includes: the deviation value between the first positional relationship representation data and the second positional relationship representation data between at least two marker points is greater than a specified deviation threshold.
2. The method of claim 1, wherein, First positional relationship representation data is generated based on multiple first marker point data, including: From the first marker data of the plurality of marker points, at least two first marker data are obtained by filtering according to specified filtering conditions; wherein, the specified filtering conditions include at least one of the following: the marker point corresponding to the first marker data cannot be located on the edge of the plurality of marker points; the distance between the marker points corresponding to the selected first marker data is greater than a specified distance threshold; the marker points corresponding to the selected first marker data are arranged along the horizontal direction or the direction of gravity.
3. The method of claim 1, wherein, The relative positional relationship includes the relative distance between the marker points.
4. The method of claim 1, wherein, The specified anomaly identification conditions also include: the duration for which the deviation value is greater than a specified deviation threshold, exceeding the anomaly duration threshold.
5. The method of claim 1, wherein, The multiple marker points form multiple marker point pairs, and each marker point pair corresponds to the first positional relationship representation data and the second positional relationship representation data; The specified anomaly identification conditions include: the ratio of the number of marker pairs with deviation values greater than a specified deviation threshold to the total number of the multiple marker pairs, which exceeds a specified ratio threshold.
6. The method of claim 5, wherein, The specified percentage threshold value shall not be less than 45%.
7. The method of claim 1, wherein, The specified deviation threshold is no higher than 0.05 mm.
8. An electronic device, comprising: The electronic device includes a memory and a processor, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the method as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, is capable of implementing the method as described in any one of claims 1 to 7.
10. A computer program product, characterised in that, The computer program product is used to implement the method as described in any one of claims 1 to 7.