Three-dimensional reconstruction method and related apparatus

Abstract

This application provides a three-dimensional reconstruction method and related apparatus. The method includes: acquiring environmental disturbance information during the scanning of point cloud data of a test object; wherein the environmental disturbance information includes the difference between the reconstructed position information of a specified reference object and the baseline position information of the specified reference object; generating a three-dimensional reconstruction model of the test object based on the point cloud data of the test object; and displaying the three-dimensional reconstruction model; wherein at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data. The three-dimensional reconstruction method and related apparatus facilitate operators' understanding of the accuracy of the three-dimensional reconstruction model.

Description

Technical Field

[0001] The embodiments described in this application relate to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method and related apparatus. Background Technology

[0002] In the field of 3D reconstruction, point cloud data can usually be obtained by scanning the object under test, and a 3D reconstruction model of the object under test can be generated based on the point cloud data to realize the 3D shape reconstruction of the object under test.

[0003] However, in existing technologies, it is difficult for operators to quickly identify when point cloud data is interfered with by environmental factors. Summary of the Invention

[0004] In view of this, this application provides a three-dimensional reconstruction method and related apparatus, which can facilitate staff to understand the accuracy of the three-dimensional reconstruction model to a certain extent.

[0005] In a first aspect, one embodiment of this application provides a three-dimensional reconstruction method, comprising: acquiring environmental disturbance information during the scanning of point cloud data of a test object; wherein the environmental disturbance information includes the difference between the reconstructed position information of a specified reference object and the baseline position information of the specified reference object; generating a three-dimensional reconstruction model of the test object based on the point cloud data of the test object; and displaying the three-dimensional reconstruction model; wherein at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data.

[0006] Optionally, the designated reference point includes a fixed marker point set in the environment where the object under test is located.

[0007] Optionally, obtaining environmental disturbance information during the scanning of point cloud data of the object under test includes: acquiring point cloud data of the object under test and reconstructed position information of the fixed marker points; comparing the reconstructed position information corresponding to the point cloud data with the reference position information of the fixed marker points to obtain environmental disturbance information corresponding to the point cloud data.

[0008] Optionally, the method further includes: during the scanning of the point cloud data of the object under test, if it is determined that the environmental disturbance information exceeds a specified disturbance threshold, outputting a first prompt message; wherein the first prompt message is used to remind the user that there is environmental disturbance.

[0009] Optionally, the method further includes: upon receiving a pause command, pausing the generation of point cloud data while continuously generating environmental disturbance information; and outputting a second prompt message when the environmental disturbance information is lower than or equal to a specified disturbance threshold; wherein the second prompt message indicates that the object under test can be scanned normally to generate point cloud data.

[0010] Optionally, displaying the three-dimensional reconstructed model includes: displaying the three-dimensional reconstructed model with different visual attributes based on the environmental disturbance information.

[0011] Optionally, the visual attributes include at least one of color, transparency, texture, or highlighting.

[0012] Optionally, acquiring environmental disturbance information during the scanning of point cloud data of the object under test includes: acquiring environmental disturbance information generated by a tracker that tracks the measuring device during the process of generating point cloud data by scanning the object under test with the measuring device.

[0013] Secondly, one embodiment of this application provides a three-dimensional reconstruction apparatus, the three-dimensional reconstruction apparatus comprising: an acquisition module, configured to acquire environmental disturbance information during the scanning of point cloud data of a test object; wherein the environmental disturbance information includes the difference between the reconstructed position information of a specified reference object and the baseline position information of the specified reference object; a generation module, configured to generate a three-dimensional reconstruction model of the test object based on the point cloud data of the test object; and a display module, configured to display the three-dimensional reconstruction model; wherein at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data.

[0014] Thirdly, one embodiment of this application also provides an electronic device, the electronic device including a memory and a processor, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the method as described above.

[0015] Fourthly, one embodiment of this application also provides a computer-readable storage medium storing at least one computer program that, when executed by a processor, can implement the method described above.

[0016] Fifthly, one embodiment of this application also provides a computer program product for implementing the method as described above.

[0017] The embodiments provided in this application acquire environmental disturbance information during the scanning of point cloud data of the object under test. The environmental disturbance information includes the difference between the reconstructed position information of a specified reference object and the baseline position information of the specified reference object. A three-dimensional reconstruction model of the object under test is generated based on the point cloud data of the object under test. When displaying the three-dimensional reconstruction model, at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data. This realizes that the influence of environmental factors on the three-dimensional reconstruction results during the scanning process is intuitively reflected in the three-dimensional reconstruction model, making it easier for staff to understand the accuracy of the three-dimensional reconstruction model. Attached Figure Description

[0018] Figure 1 A schematic diagram of a three-dimensional reconstruction system provided in one embodiment of this application.

[0019] Figure 2 A schematic diagram of the operation of a three-dimensional reconstruction system provided in one embodiment of this application.

[0020] Figure 3 A flowchart of a three-dimensional reconstruction method provided for one embodiment of this application.

[0021] Figure 4 A schematic diagram of a three-dimensional reconstruction apparatus provided in one embodiment of this application.

[0022] Figure 5 A schematic diagram of an electronic device provided according to one embodiment of this application. Detailed Implementation

[0023] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0024] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] In the field of 3D reconstruction, point cloud data can usually be obtained by scanning the object under test, and a 3D reconstruction model of the object under test can be generated based on the point cloud data to achieve 3D shape reconstruction of the object under test.

[0026] However, in actual scanning processes, environmental factors such as vibration and airflow changes can easily affect the point cloud data acquired at certain times, causing deviations. Especially in continuous scanning scenarios, the degree to which point cloud data acquired at different times is affected by environmental factors may vary, resulting in differences in accuracy between different regions in the 3D reconstruction results.

[0027] In related technologies, typically only the final generated 3D reconstructed model is displayed, or the operator relies solely on their scanning experience to determine whether the scanning process was affected by environmental factors. As a result, operators often find it difficult to quickly identify instances of environmental interference with point cloud data from the displayed results.

[0028] It is evident that the relevant technologies still have the problem of operators finding it difficult to quickly identify interference from environmental factors in point cloud data, and improvements are necessary.

[0029] Please see Figure 1 and Figure 2 In various embodiments of this application, the 3D reconstruction system may include a variety of electronic devices. Specifically, the 3D reconstruction system may include a tracker and a measuring device. Both the tracker and the measuring device may integrate computing chips and memory, enabling them to possess certain data processing capabilities. In some embodiments, the electronic devices of the 3D reconstruction system may further include a host computer, which can receive data provided by the tracker and the measuring device and perform data processing. The host computer may be a desktop computer, laptop computer, tablet computer, workstation, or server, etc. In some embodiments, the 3D reconstruction system may not include a tracker, and the measuring device may be a handheld scanner.

[0030] Measurement equipment can include, but is not limited to, optical measurement equipment. Optical measurement equipment can include, but is not limited to, structured light scanners, laser scanners, and light pens. For example, a laser scanner can measure the distance to an object's surface by emitting a laser beam and detecting changes in the laser's reflection time or phase, thus generating high-precision scan data. Specifically, this scan data can be three-dimensional point cloud data. A structured light scanner can project a structured light pattern (such as stripes or a dot matrix) onto an object's surface and generate scan data by detecting the light reflected from the object's surface.

[0031] This application does not specify the particular type and principle of the measuring equipment.

[0032] The measuring device possesses tracking features, which can serve as its positioning characteristics. Multiple tracking features can be present and deployed at various locations on the measuring device, allowing for tracking of the device's spatial pose. In some embodiments, the tracking features may include, but are not limited to, marker points, coded points, stereo targets, geometric features of objects, and other features that can be acquired and identified by the tracker. The marker points can be reflective, in which case the tracker emits light and receives the reflected light. In some embodiments, the marker points can be luminescent, in which case the tracker can directly receive the light emitted by the marker points.

[0033] A tracker can be used based on stereo vision tracking principles to output tracking information corresponding to the position and orientation of a measuring device in space. This tracking information can be used to determine pose information representing the spatial orientation of the measuring device. Each tracker includes a camera. The number of cameras in a tracker can be one or more. Preferably, the tracker is a binocular tracker or a multi-view tracker. The number of cameras in different trackers can be the same or different. The tracker can form tracking information from images captured by the cameras. Specifically, the camera can continuously capture multiple image frames, with small time intervals between the multiple image frames, resulting in corresponding differences between the multiple image frames as the measuring device moves in space. The tracking information can include frame information. Each frame information can include image frames captured by multiple cameras of the corresponding tracker at the same time. It can be understood that each frame information can include at least one image frame.

[0034] The 3D reconstruction method can be applied to the aforementioned 3D reconstruction system. Specifically, it can be applied to one or more electronic devices within the system. Those skilled in the art can deploy the electronic devices that execute the 3D reconstruction method according to the specific circumstances.

[0035] Please see Figure 3 This application provides a three-dimensional reconstruction method. This three-dimensional reconstruction method is applied to an electronic device within a three-dimensional reconstruction system. The three-dimensional reconstruction method may include the following steps.

[0036] Step S110: Obtain environmental disturbance information during the scanning of point cloud data of the object under test; wherein, the environmental disturbance information includes the difference between the reconstructed position information of the specified reference object and the reference position information of the specified reference object.

[0037] Step S120: Generate a three-dimensional reconstruction model of the object under test based on the point cloud data of the object under test.

[0038] Step S130: Display the three-dimensional reconstruction model; wherein at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data.

[0039] In this embodiment, the electronic device can be used to perform a 3D reconstruction method. The electronic device can be at least one of a tracker, a measuring device, or a host computer in the 3D reconstruction system. Specifically, the measuring device can be used to scan the object under test to obtain point cloud data, and the tracker and / or the host computer can be used to process the relevant data generated during the scanning process, thereby cooperating to complete the acquisition of environmental disturbance information, the generation of the 3D reconstruction model, and the display of the 3D reconstruction model. The point cloud data can be used to characterize the 3D spatial point information of the surface of the object under test, and the electronic device can perform 3D morphological reconstruction of the object under test based on the point cloud data.

[0040] In this embodiment, the electronic device can be used to acquire environmental disturbance information during the scanning of point cloud data of the object under test. The environmental disturbance information can characterize the degree of influence of environmental factors on the point cloud data acquisition state during the scanning process. Specifically, the environmental disturbance information may include the difference between the reconstructed position information of a specified reference object and the reference position information of the specified reference object. The specified reference object can be a reference object whose position remains stable in the scanning environment, used to reflect whether the environmental state changes during the scanning process. The reference position information can be the position information determined by the electronic device for the specified reference object before the scanning begins and when the environment is relatively stable. Of course, in some embodiments, the reference position information can also be the position information of the specified reference object measured after the electronic device begins scanning the point cloud data of the object under test.

[0041] In this embodiment, the reconstructed position information can be the position information corresponding to a specified reference object determined by the electronic device during the scanning process. The difference quantity can be used to characterize the degree of offset of the reconstructed position information relative to the reference position information. For example, the difference quantity can include at least one of position offset, coordinate difference, distance difference, or attitude change. In some embodiments, the electronic device can continuously determine the reconstructed position information corresponding to the specified reference object while acquiring point cloud data, and compare the reconstructed position information at different times with the reference position information to obtain environmental disturbance information corresponding to the corresponding time or the corresponding point cloud data. In this way, the electronic device can use the positional change of the specified reference object to determine whether there are environmental factors such as vibration or airflow changes interfering with the scanning process.

[0042] In this embodiment, the electronic device can be used to generate a three-dimensional reconstruction model of the object under test based on point cloud data of the object under test. The three-dimensional reconstruction model can be a digital model used to characterize the three-dimensional shape of the object under test. In some embodiments, the electronic device can perform registration, stitching, fusion, and reconstruction processing on multiple sets of point cloud data acquired during the scanning process to generate a three-dimensional reconstruction model of the object under test. Since different point cloud data during the scanning process may correspond to different acquisition times, and the degree of influence of environmental factors may vary at different acquisition times, the electronic device can also establish a correspondence between point cloud data and environmental disturbance information during the generation of the three-dimensional reconstruction model. Specifically, the electronic device can determine the environmental disturbance information corresponding to at least some areas in the three-dimensional reconstruction model based on information such as the acquisition time or acquisition batch of the point cloud data, so that the subsequent display can reflect the interference of environmental factors on different areas.

[0043] In this embodiment, the electronic device can display the 3D reconstruction model, wherein at least a portion of the 3D reconstruction model reflects environmental disturbance information corresponding to the point cloud data. This can be understood as the electronic device, when displaying the 3D reconstruction model, enabling different regions of the 3D reconstruction model to represent the environmental interference experienced by the corresponding point cloud data. Specifically, the electronic device can associate the environmental disturbance information corresponding to each part of the point cloud data with the corresponding regions of the 3D reconstruction model while displaying the 3D reconstruction model, so that operators can simultaneously understand the environmental disturbance situation corresponding to different regions when observing the 3D reconstruction model. In some embodiments, regions with smaller environmental disturbances can be used to represent relatively reliable reconstruction results, while regions with larger environmental disturbances can be used to represent reconstruction results subject to more significant environmental interference. In this way, the electronic device can intuitively reflect the impact of environmental factors on the 3D reconstruction results during the scanning process in the 3D reconstruction model, facilitating the understanding of the accuracy of the 3D reconstruction model by the operator.

[0044] In some implementations, the designated reference point includes a fixed marker point set in the environment where the object under test is located.

[0045] In this embodiment, the designated reference point may include a fixed marker point set in the environment where the object under test is located. The fixed marker point can be a relatively stable reference object pre-set in the scanning environment to reflect whether environmental disturbances occur during the scanning process. Specifically, the electronic device can determine the reference position information of the fixed marker point before scanning the object under test, and acquire the reconstructed position information of the fixed marker point during the scanning process. This allows for subsequent determination of environmental disturbance information based on the difference between the reconstructed position information and the reference position information. Thus, the stability of the scanning environment can be more intuitively characterized by the fixed marker point. For example, when scanning industrial parts, the electronic device can pre-set multiple fixed marker points on the workbench, background plate, or support structure around the industrial parts. Under stable environmental conditions, the difference between the reconstructed position information and the reference position information of the fixed marker point is small; however, in the presence of vibration or airflow interference, the difference may increase, thus facilitating the electronic device's identification of environmental disturbances during the scanning process.

[0046] In some implementations, the electronic device can acquire point cloud data of the object under test and reconstructed position information of the fixed marker points; compare the reconstructed position information corresponding to the point cloud data with the reference position information of the fixed marker points to obtain environmental disturbance information corresponding to the point cloud data.

[0047] In this embodiment, when acquiring environmental disturbance information during the scanning of point cloud data of the object under test, the electronic device can correspondingly acquire the point cloud data of the object under test and the reconstructed position information of the fixed marker points. Specifically, corresponding acquisition can be understood as the electronic device simultaneously acquiring the reconstructed position information of the fixed marker points at a certain moment or in a certain batch of point cloud data, thereby enabling the subsequently obtained environmental disturbance information to establish a correspondence with the corresponding point cloud data. The reconstructed position information can be used to characterize the position result of the fixed marker points in the current scanning state.

[0048] In this embodiment, the electronic device can compare the reconstructed location information corresponding to the point cloud data with the reference location information of the fixed marker points to obtain the environmental disturbance information corresponding to the point cloud data. Specifically, the electronic device can calculate the difference between the reconstructed location information and the reference location information, and use the difference as the environmental disturbance information of the corresponding point cloud data. In this way, the influence of environmental factors during the scanning process can be characterized according to the acquisition time or batch of point cloud data.

[0049] In some implementations, the electronic device may output a first prompt message when it determines that the environmental disturbance information exceeds a specified disturbance threshold during the scanning of the point cloud data of the object under test; wherein the first prompt message is used to remind the user that there is environmental disturbance.

[0050] In this embodiment, during the scanning of the point cloud data of the object under test, the electronic device can also determine whether the environmental disturbance information exceeds a specified disturbance threshold, and output a first prompt message if the environmental disturbance information exceeds the specified disturbance threshold. Specifically, the specified disturbance threshold can be used to characterize the allowable range of environmental disturbance. The electronic device can compare the currently acquired environmental disturbance information with the specified disturbance threshold. When the environmental disturbance information is greater than the specified disturbance threshold, it determines that there is environmental disturbance in the current scanning process and outputs the first prompt message. The first prompt message can be used to remind the user that the current scanning process is affected by environmental factors. In this way, the user can be aware of abnormal environmental conditions in a timely manner during the scanning process, without having to determine whether the point cloud data is disturbed after the 3D reconstruction is completed.

[0051] In some implementations, the electronic device can further determine whether to output the first prompt message based on the number of consecutive anomalies. Specifically, the electronic device can identify a frame as an abnormal frame when the environmental disturbance information corresponding to a certain frame exceeds a specified disturbance threshold, and continuously count the number of consecutive abnormal frames. When the number of consecutive anomalies reaches an anomaly count threshold, the electronic device outputs the first prompt message. The anomaly count threshold can be a pre-set upper limit for the number of consecutive anomalies, such as 20 consecutive frames being abnormal frames. In this way, premature prompts to the user can be avoided due to occasional fluctuations in individual frames, thereby making the reminders for environmental disturbances more stable.

[0052] In this embodiment, the first prompt information can be at least one of text prompts, graphic prompts, color prompts, sound prompts, or vibration prompts. For example, when the electronic device continuously scans the object under test, if the environmental disturbance information corresponding to the current moment exceeds the specified disturbance threshold based on the position deviation of the fixed marker point, and the disturbance exceeds the specified disturbance threshold for 20 consecutive frames, then the device can output a prompt message "There is currently an environmental disturbance" on the display interface, or prompt the user through highlighting, flashing, or other means. This allows the user to adjust the scanning operation in a timely manner and reduces the impact of environmental disturbances on subsequent point cloud data acquisition.

[0053] In some implementations, the electronic device can pause the generation of point cloud data and continue to generate environmental disturbance information upon receiving a pause command; if the environmental disturbance information is lower than or equal to a specified disturbance threshold, it can output a second prompt message; wherein the second prompt message is used to indicate that the object under test can be scanned normally to generate point cloud data.

[0054] In this embodiment, after outputting the first prompt information, the electronic device can also receive a pause command input by the user. Upon receiving the pause command, it pauses the generation of point cloud data and continues to generate environmental disturbance information. Specifically, the pause command can be a scan pause control command triggered by the user based on the first prompt information, used to pause the point cloud data acquisition process for the object under test. Continuously generating environmental disturbance information can be understood as the electronic device continuing to acquire the reconstructed position information of the fixed marker points during the pause of point cloud data generation, and determining the current environmental disturbance information based on the difference between the reconstructed position information and the reference position information. In this way, the electronic device can continuously monitor whether the environmental state has returned to stability during the pause of scanning.

[0055] In this embodiment, the electronic device can also output a second prompt message when the environmental disturbance information is lower than or equal to a specified disturbance threshold. The second prompt message can be used to indicate that the object under test can be scanned normally to generate point cloud data. Specifically, the electronic device can continuously compare the environmental disturbance information generated during the pause with the specified disturbance threshold. When it is determined that the current environmental disturbance information has fallen back below the specified disturbance threshold, the second prompt message is output to indicate to the user that the current environment meets the conditions for continuing scanning. This allows the user to continue scanning after the environment stabilizes, reducing the impact of environmental disturbances on subsequent point cloud data acquisition.

[0056] In some implementations, the electronic device can display the 3D reconstructed model with different visual attributes based on the environmental disturbance information.

[0057] In this embodiment, when displaying the 3D reconstruction model, the electronic device can also display the 3D reconstruction model with different visual attributes based on the environmental disturbance information. Specifically, visual attributes can be used to characterize the environmental interference affecting the point cloud data corresponding to different regions in the 3D reconstruction model. The electronic device can first determine the environmental disturbance information corresponding to at least some regions in the 3D reconstruction model, and then determine the visual attributes corresponding to each region based on the environmental disturbance information of each region, thereby distinguishing and displaying different regions with different visual attributes when displaying the 3D reconstruction model. In this way, when the operator observes the 3D reconstruction model, they can not only view the 3D shape of the object being measured, but also simultaneously identify the environmental disturbance situation corresponding to each region.

[0058] In this embodiment, the electronic device can pre-establish a correspondence between environmental disturbance information and visual attributes. This correspondence can be such that the greater the environmental disturbance, the more pronounced the corresponding visual attribute, or that different types or degrees of visual attributes correspond to different ranges of environmental disturbance information. For example, the electronic device can divide the environmental disturbance information into multiple disturbance levels and configure different display methods for each level. For areas with relatively small environmental disturbance information, a first visual attribute can be used for display; for areas with relatively large environmental disturbance information, a second visual attribute, different from the first visual attribute, can be used for display. In this way, the degree of influence of environmental factors on point cloud data acquisition during the scanning process can be visualized in the 3D reconstruction model.

[0059] In some implementations, the visual attribute may include color. Specifically, the electronic device can use different colors to display different regions of the 3D reconstructed model based on the magnitude of the environmental disturbance information. For example, regions with smaller environmental disturbance information can be displayed in a first color, and regions with larger environmental disturbance information can be displayed in a second color. Furthermore, the electronic device can also set a color gradient relationship according to the magnitude of the environmental disturbance information, so that the displayed color gradually changes as the environmental disturbance information gradually increases. In this way, the reliability of data corresponding to different regions in the 3D reconstructed model can be more intuitively distinguished through color differences.

[0060] In some implementations, the visual attribute may also include transparency. Specifically, the electronic device can adjust the transparency of corresponding areas in the 3D reconstructed model based on the environmental disturbance information. For example, areas with less environmental disturbance information can be displayed with lower transparency or opacity to indicate that the point cloud data corresponding to that area is less affected by environmental disturbances; areas with more environmental disturbance information can be displayed with higher transparency to indicate that the point cloud data corresponding to that area is more affected by environmental disturbances. In this way, by varying the degree of transparency, operators can more easily focus on areas with less environmental disturbance and relative stability when observing the 3D reconstructed model as a whole.

[0061] In some implementations, the visual attributes may further include texture. Specifically, the electronic device can overlay different texture identifiers for areas corresponding to different environmental disturbance information. For example, for areas with less environmental disturbance information, no texture may be overlaid, or a sparser texture may be overlaid; for areas with more environmental disturbance information, a denser texture, diagonal line texture, grid texture, or warning texture may be overlaid. In this way, without changing the basic color display method of the 3D reconstruction model, the environmental disturbance situation corresponding to different areas can be further represented by texture differences.

[0062] In some implementations, the visual attributes may also include highlighting. Specifically, the electronic device can highlight areas where environmental disturbance information exceeds a specified level, thereby emphasizing that the point cloud data corresponding to that area has been significantly affected by environmental factors during the scanning process. For example, the electronic device can add contour highlighting, boundary enhancement, or local flickering to areas where environmental disturbance information is higher than a preset level. In this way, operators can quickly locate the parts of the 3D reconstructed model that are significantly affected by environmental disturbances.

[0063] Furthermore, the electronic device can display the 3D reconstructed model using only one of the visual attributes—color, transparency, texture, or highlighting—or it can combine at least two of these visual attributes for joint display. For example, the electronic device can use color to distinguish the size range of environmental disturbance information and further use highlighting to emphasize areas where the environmental disturbance information exceeds a preset level; alternatively, the electronic device can overlay texture while adjusting transparency to enhance display differentiation. In this way, the display method of the 3D reconstructed model can be flexibly adjusted according to the display requirements of different application scenarios.

[0064] For example, after continuously scanning industrial parts and generating the 3D reconstructed model, the electronic device can divide the 3D reconstructed model into multiple regions for display based on the environmental disturbance information corresponding to each batch of point cloud data. Specifically, the electronic device can display regions with minimal environmental disturbance as normal colors without overlaying textures, display regions with moderate environmental disturbance as a different color with added sparse textures, and display regions with significant environmental disturbance as highlighted or highly transparent. In this way, operators can quickly identify which regions were acquired under relatively stable environmental conditions and which were acquired under conditions of significant environmental disturbance when viewing the 3D reconstructed model, thus facilitating an understanding of the accuracy of the 3D reconstructed model.

[0065] In some implementations, electronic devices can acquire environmental disturbance information generated by a tracker that tracks the measuring device during the process of generating point cloud data by scanning the object under test with the measuring device.

[0066] In this embodiment, the tracker serves as an electronic device for tracking the measuring equipment. During the scanning process of the measuring equipment on the object under test, it tracks the spatial position and orientation of the measuring equipment and outputs tracking results corresponding to the measuring equipment. The environmental disturbance information is generated by the tracker. This can be understood as the tracker, while tracking the measuring equipment, further combining position reference information in the scanning environment to determine the changes in the environmental state during the scanning process and outputting information characterizing the degree of environmental disturbance. In this way, a correspondence can be established between the environmental disturbance information and the scanning process of the measuring equipment.

[0067] Specifically, during the process of the measuring device scanning the object under test to generate point cloud data, the tracker can continuously capture image data including the measuring device and the designated reference object, and track the measuring device based on the image data. Simultaneously, the tracker can also determine the reconstructed position information of the designated reference object in the current scanning state, and generate corresponding environmental disturbance information based on the difference between the reconstructed position information and the reference object's baseline position information. The electronic device can directly acquire the environmental disturbance information generated by the tracker as the environmental disturbance information corresponding to the point cloud data. In this way, the data processing results of the tracker during the tracking of the measuring device can directly reflect the influence of environmental factors on the scanning process.

[0068] In some implementations, when generating environmental disturbance information, the tracker can synchronously output the corresponding environmental disturbance information according to the acquisition time or batch of point cloud data. In this way, when the electronic device receives the point cloud data output by the measuring device, it can also receive the corresponding environmental disturbance information generated by the tracker, thus facilitating the subsequent association of different point cloud data affected by environmental factors with the corresponding areas of the 3D reconstruction model.

[0069] For example, the measuring device is a handheld laser scanner, and the tracker is a binocular tracker. When an operator uses the measuring device to scan a shell-like part, the binocular tracker can continuously track the measuring device and simultaneously capture images of multiple designated reference objects set on a background plate. Under stable environmental conditions, the reconstructed position information of the designated reference objects differs little from the baseline position information, resulting in minimal environmental disturbance information generated by the tracker. However, in the presence of vibration or airflow interference, the positional fluctuations of the designated reference objects increase, leading to a corresponding increase in the environmental disturbance information generated by the tracker. This allows the electronic device to determine, based on the environmental disturbance information generated by the tracker, whether the point cloud data is affected by environmental factors during the scanning process.

[0070] One embodiment of this application provides an example application scenario of a 3D reconstruction method. The 3D reconstruction system may include a tracker, a measuring device, and a host computer. The measuring device can be used to scan the object under test to obtain point cloud data, the tracker can be used to acquire tracking information from the measuring device, and the host computer can communicate with the tracker and the measuring device, receiving data provided by the tracker and the measuring device to execute the 3D reconstruction method. In this scenario example, the host computer is used as an electronic device to execute the 3D reconstruction method. Specifically, the example describes the use of the host computer in an industrial inspection scenario to perform 3D reconstruction of shell-type parts on a production line.

[0071] For example, on a mechanical parts inspection production line, operators need to use a 3D reconstruction system to scan a metal shell to generate a corresponding 3D reconstruction model for subsequent dimensional analysis and shape inspection. The 3D reconstruction system may include a binocular tracker, a handheld laser scanner, and a host computer. The handheld laser scanner, as a measuring device, can continuously scan the metal shell by emitting laser light and receiving reflected signals, thereby generating point cloud data. The binocular tracker can be used to capture tracking features on the measuring device and output the tracking information of the measuring device in space. The host computer can receive the point cloud data output by the measuring device and the tracking information output by the tracker, and process these data uniformly to acquire environmental disturbance information, generate the 3D reconstruction model, and display the 3D reconstruction model.

[0072] In this scenario example, the operator can pre-set multiple fixed marker points in the environment where the object under test is located before starting the scan. Specifically, the fixed marker points can be set on the workbench surface, background plate, or supporting structure, and their positions should remain stable under normal circumstances. These fixed marker points can serve as designated reference points to reflect whether environmental disturbances occur during the scan. For example, four fixed marker points can be set on the background plate around the metal casing, denoted as fixed marker point A, fixed marker point B, fixed marker point C, and fixed marker point D. Before the scan begins, the host computer can receive relevant data collected by the tracker or measuring device and determine the reference position information corresponding to each of the fixed marker points A to D. The reference position information can be the position reference information corresponding to the fixed marker point when the environment is relatively stable.

[0073] When the operator continuously scans the metal casing using the handheld measuring device, the measuring device continuously outputs point cloud data of the object under test, and the tracker continuously outputs the tracking information of the measuring device, while simultaneously capturing images of the fixed marker points. The host computer, as an electronic device, can acquire environmental disturbance information during the scanning of the point cloud data of the object under test. Specifically, the host computer can acquire the point cloud data of the object under test and the reconstructed position information of the fixed marker points. This acquisition can be understood as the host computer simultaneously receiving or determining the reconstructed position information of the fixed marker points at a specific moment or batch of point cloud data, thereby enabling the subsequently obtained environmental disturbance information to establish a correspondence with the corresponding point cloud data.

[0074] Continuing with the above scenario as an example, the measuring device can output point cloud data frame by frame or batch by batch during the scanning process. For example, when acquiring the first frame of point cloud data, the host computer can simultaneously determine the reconstructed position information of fixed marker points A to D in the first frame based on the image frames captured by the tracker; when acquiring the second frame of point cloud data, the host computer can determine the reconstructed position information of fixed marker points A to D in the second frame; subsequent frames can also be processed in the same way. The reconstructed position information can be used to characterize the position result of the fixed marker points in the current scanning state.

[0075] The host computer can compare the reconstructed position information corresponding to the point cloud data with the reference position information of the fixed marker points to obtain the environmental disturbance information corresponding to the point cloud data. Specifically, the host computer can calculate the difference between the reconstructed position information of fixed marker points A to D in the current frame and the reference position information, and determine the environmental disturbance information corresponding to the point cloud data in the current frame based on the difference corresponding to one or more fixed marker points. For example, if the position deviations of fixed marker points A to D are small during the first 20 frames of scanning, the host computer can determine that the environmental disturbance information corresponding to the point cloud data in the first 20 frames is small; however, during the scanning process from frame 21 to frame 35, vibrations occur near the workstation due to the operation of nearby equipment or personnel movement, and the position deviations of fixed marker points A to D increase significantly. Therefore, the host computer can determine that the environmental disturbance information corresponding to the point cloud data from frame 21 to frame 35 is large. In this way, the host computer can characterize the impact of environmental factors during the scanning process according to the acquisition time or batch of point cloud data.

[0076] During the scanning of the point cloud data of the object under test, the host computer can also determine whether the environmental disturbance information exceeds a specified disturbance threshold, and if it determines that the environmental disturbance information exceeds the specified disturbance threshold, it outputs a first prompt message. This first prompt message is used to remind the user of the existence of environmental disturbance. In this scenario example, the host computer can pre-set a specified disturbance threshold to characterize the allowable range of environmental disturbance. When the environmental disturbance information corresponding to the 21st frame exceeds the specified disturbance threshold, the host computer can output the first prompt message "Currently there is environmental disturbance" on the display interface, or remind the operator through pop-ups, flashing icons, sound alarms, etc. In this way, the operator can promptly know that the current point cloud data acquisition is affected by environmental factors during the scanning process.

[0077] Upon receiving a pause command, the host computer can pause the generation of point cloud data while continuing to generate environmental disturbance information. Specifically, after seeing the first prompt message, the operator can input a pause command through the host computer interface. Upon receiving the pause command, the host computer can control the measuring equipment to pause the output of point cloud data for reconstruction, or pause the reception and reconstruction processing of subsequent point cloud data, thereby pausing point cloud data generation. Simultaneously, the host computer can continue to receive image data captured by the tracker, or continue to receive data related to the fixed marker points, and continue to generate environmental disturbance information. Specifically, during the pause of point cloud data generation, the host computer continues to acquire the reconstructed position information from fixed marker point A to fixed marker point D, and continuously determines the current environmental disturbance information based on the difference between the reconstructed position information and the reference position information. Thus, during the pause of scanning, the host computer can continuously monitor whether the environmental state has returned to stability.

[0078] When the environmental disturbance information is lower than or equal to a specified disturbance threshold, the host computer can output a second prompt message. This second prompt message indicates that the measured object can be scanned normally to generate point cloud data. Continuing with the above scenario, after the forklift passes through the workstation area and the surrounding vibrations gradually disappear, the host computer detects that the positional deviation between fixed marker A and fixed marker D has decreased again, and the corresponding environmental disturbance information is lower than or equal to the specified disturbance threshold. Therefore, it can output the second prompt message "Scanning can proceed normally" on the display interface. After receiving the second prompt message, the operator can continue to use the measuring device to scan the metal casing, and the host computer can resume receiving and processing subsequent point cloud data. This reduces the impact of environmental disturbances on subsequent point cloud data acquisition.

[0079] After acquiring complete point cloud data, the host computer can generate a 3D reconstruction model of the object under test based on the point cloud data. Specifically, the host computer can combine the point cloud data output by the measuring device and the tracking information output by the tracker to perform pose registration, stitching, and fusion processing on each frame or batch of point cloud data to obtain a 3D reconstruction model characterizing the 3D morphology of the metal shell. Furthermore, the host computer can also determine the environmental disturbance situation corresponding to different regions in the 3D reconstruction model based on the environmental disturbance information corresponding to each frame or batch of point cloud data. For example, the left outer surface of the metal shell is mainly reconstructed from the first 20 frames of point cloud data, so this region can correspond to relatively small environmental disturbance information; a part of the top of the metal shell is mainly reconstructed from the point cloud data from the 21st to the 35th frames, so this region can correspond to relatively large environmental disturbance information; the right outer surface of the metal shell is mainly reconstructed from the point cloud data of subsequent frames after the recovery scan, so this region can again correspond to relatively small environmental disturbance information.

[0080] When displaying the 3D reconstructed model, the host computer can make at least a portion of the 3D reconstructed model reflect the environmental disturbance information corresponding to the point cloud data. Specifically, the host computer can display the 3D reconstructed model with different visual attributes based on the environmental disturbance information. The visual attributes may include at least one of color, transparency, texture, or highlighting. The host computer can pre-establish the correspondence between environmental disturbance information and visual attributes. For example, for areas with small environmental disturbance information, the host computer can use a normal color; for areas with moderate environmental disturbance information, the host computer can use a different color than the normal color, or overlay a sparse texture; for areas with large environmental disturbance information, the host computer can use highlighting, high transparency, warning texture, or a combination of these visual attributes.

[0081] Continuing with the example of scanning the metal casing, when displaying the 3D reconstructed model, the host computer can show the left outer surface area with less environmental disturbance as normal gray; the local area with moderate environmental disturbance as yellow with an overlaid diagonal texture; and the top area with significant environmental disturbance as highlighted red, or with increased transparency, to emphasize that the point cloud data corresponding to that area has been significantly affected by environmental factors during the scanning process. Furthermore, the host computer can also display the corresponding environmental disturbance value or disturbance level when the user selects a specific area. In this way, when viewing the 3D reconstructed model, the operator can not only view the 3D shape of the metal casing but also quickly identify which areas were acquired under stable environmental conditions and which were acquired under conditions of significant environmental disturbance. This facilitates understanding the accuracy of the 3D reconstructed model and allows for a decision on whether to rescan local areas.

[0082] In this scenario example, the host computer in the 3D reconstruction system, acting as an electronic device, executes the 3D reconstruction method. It receives point cloud data output from the measuring device, tracking information provided by the tracker, and data related to fixed marker points to acquire environmental disturbance information corresponding to each frame of point cloud data. When the environmental disturbance information exceeds a specified disturbance threshold, a first prompt message is output. Upon receiving a pause command, point cloud data generation is paused while environmental disturbance information generation continues. When the environmental disturbance information is below or equal to the specified disturbance threshold, a second prompt message is output. Finally, when displaying the 3D reconstruction model, different visual attributes are used to represent the environmental disturbance information corresponding to the point cloud data. In this way, the host computer can intuitively reflect the impact of environmental factors on the 3D reconstruction results during the scanning process in the 3D reconstruction model, facilitating staff understanding the accuracy of the 3D reconstruction model.

[0083] Please see Figure 4This application provides a three-dimensional reconstruction device. The three-dimensional reconstruction device includes: an acquisition module, a generation module, and a display module.

[0084] The acquisition module is used to acquire environmental disturbance information during the scanning of point cloud data of the object under test; wherein, the environmental disturbance information includes the difference between the reconstructed position information of the specified reference object and the baseline position information of the specified reference object.

[0085] The generation module is used to generate a three-dimensional reconstruction model of the object under test based on the point cloud data of the object under test. The display module is used to display the three-dimensional reconstruction model; wherein at least a portion of the three-dimensional reconstruction model reflects the environmental disturbance information corresponding to the point cloud data.

[0086] In this embodiment, the specific functions and effects achieved by the three-dimensional reconstruction device can be explained by referring to other embodiments of this application, and will not be repeated here.

[0087] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method as described above.

[0088] This application also provides a computer program product containing instructions that, when executed by a processor, implement the method as described above.

[0089] Please see Figure 5 Embodiments of this application may provide an electronic device, the electronic device comprising: a memory, and one or more processors communicatively connected to the memory; the memory storing instructions executable by the one or more processors, the instructions being executed by the one or more processors to cause the one or more processors to implement the method as described above.

[0090] In some embodiments, the electronic device may include a processor, a storage medium, and a communication interface connected to a system bus. The storage medium may store related computer programs.

[0091] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments of this application, and are not intended to limit the scope of the invention.

[0092] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0094] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0095] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0096] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0097] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0101] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0102] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: Acquire environmental disturbance information during the scanning of point cloud data of the object under test; wherein, the environmental disturbance information includes the amount of difference between the reconstructed position information of the specified reference object and the baseline position information of the specified reference object; A three-dimensional reconstruction model of the object under test is generated based on the point cloud data of the object under test; The three-dimensional reconstruction model is displayed; wherein at least a portion of the three-dimensional reconstruction model reflects environmental disturbance information corresponding to the point cloud data.

2. The method according to claim 1, characterized in that, The designated reference points include fixed marker points set in the environment where the object under test is located.

3. The method according to claim 2, characterized in that, Acquire environmental disturbance information during the scanning of point cloud data of the object under test, including: The point cloud data of the object under test and the reconstructed location information of the fixed marker points are obtained accordingly. The reconstructed location information corresponding to the point cloud data is compared with the reference location information of the fixed marker point to obtain the environmental disturbance information corresponding to the point cloud data.

4. The method according to claim 1, characterized in that, The method further includes: During the scanning of the point cloud data of the object under test, if the environmental disturbance information is determined to exceed a specified disturbance threshold, a first prompt message is output; wherein, the first prompt message is used to remind the user that there is environmental disturbance.

5. The method according to claim 4, characterized in that, The method further includes: Upon receiving a pause command, the generation of point cloud data is paused while environmental disturbance information is continuously generated. If the environmental disturbance information is lower than or equal to a specified disturbance threshold, a second prompt message is output; wherein, the second prompt message is used to indicate that the point cloud data of the object under test can be generated normally.

6. The method according to claim 1, characterized in that, The three-dimensional reconstruction model is shown, including: Based on the environmental disturbance information, the three-dimensional reconstructed model is displayed with different visual attributes.

7. The method according to claim 6, characterized in that, The visual attributes include at least one of color, transparency, texture, or highlighting.

8. The method according to claim 1, characterized in that, Acquire environmental disturbance information during the scanning of point cloud data of the object under test, including: During the process of generating point cloud data by scanning the object under test with a measuring device, environmental disturbance information based on the tracker generated by the measuring device is acquired.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, is capable of implementing the method as described in any one of claims 1 to 8.

11. A computer program product, characterized in that, The computer program product is used to implement the method as described in any one of claims 1 to 8.

Images