Method, apparatus, device and storage medium for object reconstruction

Abstract

This application discloses a method, apparatus, device, and storage medium for object reconstruction, belonging to the field of 3D reconstruction technology. The method includes: when a terminal detects a target object, performing feature position detection on the target object to obtain feature information; if feature information is detected, scanning a local area around the target object to obtain first scan information; and establishing a 3D model of the target object based on the first scan information. After detecting the target object to be modeled, this application first determines whether the target object has feature information through feature position detection. If the target object has feature information, a 3D model of the target object is then established based on the first scan information. Therefore, the 3D model established in this embodiment closely matches the actual shape of the target object, has high accuracy, and achieves good object reconstruction results.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and in particular to methods, apparatus, equipment and storage media for object reconstruction. Background Technology

[0002] With the development of 3D reconstruction technology, its applications on mobile devices are increasing, and object reconstruction is one such application. Object reconstruction refers to the process of building a 3D model based on information obtained from scanning the object to be reconstructed. Therefore, how to perform object reconstruction is crucial to ensuring the accuracy of the resulting 3D model.

[0003] In related technologies, when a terminal detects a target object to be modeled, it first confirms the distance between the target object and the terminal. If the distance meets the requirements, the user is further prompted to scan around the target object using the mobile terminal to obtain the scan information of the target object. Finally, a 3D model of the target object is built based on the scan information, resulting in a 3D model.

[0004] It can be seen that the relevant technology triggers scanning of the target object when the distance between the target object and the terminal meets the required condition. If other objects exist around the target object, the information of these other objects is also used as scanning information for the target object to build its 3D model. However, in this case, the 3D model built will deviate significantly from the actual shape of the target object. Therefore, the accuracy of the 3D model obtained by applying the relevant technology for object reconstruction is low, and the object reconstruction effect is poor. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for object reconstruction to solve the problems provided by related technologies. The technical solution is as follows:

[0006] In a first aspect, a method for object reconstruction is provided, the method being applied to a terminal, the method comprising: when the terminal detects a target object, performing feature location detection on the target object, the feature location detection being used to obtain feature information of the target object; if the feature information of the target object is detected, scanning around a local region of the target object to obtain first scan information of the target object; and establishing a three-dimensional model of the target object based on the first scan information.

[0007] After detecting the target object to be modeled, feature location detection is first used to determine whether the target object has feature information. If the target object is found to have feature information, a 3D model of the target object is then built based on the first scan information. Therefore, the 3D model built in this embodiment closely matches the actual shape of the target object, has high accuracy, and achieves good object reconstruction results.

[0008] In an exemplary embodiment, if the target object is a human body, the feature location is the face.

[0009] In an exemplary embodiment, if feature information of the target object is detected, the method further includes: determining pose information of the terminal relative to the target object based on the feature information, wherein the pose information includes one or both of position information and posture information; if the pose information satisfies a condition, then performing a scan of a local region around the target object to obtain first scan information of the target object.

[0010] In an exemplary embodiment, after scanning a local region around the target object to obtain first scan information of the target object, the method further includes: determining a target region of the target object based on the local region of the target object, the target region including at least a portion of other regions on the target object besides the local region; and obtaining second scan information corresponding to the target region.

[0011] The step of establishing a three-dimensional model of the target object based on the first scanning information includes: establishing a three-dimensional model of the target object based on the first scanning information and the second scanning information.

[0012] In an exemplary embodiment, before scanning a local region around the target object to obtain first scanning information of the target object, the method further includes: acquiring reference scanning information, the reference scanning information including one or more of reference scanning trajectory, reference scanning speed, and reference scanning posture; and outputting the reference scanning information, the reference scanning information being used to guide scanning of a local region of the target object.

[0013] In an exemplary embodiment, obtaining reference scan information includes: determining the core region of the target object based on the feature information, and establishing a feature position model of the target object; determining the completeness information of the feature position model based on the core region; and determining the reference scan information based on the completeness information.

[0014] In an exemplary embodiment, determining the completeness information of the feature location model based on the core region includes: determining an upper region, a lower region, a left region, and a right region on the feature location model with the core region as the center; obtaining the normalized completeness corresponding to each region; and using the normalized completeness corresponding to each region as the completeness information of the feature location model.

[0015] In an exemplary embodiment, when the reference scan information includes the reference scan trajectory, determining the reference scan information based on the integrity information includes: determining the scan distance between the terminal and the target object based on the normalized integrity corresponding to the core region; determining the scan height and the number of scans based on the normalized integrity corresponding to the upper region and the lower region; determining the scan direction based on the normalized integrity corresponding to the left region and the right region; and determining the reference scan trajectory based on the scan distance, the scan height, the number of scans, and the scan direction.

[0016] In an exemplary embodiment, when the reference scan information includes the reference scan trajectory, the reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories, and outputting the reference scan information includes: sequentially displaying the number of reference sub-scan trajectories according to the position of the terminal.

[0017] In an exemplary embodiment, after outputting the reference scanning information, the method further includes: detecting the actual scanning information of the terminal, the actual scanning information including one or more of the following: actual scanning trajectory, actual scanning speed, and actual scanning posture; if the deviation between the actual scanning information and the reference scanning information is greater than a reference threshold, displaying a prompt message based on the deviation, the prompt message being used to prompt correction of the actual scanning information.

[0018] In an exemplary embodiment, after scanning a local area around the target object to obtain first scan information of the target object, the method further includes: determining the scanned area and the unscanned area of ​​the target object; displaying a projection image of the target object, and marking the scanned area and the unscanned area on the projection image of the target object respectively.

[0019] In an exemplary embodiment, after establishing a three-dimensional model of the target object based on the first scanning information, the method further includes: performing post-processing on the three-dimensional model of the target object to obtain a processed three-dimensional model; and displaying the processed three-dimensional model.

[0020] In an exemplary embodiment, when the reference scan information includes the reference scan trajectory, after the actual scan information of the detection terminal, the method further includes: displaying the actual scan trajectory of the target object and the reference scan trajectory.

[0021] In an exemplary embodiment, after scanning a local area around the target object to obtain first scan information of the target object, the method further includes: obtaining scan evaluation information based on the deviation between the actual scan information and the reference scan information, wherein the scan evaluation information is used to evaluate the scan situation; and displaying the scan evaluation information.

[0022] Secondly, an apparatus for object reconstruction is provided, the apparatus comprising:

[0023] The detection module is used to perform feature location detection on the target object when the terminal detects the target object, and the feature location detection is used to obtain feature information of the target object;

[0024] The acquisition module is used to scan a local area around the target object if feature information of the target object is detected, and obtain first scan information of the target object;

[0025] A module is established to create a three-dimensional model of the target object based on the first scan information.

[0026] In an exemplary embodiment, if the target object is a human body, the feature location is the face.

[0027] In an exemplary embodiment, the apparatus further includes: a first determining module, configured to determine the pose information of the terminal relative to the target object based on the feature information if feature information of the target object is detected, wherein the pose information includes one or both of position information and posture information; and an acquiring module, configured to perform scanning of a local region around the target object to obtain first scanning information of the target object if the pose information satisfies a condition.

[0028] In an exemplary embodiment, the apparatus further includes: a second determining module, configured to determine a target region of the target object based on a local region of the target object, the target region including at least a portion of other regions on the target object besides the local region; and to acquire second scanning information corresponding to the target region; and the establishing module, configured to establish a three-dimensional model of the target object based on the first scanning information and the second scanning information.

[0029] In an exemplary embodiment, the device further includes: a first display module, configured to acquire reference scanning information, the reference scanning information including one or more of reference scanning trajectory, reference scanning speed, and reference scanning posture; and output the reference scanning information, the reference scanning information being used to guide scanning of a local area of ​​the target object.

[0030] In an exemplary embodiment, the first display module includes: an establishment unit, configured to determine the core region of the target object based on the feature information, and establish a feature position model of the target object; a first determination unit, configured to determine the completeness information of the feature position model based on the core region; and a second determination unit, configured to determine the reference scan information based on the completeness information.

[0031] In an exemplary embodiment, the first determining unit is configured to determine an upper region, a lower region, a left region, and a right region on the feature location model, with the core region as the center; obtain the normalized completeness corresponding to each region, and use the normalized completeness corresponding to each region as the completeness information of the feature location model.

[0032] In an exemplary embodiment, the second determining unit is configured to, when the reference scanning information includes the reference scanning trajectory, determine the scanning distance between the terminal and the target object based on the normalized completeness corresponding to the core region, determine the scanning height and the number of scans based on the normalized completeness corresponding to the upper region and the lower region, and determine the scanning direction based on the normalized completeness corresponding to the left region and the right region; and determine the reference scanning trajectory based on the scanning distance, the scanning height, the number of scans, and the scanning direction.

[0033] In an exemplary embodiment, when the reference scan information includes the reference scan trajectory, the reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories, and the first display module is used to sequentially display each sub-scan trajectory according to the position of the terminal.

[0034] In an exemplary embodiment, the device further includes: a second display module, configured to detect actual scanning information of the terminal, the actual scanning information including one or more of actual scanning trajectory, actual scanning speed, and actual scanning posture; if the deviation between the actual scanning information and the reference scanning information is greater than a reference threshold, displaying prompt information according to the deviation, the prompt information being used to prompt correction of the actual scanning information.

[0035] In an exemplary embodiment, the apparatus further includes: a labeling module, configured to determine the scanned and unscanned areas of the target object; display a projected image of the target object, and label the scanned and unscanned areas respectively on the projected image of the target object.

[0036] In an exemplary embodiment, the apparatus further includes: a processing module, configured to perform post-processing on the three-dimensional model of the target object to obtain a processed three-dimensional model; and to display the processed three-dimensional model.

[0037] In an exemplary embodiment, the apparatus further includes a third display module, configured to display the actual scanning trajectory of the target object and the reference scanning trajectory when the reference scanning information includes the reference scanning trajectory.

[0038] In an exemplary embodiment, the device further includes: a fourth display module, configured to obtain scan evaluation information based on the deviation between the actual scan information and the reference scan information, the scan evaluation information being used to evaluate the scan status; and to display the scan evaluation information.

[0039] Thirdly, an apparatus for object reconstruction is provided, the apparatus including a memory and a processor; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first aspect or any possible implementation thereof.

[0040] Fourthly, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the instruction being loaded and executed by a processor to implement the method of the first aspect or any possible implementation thereof.

[0041] Fifthly, an object reconstruction apparatus is provided, comprising a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory to control the transceiver to receive and transmit signals. When the processor executes the instructions stored in the memory, it causes the processor to perform the method in any possible implementation of the first aspect.

[0042] Optionally, the processor may be one or more, and the memory may be one or more.

[0043] Optionally, the memory may be integrated with the processor, or the memory may be separated from the processor.

[0044] In specific implementation, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. The embodiments of this application do not limit the type of memory or the way the memory and processor are set.

[0045] In a sixth aspect, a computer program (product) is provided, the computer program (product) comprising: computer program code, which, when executed by a computer, causes the computer to perform the method in any of the possible implementations of the first aspect described above.

[0046] In a seventh aspect, a chip is provided, including a processor for retrieving and executing instructions stored in a memory, such that a communication device on which the chip is mounted performs the method in any of the possible embodiments of the first aspect described above.

[0047] Eighthly, another chip is provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is configured to execute code in the memory, wherein when the code is executed, the processor is configured to perform a method in any possible implementation of the first aspect described above. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of a terminal provided in an embodiment of this application;

[0049] Figure 2 A flowchart of the object reconstruction method provided in the embodiments of this application;

[0050] Figure 3 A schematic diagram of the user interface provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram of the user interface provided in an embodiment of this application;

[0052] Figure 5 A schematic diagram of the user interface provided in an embodiment of this application;

[0053] Figure 6 A schematic diagram of the user interface provided in an embodiment of this application;

[0054] Figure 7 A schematic diagram of the user interface provided in an embodiment of this application;

[0055] Figure 8 A schematic diagram of the user interface provided in an embodiment of this application;

[0056] Figure 9 A schematic diagram illustrating the object reconstruction method provided in an embodiment of this application;

[0057] Figure 10 A schematic diagram of the user interface provided in an embodiment of this application;

[0058] Figure 11 A schematic diagram of a reference scan trajectory provided for an embodiment of this application;

[0059] Figure 12 A schematic diagram of the user interface provided in an embodiment of this application;

[0060] Figure 13 A schematic diagram of the user interface provided in an embodiment of this application;

[0061] Figure 14 This is a scanning diagram provided for an embodiment of this application;

[0062] Figure 15 A schematic diagram of the user interface provided in an embodiment of this application;

[0063] Figure 16 A schematic diagram of the user interface provided in an embodiment of this application;

[0064] Figure 17 A schematic diagram of the user interface provided in an embodiment of this application;

[0065] Figure 18 A schematic diagram of the user interface provided in an embodiment of this application;

[0066] Figure 19 A schematic diagram of the user interface provided in an embodiment of this application;

[0067] Figure 20 A schematic diagram of the supplementary scanning trajectory provided for embodiments of this application;

[0068] Figure 21 A schematic diagram of the complete scan trajectory provided in the embodiments of this application;

[0069] Figure 22 A schematic diagram of the user interface provided in an embodiment of this application;

[0070] Figure 23 A schematic diagram of the user interface provided in an embodiment of this application;

[0071] Figure 24 This is a schematic diagram of a projection provided for an embodiment of this application;

[0072] Figure 25 This is a schematic diagram of a projection provided for an embodiment of this application;

[0073] Figure 26 This is a schematic diagram of trajectory comparison provided in the embodiments of this application;

[0074] Figure 27 A schematic diagram of the user interface provided in an embodiment of this application;

[0075] Figure 28 A schematic diagram of an object reconstruction apparatus provided in an embodiment of this application;

[0076] Figure 29 This is a structural diagram of the network device provided in the embodiments of this application. Detailed Implementation

[0077] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0078] With the development of computer graphics (CG) and computer vision technologies, 3D reconstruction technology has been widely applied in various aspects of people's lives, such as movie special effects, 3D games, and virtual reality (VR). Among these, object reconstruction technology, as a core direction of 3D reconstruction, has considerable development prospects and potential value. Object reconstruction refers to the process of building a 3D model based on information obtained from scanning the object to be scanned. Currently, due to the widespread use of consumer-grade 3D cameras, object reconstruction technology is increasingly being used on mobile devices. Therefore, how to complete object reconstruction on mobile devices is crucial to ensuring the accuracy of the established 3D model.

[0079] In related technologies, when a terminal detects a target object to be modeled, it first confirms the distance between the target object and the terminal. If the distance meets the requirements, the user is further prompted to maintain this distance while scanning around the target object. If the scanning speed is detected to be too fast during the scanning process, the user is prompted to slow down the scanning speed, thus ultimately obtaining the scanning information of the target object. Finally, a 3D model of the target object is built based on the scanning information.

[0080] The relevant technologies have the following three technical problems:

[0081] Technical Issue 1: The relevant technology triggers scanning of the target object when the distance between the target object and the terminal meets certain conditions. If other objects exist around the target object, the information of these other objects is also used as scanning information for the target object to detect its 3D model. In this case, the detected 3D model will deviate significantly from the actual shape of the target object. Therefore, the accuracy of the reconstructed 3D model is low, and the object reconstruction effect is poor.

[0082] Technical Issue 2: During the scanning process, the distance between the target object and the terminal should be between 20cm and 80cm to ensure the accuracy of the scanned information and, consequently, the accuracy of the 3D model. However, within this 20cm-80cm range, the entire target object may not be scanned, resulting in only a localized area. Consequently, the established 3D model is only a model of a localized area, leading to inflexible object reconstruction methods and poor reconstruction results.

[0083] Technical Issue 3: The relevant technology only prompts the user to slow down the scanning speed when it detects that the scanning speed is too fast. This indicates that the prompts provided by the technology are poorly guiding for the user, potentially leading to incorrect operations while the user moves the device around the target object, resulting in the failure of 3D model creation. Therefore, this not only affects the success rate and efficiency of object reconstruction but also degrades the user experience.

[0084] This application provides a method for object reconstruction, which can be applied to, for example... Figure 1 The terminal shown includes a data acquisition system 11, a data processing system 12, a display system 13, and a data storage system 14. The data acquisition system 11 collects data related to the target object to be modeled and inputs the collected data into the data processing system 12. The data processing system 12 builds a 3D model of the target object based on the input data. The display system 13 displays the built 3D model and the interaction information between the terminal and the user. The data storage system 14 stores the data collected by the data acquisition system 11 and the 3D model built by the data processing system 12.

[0085] Furthermore, see still Figure 1 The data acquisition system 11 includes a camera 1101 and an inertial measurement unit (IMU) 1102. The camera 1101 can be an array camera, which can be a color camera or include both a color camera and a depth camera. The color camera is used to acquire color images of the target object in red-green-blue (RGB) color mode, and the depth camera is used to acquire depth maps of the target object. In practice, the depth camera can be a time-of-flight (TOF) camera or a stereo camera.

[0086] In addition, the IMU1102 collects the terminal's rotational angular velocity through its built-in gyroscope and the terminal's acceleration through its built-in accelerometer, thereby calculating the terminal's position, attitude, and velocity. It can be seen that the data collected by the data acquisition system 11 includes: a color image (and depth image) of the target object acquired by the camera 1101, and the terminal's position, attitude, and velocity acquired by the IMU1102.

[0087] The display system 13 can use a liquid crystal display (LCD) screen, a light emitting diode (LED) screen, or an organic light-emitting diode (OLED) screen. This embodiment does not limit the display system 13. The data processing system 12 includes the following modules: a feature position detection module 1201, a feature position verification module 1202, a reference scan information recommendation module 1203, a modeling module 1204, a missing region feedback module 1205, a trajectory comparison module 1206, a scan evaluation module 1207, and a model post-processing module 1208.

[0088] Next, combined Figure 1 The system and modules shown illustrate the method for object reconstruction:

[0089] The data acquisition system 11 is used to acquire data through the camera 1101 and the IMU 1102, and the data storage system 14 stores the acquired data.

[0090] The feature location detection module 1201 is used to determine whether the target object has feature information based on the data collected by the data acquisition system 11.

[0091] The feature position verification module 1202 is used to detect whether the terminal's pose information relative to the target object meets the requirements based on the feature information of the target object obtained by the feature position detection module 1201. If the terminal's pose information relative to the target object meets the requirements, then formal scanning is allowed, that is, the data acquisition system 11 is triggered again to acquire the first scan information obtained from scanning the target object. Otherwise, the user is guided to adjust the terminal to a more suitable pose state.

[0092] The data acquisition system 11 is used to acquire the first scan information obtained from scanning the target object.

[0093] The reference scan information recommendation module 1203 is used to determine reference scan information based on the feature information of the target object. The determined reference scan information can be displayed through the display system 13, thereby guiding the user.

[0094] The modeling module 1204 is used to synchronously build a 3D model of the target object based on the first scan information obtained from the scan, and the display system 13 displays the 3D model. In addition, the data storage system 14 can store the 3D model.

[0095] The missing area feedback module 1205 provides real-time feedback on the areas of the currently established 3D model and the areas not yet modeled, guiding the user to scan the complete target object. The display system 13 can display the areas of the established 3D model and the areas not yet modeled.

[0096] The trajectory comparison module 1206 is used to compare and display the actual scanning trajectory and the reference scanning trajectory through the display system 13, so as to help users improve their scanning skills.

[0097] The scanning evaluation module 1207 is used to evaluate the scanning operation based on the actual scanning information and the reference scanning information, obtain scanning evaluation information, and display the obtained scanning evaluation information through the display system 13.

[0098] The model post-processing module 1208 is used to process the 3D model established by the modeling module 1204 and display the processed 3D model through the display system 13. This module can run synchronously with the trajectory comparison module 1206 and the scan evaluation module 1207.

[0099] It should be noted that, among the above modules, the feature position acquisition module 1201 and the feature position verification module 1202 are the core modules of this embodiment. The process involved in the data acquisition system 11 - feature position detection module 1201 - feature position verification module 1202 is the core process of this embodiment.

[0100] Based on the above Figure 1 As shown in the example, this embodiment provides a method for object reconstruction, which is applied to terminals such as... Figure 2 As shown, the method includes:

[0101] Step 201: When the terminal detects the target object, it performs feature location detection on the target object. Feature location detection is used to obtain the feature information of the target object.

[0102] The terminal can provide an application (APP) for object reconstruction and display an APP identifier on the terminal interface. The APP identifier can include one or both text and icons. When the APP identifier is selected, the terminal interface can jump to the user interface (UI) within the APP. Then, the APP application can call the terminal's camera and the terminal's built-in IMU to trigger the detection of the target object.

[0103] In implementation, the camera used by the application is related to the algorithm employed for object modeling. If the algorithm is RGB-based, the application uses a color camera to capture a color image of the target object. If the algorithm is based on red-green-blue depth (RBD), the application uses both a color camera and a depth camera to obtain a color image and a depth image of the target object. Since the color and depth images are captured simultaneously (or at very short intervals), their content can be considered identical. Each pixel in the depth image has a depth value, which represents the distance between the corresponding pixel's location on the target object and the optical center of the terminal camera.

[0104] During target object detection, the user can be prompted via UI to aim the camera at the target object to be modeled. Prompts can be one or more of the following: text prompts, voice prompts, and animated demonstrations. If the IMU detects the terminal's rotational angular velocity and acceleration, it indicates that the user is moving the terminal according to the prompts, allowing the camera to further capture an image of the target object for detection. In this embodiment, when the IMU detects terminal movement and the camera captures an image of the target object, the detection of the target object is confirmed.

[0105] It should be noted that when the camera captures images of the target object, it can continuously capture images within a reference time period to obtain a video containing multiple images, or it can capture one or more images at intervals. This embodiment does not limit the camera's capture method.

[0106] After confirming the detection of a target object, a detection algorithm can be used to detect the feature location of the image of the target object captured by the camera, thereby determining whether the target object possesses feature information at the feature location. If feature information is detected, the object type of the target object can be determined, facilitating the subsequent acquisition of the target object's feature information. It should be noted that the feature location varies for different target objects. In an exemplary embodiment, if the target object is a human body, the feature location of the target object can be the face (i.e., a human face). Therefore, if facial information matching human characteristics is detected in the image of the target object, such as facial features, facial contour information, etc., it can be determined that the target object has a human face, thus determining that the target object is a human body. Alternatively, if the target object is an animal, the feature location of the target object can also be the face. If the target object is an object other than a human body or animal, such as a teacup, the feature location can be the handle of the teacup.

[0107] In practice, the app used for 3D modeling can be an app targeting a specific type of object. In this case, if the feature information detected by the app does not match the target object type, the app can prompt the user through the UI to guide the user to point the camera at the target object of that type. For example, if the target object is a human body, and no facial information matching human features is detected in the image of the target object, it can be determined that the target object is not a human body, indicating that the user has pointed the camera at an animal or object other than a human body. Therefore, see [link to relevant documentation]. Figure 3 The system can prompt users via the UI that "no face detected," thus guiding them to refocus the camera on the human body.

[0108] Step 202: If the feature information of the target object is detected, a scan is performed around the local area of ​​the target object to obtain the first scan information of the target object.

[0109] In this embodiment, different types of target objects have different feature information. Taking the human body as an example, the feature information obtained through feature location detection is facial information, which includes, but is not limited to, facial features, facial contours, facial dimensions, facial angles, and depth values ​​of each pixel in the facial image. Regardless of the type of feature information obtained, a scan can be performed directly around a local area of ​​the target object after obtaining the feature information to obtain the first scan information of the target object. Alternatively, this embodiment also provides the following optional method to determine the timing of obtaining the first scan information: if feature information of the target object is detected, the pose information of the terminal relative to the target object is determined based on the feature information. The pose information includes one or both of position information and posture information. If the pose information meets the conditions, a scan is performed around a local area of ​​the target object to obtain the first scan information of the target object.

[0110] The position information of the terminal relative to the target object includes one or both of the target object's feature position on the UI and the distance between the target object's feature position and the terminal. The pose information of the terminal relative to the target object includes the angle of the terminal relative to the target object's feature position. The position information and pose information will be explained below:

[0111] To determine the location of a target object's features on the UI, the region of interest (ROI) for that feature location can be determined first. If the ROI of the target object's features lies within a reference area on the UI, then the location of the target object's features on the UI satisfies the condition. In implementation, the reference area can be identified in the UI using one or more of the following: text, shadow, color, or boundary lines. The reference area can be a region slightly above the center of the UI, or it can be another region within the UI; this embodiment does not limit the location or area of ​​the reference area.

[0112] When determining the Region of Interest (ROI) of a target object, it can be based on the object's feature information. Taking a human body as the target object and facial information as the feature information, the complete face of the target object can be directly determined based on the facial features and contour information, and this complete face can be used as the target object's facial ROI. If the complete face of the target object is within the aforementioned reference range, then the position of the target object's face on the UI is considered to meet the conditions. Alternatively, the facial features of the target object can be determined based on their information. One or more features can be selected from the facial features according to actual needs or experience to determine the target object's facial ROI. For example, eyebrows, eyes, nose, and mouth can be determined as the target object's facial ROI. If the eyebrows, eyes, nose, and mouth of the target object are within the aforementioned reference range, then the position of the target object on the UI is considered to meet the conditions.

[0113] Correspondingly, if the location of the target object's features on the UI does not meet the conditions, the user can be prompted accordingly. See also Figure 4 Taking a human face as an example, prompts such as "Please align your face with the reference area" and "Please frame the face" can be displayed in the UI to remind the user to adjust the terminal so that the target object's face is in the reference area on the UI.

[0114] Furthermore, according to the description in step 201, the image of the target object captured by the camera may be a video containing multiple images, or it may be one or more images captured at intervals. Therefore, before determining the feature location (ROI) of the target object, a target image can be selected from the video or multiple images, and the corresponding feature information of the target image can be determined from the detected feature information. Thus, the feature location (ROI) can be determined based on the feature information corresponding to the target image. Taking facial information as an example, in implementation, the image closest to the terminal can be selected from the video or multiple images as the target image based on the facial dimensions and the depth values ​​of each pixel in the facial image. Alternatively, the image with the highest clarity can also be selected as the target image. This embodiment does not limit the selection of the target image.

[0115] Whether the distance between the target object's feature location and the terminal meets the condition can be confirmed based on the depth values ​​of each pixel in the target object's image. In implementation, valid points can be first identified from the image's pixels; valid points are pixels with a depth value greater than 0, while pixels with a depth value of 0 or less are considered invalid. Then, the proportion of valid points among all pixels in the image is used as the target proportion, or the proportion of valid points in the region of interest (ROI) among all pixels corresponding to the ROI is used as the target proportion. The distance between the target object and the terminal is then determined as follows:

[0116] If the target proportion is not lower than the reference proportion (e.g., 30%), then the distance between the feature location of the target object and the terminal is obtained. For example, the average depth value corresponding to the effective points can be used as this distance, or the average depth value of the pixels corresponding to the key points of the feature location in the image captured by the camera can be used as this distance. This embodiment does not limit the method of obtaining the distance between the feature location of the target object and the terminal. Taking the feature location as a face as an example, the key points of the feature location are facial key points, which include but are not limited to facial features and facial contours.

[0117] After obtaining the distance, if it falls within the reference range (e.g., 40cm-60cm), the distance between the target object and the terminal can be confirmed to meet the condition. If the distance is less than the lower limit of the reference range, it indicates that the distance between the target object and the terminal is too close. The user can be prompted via the UI to increase the distance between the target object and the terminal, for example, by displaying a message. Figure 5 The system can display a "too close" warning. If the distance exceeds the upper limit of the reference range, it indicates that the distance between the target object and the terminal is too far. The UI can then prompt the user to reduce the distance between the target object and the terminal, for example, by displaying a message indicating that the distance is too close. Figure 6 The system will display a "too far" message to indicate this.

[0118] If the target ratio is lower than the reference ratio, the distance between the target object and the terminal can be determined based on the number of far and near points among the valid points. Far and near points are defined relative to the average distance between the target object's key feature points and the terminal; distances greater than this average distance are considered far points, and distances less than this average distance are considered near points. If the number of near points is greater than the number of far points, it indicates that the distance between the target object and the terminal is relatively short, and the distance can be determined based on the above... Figure 5 The method described above should be used to prompt the user. If the number of distant points is greater than the number of nearby points, it indicates that the distance between the target object and the terminal is relatively far, and can be addressed according to the above. Figure 6 The method used to prompt the user.

[0119] Whether the angle of the terminal relative to the target object's feature position meets the conditions can be determined directly based on the feature information. Taking the target object as a human body and the feature information as facial information as an example, the angle of the terminal relative to the target object's face can be determined based on the facial angle information. This facial angle information includes, but is not limited to, the pitch angle, yaw angle, and roll angle of the terminal relative to the target object's face. When the terminal is directly facing the target object's face, the pitch angle, yaw angle, and roll angle are all 0. When the terminal is slightly lower or higher than the facing position (or the terminal remains stationary while the target object tilts its head down or up), a pitch angle is formed relative to the terminal; when the terminal is slightly left or right relative to the facing position (or the terminal remains stationary while the target object shakes its head left or right), a yaw angle is formed relative to the terminal; when the terminal is tilted left or right relative to the facing position (or the terminal remains stationary while the target object tilts its head left or right), a roll angle is formed relative to the terminal.

[0120] In implementation, the angle of the terminal relative to the target object can be determined to meet the condition if the pitch, yaw, and roll angles are all no greater than the reference angle. It should be noted that pitch, yaw, and roll angles each correspond to two directions; as long as the angles in both directions are no greater than the reference angle, the angle relative to the terminal is considered to meet the condition. Taking pitch angle as an example, a terminal moving from a directly facing position to a slightly downward position creates a pitch angle, and moving from a directly facing position to a slightly upward position creates a pitch angle. When both pitch and roll angles are no greater than the reference angle, the terminal can actually rotate relative to the target object's face within a sector area centered at twice the reference angle, thus ensuring the terminal's pitch angle relative to the target object meets the condition. If any of the pitch, yaw, or roll angles is greater than the reference angle, the user can be prompted to adjust the terminal to a position directly facing the target object's face. For example, see... Figure 7 The UI can display "Please turn to the front view" to prompt the user.

[0121] Furthermore, in addition to the terminal's pose information relative to the target object, when the target object is a human body, this embodiment can also detect the facial features of the target object. If the facial features of the target object meet the requirements, the first scan information of the target object is then obtained. For example, if the eyes are selected as the target facial features, and the shape of the eyes meets the requirements, then the shape of the target object is determined to meet the requirements. In practice, the distance between the upper and lower eyelids can be determined based on the facial feature information, and this distance can be used to determine whether the target object has closed its eyes. Alternatively, the pupil area can be used to determine whether the target object has closed its eyes. If the target object has not closed its eyes, then the shape of the eyes is determined to meet the requirements, thereby further determining that the facial features of the target object meet the conditions. If the target object has closed its eyes, such as Figure 8As shown, the UI can display prompts such as "Please open your eyes" or "You have closed your eyes" to remind the target to open their eyes.

[0122] It can be seen that the information required to be determined before executing the step of obtaining the first scan information includes, but is not limited to, the position of the target object's feature location on the UI, the distance between the target object's feature location and the terminal, the angle of the terminal relative to the target object's feature location, and one or more of the target object's facial features. The process of determining whether each piece of information meets the conditions can be performed simultaneously or sequentially according to a reference order. For example, when the target object is a human body, such as... Figure 9 As shown, the steps can be performed sequentially in the order of steps 901-907, 9071 or 9072, and 908-910.

[0123] In this embodiment, after determining that the pose information meets the conditions, a scan of a local area around the target object can be triggered to obtain the first scan information of the target object. By determining that the pose information meets the conditions, the position of the terminal relative to the target object is appropriate before the formal scan, and the terminal's pose is also more consistent with the pose of the feature positions facing the target object. Therefore, the feature positions of the 3D model built based on the first scan information obtained from the formal scan are more accurate and richer in detail. See also Figure 10 A scan button can be displayed in the UI, which can be identified by words such as "Start," "START," or "CAPTUER" to prompt the user to select the scan button to begin the scan. If the scan button is selected, the UI will prompt the user to remain still in the current state and trigger the acquisition of the first scan information for the target object.

[0124] The first scan information is used for object reconstruction. Since the object to be modeled often has rich details, it is necessary to obtain first scan information that meets the requirements to ensure the accuracy and completeness of the reconstructed 3D model. Based on the above considerations, this embodiment can play a demonstration video of the scan via the UI before formally entering the scanning process. Alternatively, this embodiment can also provide the user with reference scan information as a reference to obtain first scan information that meets the requirements. That is, in the exemplary embodiment, before scanning a local area around the target object to obtain the first scan information of the target object, the method provided in this embodiment further includes: obtaining reference scan information, which includes one or more of the following: reference scan trajectory, reference scan speed, and reference scan posture; and outputting the reference scan information, which is used to guide the scanning of a local area of ​​the target object.

[0125] In implementation, default reference scanning information can be directly provided based on feature information. For example, when the reference scanning information includes a reference scanning trajectory, the reference scanning trajectory can be a reference scanning trajectory that takes the feature position of the target object as the starting point based on the feature information. Alternatively, targeted reference scanning information can be planned based on feature information. In an exemplary embodiment, obtaining reference scanning information includes the following steps A1-A3:

[0126] Step A1: Determine the core region of the target object based on the feature information and establish a feature location model of the target object.

[0127] The core area can be determined according to different types of target objects. For example, when the target object is a human body, the core area can be the smallest rectangular area including eyebrows, eyes, nose, and mouth. When the target object is an animal, the core area can be the smallest rectangular area including eyes, nose, and mouth. When the target object is a teacup, the core area can be the smallest rectangular area including the handle. This embodiment does not limit the method of determining the core area or the area of ​​the core area. In addition, for the feature position model of the target object, a camera can be used to collect the feature positions of the target object to obtain the feature position scanning information of the target object, thereby establishing the feature position model of the target object from the feature position scanning information.

[0128] Step A2: Determine the completeness information of the feature location model based on the core region.

[0129] Completeness information is used to indicate the degree of completeness of the feature location model. By determining the completeness information of the feature location model, it is possible to identify which regions of the feature location model are more complete (i.e., easier to model) and which regions are less complete (i.e., more difficult to model), so as to plan appropriate reference scan information based on the completeness of different regions.

[0130] In an exemplary embodiment, the determination method includes: determining an upper region, a lower region, a left region, and a right region on the feature location model, centered on the core region. The normalized completeness corresponding to each region is obtained, and this normalized completeness is used as the completeness information of the feature location model.

[0131] Taking the human body as an example, the core area is the smallest rectangular area including the eyebrows, eyes, nose, and mouth as described in step A1. Correspondingly, the upper area defined by the core area can include the forehead and top of the head, the lower area includes the chin, and the left and right areas include the cheekbones and ears.

[0132] Furthermore, normalized completeness indicates the degree of completeness of a region on the feature location model; a higher normalized completeness value indicates a higher degree of completeness for that region. The normalized completeness for each of the five regions can be determined using a two-dimensional planar density method, a three-dimensional spatial volume density method, or a distance-weighted method. The two-dimensional planar density method determines the number of effective point clouds per unit pixel area in each region; a higher number of effective point clouds results in a higher normalized completeness value. The three-dimensional spatial volume density method determines the number of effective point clouds per unit volume in each region; the normalized completeness value increases with the number of effective point clouds. The distance-weighted method determines the distance between each pixel within the region and the terminal, and determines the normalized completeness value based on the weighted value of each distance.

[0133] Regardless of the method used to determine the normalized completeness of each region, the normalized completeness of each region can be used as the completeness information of the feature location model, thereby triggering the determination of the reference scan information.

[0134] Step A3: Determine the reference scan information based on the integrity information.

[0135] The method for determining the reference scan information based on its completeness varies depending on the information included in the reference scan. Next, the normalized completeness corresponding to the core region, upper region, lower region, left region, and right region will be represented as I. center I top I down I left I right The following explains three different cases of reference scan information, including reference scan trajectory, reference scan speed, and reference scan attitude:

[0136] When the reference scan information includes a reference scan trajectory, it can be based on I center To determine the scanning distance between the terminal and the target object. If I canter If the scan distance is less than the first threshold, it indicates that the core region in the feature location model has a low degree of integrity and is more difficult to model. Therefore, a smaller scan distance is determined to ensure that the scan obtains richer and more accurate scanning information, thereby better modeling the core region in the feature location model. This smaller scan distance can be obtained by shortening the reference distance. For example, if the reference distance is 40-60cm, the smaller scan distance can be 35-55cm. Conversely, if the scan distance is greater than the threshold, it indicates that the core region has a high degree of integrity and is less difficult to model, so a larger scan distance can be determined.

[0137] Additionally, it can also be based on I top and I downDetermine the scan height and the number of scans. If I top If the value is less than the second threshold, the integrity of the upper region in the feature location model is considered low. Therefore, the scanning height can be increased, meaning the reference scanning trajectory must be moved towards the upper part of the target object (e.g., the top of the head when the target object is a human body). Furthermore, the number of scans of the upper part of the target object can be increased by one or more times, with the increase determined by I. top The difference between I and the second threshold is determined; the larger the difference, the more times it increases. Correspondingly, if I... down If the value is less than the third threshold, the integrity of the lower region in the feature location model is considered to be low. In this case, the scanning height can be reduced so that the reference scanning trajectory moves towards the lower part of the target object.

[0138] According to I left and I right The scanning direction can be determined. Wherein, if I left Greater than I right This indicates that the left region of the feature location model is more complete than the right region. Therefore, the scanning direction can be to start from the right side, circle the target object, and reach the left side, taking the side with lower completeness as the starting side of the scanning direction. Correspondingly, if i right Greater than I left The scanning direction is to start from the left, circle around the target object, and reach the right side.

[0139] After determining the scanning spacing, scanning height, number of scans, and scanning direction, a scanning trajectory that satisfies the determined scanning spacing, scanning height, number of scans, and scanning direction can be obtained. The obtained scanning trajectory is then used as a reference scanning trajectory, thereby determining the reference scanning trajectory.

[0140] When the reference scan information includes a reference scan speed, the reference scan speed can be determined based on the normalized completeness corresponding to each region. For example, for any region, if the normalized completeness corresponding to that region is less than a fourth threshold, the scan speed for that region can be reduced to obtain richer and more accurate scan information through a slower scan speed. Finally, the reference scan speed is determined by combining the scan speeds corresponding to each region. This reference scan speed can be the average of the scan speeds corresponding to each region, or different scan speeds can be applied to different regions.

[0141] When the reference scan information includes a reference scan posture, for any region, if the normalized completeness of the region is less than the fifth threshold, the reference scan posture when scanning the region can be determined as a more stringent posture, such as maintaining a vertical state, so as to obtain more accurate scan information by scanning with such a stringent reference scan posture.

[0142] It should be noted that if the reference scan information includes more than two types of information, they can be combined according to the above instructions, and will not be repeated here.

[0143] The reference scan trajectory obtained using the method described above can be found in [reference]. Figure 11 , Figure 11 This method is applicable when the target object is a human or animal. As can be seen, the reference scanning trajectory starts from a position facing the target object's face and circles the upper body of the target object in a clockwise or counter-clockwise circular motion (clockwise in the figure). After returning to the position facing the target object's face, it extends upwards and ends at the top of the target object's head. The distance between any point on the reference scanning trajectory and the target object can be between 40cm and 60cm, ensuring that the scanning information obtained according to the reference scanning trajectory is relatively accurate and rich, thus enabling the 3D model obtained from the scanning information to contain detailed head structures. Of course, this embodiment does not limit the reference scanning trajectory; it can be planned according to actual needs, provided that a complete scan of a local area of ​​the target object can be ensured. This embodiment also does not limit the distance between the reference scanning trajectory and the target object; it can be selected according to needs or experience.

[0144] Regardless of the type of reference scan information acquired, it can be output through the UI, for example, based on one or both text and sound. For instance, when the reference scan information includes a reference scan speed, "slow (or fast) moving scan" can be output based on the determined reference scan speed. When the reference scan information includes a reference scan posture, "keep the phone vertical as much as possible" can be output. In an exemplary embodiment, when the reference scan information includes a reference scan trajectory, the reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories. Accordingly, outputting the reference scan information includes: sequentially displaying the reference number of sub-scan trajectories according to the terminal's position.

[0145] Because the reference scan trajectory needs to ensure a complete scan of a local area of ​​the target object, it often needs to circle the local area at least once. If the acquired reference scan trajectory is displayed directly, the displayed reference scan trajectories may overlap. This not only makes the UI less concise but also affects user understanding and reduces the user guidance effect. Therefore, the reference scan trajectory is divided into a reference number of non-overlapping sub-scan trajectories. For example, the reference scan trajectory can be directly divided N times to obtain (N+1) sub-scan trajectory segments. Then, the terminal position can be continuously detected, or the terminal position can be detected once every reference time, so that the reference number of sub-scan trajectories are displayed sequentially according to the terminal position during the user's scanning process. In this embodiment, the reference time can be set according to actual needs or experience; for example, the reference time can be set to 0.1 seconds.

[0146] In implementation, after the reference scan trajectory is obtained, the first sub-scan trajectory is displayed first, and the starting point of the first sub-scan trajectory is the starting point of the reference scan trajectory. Then, when the terminal's position is detected to have reached the ending point of the first sub-scan trajectory, the second sub-scan trajectory, with the ending point of the first sub-scan trajectory as its starting point, can be displayed. This process continues until the last sub-scan trajectory is displayed, and the ending point of the last sub-scan trajectory is the ending point of the reference scan trajectory.

[0147] It should be noted that, in this embodiment, after displaying a sub-scan trajectory, the terminal position and attitude can be detected, and the subsequent sub-scan trajectories can be updated or replanned based on the terminal position and attitude to obtain an updated sub-scan trajectory. Then, after detecting that the terminal position has reached the end point of the currently displayed sub-scan trajectory, the updated sub-scan trajectory can continue to be displayed.

[0148] In addition, this embodiment allows for text annotation of each displayed sub-scan trajectory, for example... Figure 12 "Follow the arrow to move" and Figure 13 The "Fill in the top" feature allows for the display of animations on the sub-scan tracks, such as dynamic light effects that slide from the start to the end of the track, enhancing user guidance.

[0149] Furthermore, after outputting the reference scan information, the user begins scanning the target object, which can be scanned according to... Figure 14The user can be scanned while seated or standing. The initial scan information of the target object is obtained through the user's actual scanning process. In an exemplary embodiment, after displaying the output scan information, the method further includes: detecting the actual scan information of the terminal, which includes one or more of the following: actual scan trajectory, actual scan speed, and actual scan posture. If the deviation between the actual scan information and the reference scan information is greater than a reference threshold, a prompt message is output based on the deviation, prompting the user to correct the actual scan information.

[0150] The information included in the reference scan information corresponds to the information included in the actual scan information. For example, if the actual scan information includes the actual scan trajectory, the reference scan information will also include the reference scan trajectory accordingly. If the deviation between any information included in the actual scan information and the reference scan information is greater than a reference threshold, a prompt message can be output based on the deviation to remind the user to correct the actual scan information.

[0151] The deviation between the actual scan trajectory and the reference scan trajectory includes: the actual distance between the actual scan trajectory and the target object deviates from the reference distance between the reference scan trajectory and the target object. In this case, the reference threshold is the distance threshold. If the deviation obtained by subtracting the reference distance from the actual distance is greater than the distance threshold, it indicates that the distance between the terminal and the target object is too far, and therefore, refer to... Figure 15 Output a "Current distance too far" message, which may include one or both of the following: text and audio. If the deviation between the reference distance and the actual distance exceeds a distance threshold, refer to [the relevant documentation / reference]. Figure 16 Output the message "Current distance too close".

[0152] Additionally, if the deviation of the actual scan trajectory from the reference scan trajectory in the direction of gravity exceeds a distance threshold, a prompt to "move the phone upwards" will be output. If the deviation of the actual scan trajectory from the reference scan trajectory in the opposite direction of gravity exceeds a distance threshold, a prompt to "move the phone downwards" will be output. If the deviation between the actual scan trajectory and the reference scan trajectory is not greater than the distance threshold, but the frequency of deviation is high, a prompt to "keep the phone stable" will be output.

[0153] Regarding the actual scan speed and the reference scan speed, if the deviation between the actual scan speed and the reference scan speed is greater than the speed threshold, it indicates that the terminal's actual scan speed is relatively fast, and thus it can be referenced. Figure 17 Output the message "Current speed too fast". If the difference between the reference scan speed and the actual scan speed is greater than the speed threshold, it indicates that the actual scan speed of the terminal is too slow, and the message "Current speed too slow" can be output.

[0154] If the deviation between the actual scanning posture and the reference scanning posture used by the user during the scanning process is greater than the angle threshold, the local area to be scanned in the target object will not be completely scanned. Therefore, it is necessary to output a prompt message based on the deviation between the actual scanning posture and the reference scanning posture to prompt the user to correct the actual scanning posture.

[0155] As explained above, both the actual scanning attitude and the reference scanning attitude include pitch angle, yaw angle, and roll angle. When the deviation of the pitch angle exceeds the angle threshold, the output message can be "Pitch angle too large" or something similar. Figure 18 The error message "Overshoot angle too large" is shown. When the deviation corresponding to the yaw angle exceeds the angle threshold, the output message can be "Too much left roll angle" or "Too much right roll angle". When the deviation corresponding to the roll angle exceeds the angle threshold, the output message can be "Too much left turn angle" or "Too much right turn angle". Additionally, if the deviation between the actual scanning attitude and the reference scanning attitude causes a local area of ​​the target object to be completely undetectable, please refer to [further details needed]. Figure 19 Output the message "Unable to scan object".

[0156] Considering that the first scan information is obtained by scanning a local area around the target object, this embodiment also provides the following method to facilitate the final acquisition of a three-dimensional model of the entire target object:

[0157] In one optional implementation, after obtaining the first scan information of the target object through the user's actual scanning process, the method further includes: determining a target region of the target object based on a local region of the target object, wherein the target region includes at least a portion of other regions on the target object besides the local region; and obtaining the second scan information corresponding to the target region.

[0158] The target region determined based on the local region can be all regions of the target object except for the local region, or at least a portion of all regions except the local region. Taking the upper body of the human body as an example, the target region can be the lower body of the human body, or the lower body excluding the feet. After determining the target region of the target object, a second scan information obtained by scanning around the target region of the target object can be further acquired.

[0159] In this embodiment, a supplementary scanning trajectory for guiding the scanning of a local region of a target object can be determined based on a reference scanning trajectory used to guide the scanning of that target region. The reference scanning trajectory is as follows: Figure 11 As shown, taking the lower half of the human body as an example, the corresponding supplementary scanning trajectory can be found in [reference needed]. Figure 20The supplementary scan trajectory includes: moving the terminal downwards from the end point of the reference scan trajectory to a reference height above the ground (e.g., 0.8-1.2m), and moving it away from the target object to a reference full scan distance (e.g., 1-1.5m). Then, maintaining the reference height and reference full scan distance, it circles the lower body once. The complete trajectory consisting of the reference scan trajectory and the supplementary scan trajectory can be found in [reference needed]. Figure 21 .

[0160] It should be noted that this embodiment does not limit the aforementioned reference height and reference full-scan distance, as long as the area scanned at the reference height and reference full-scan distance includes both the complete target area and the portion overlapping with the local area. For example, if the local area is the upper body from the top of the target object's head to its hips, and the target area is the lower body from the target object's hips to its feet, then the area scanned at the reference height and reference full-scan distance may include the area from the waist to the hips in the local area, as well as the complete lower body.

[0161] Alternatively, the reference full-scan distance can be calculated as follows: Obtain the overall height of the target object and the camera intrinsics of the terminal camera, and calculate the minimum full-scan distance between the terminal and the target object based on the triangular relationship formed by the pinhole camera model. That is, the actual distance between the terminal and the target object must be greater than the minimum full-scan distance to ensure that the entire target object is scanned. Then, the reference full-scan distance is determined based on the calculated minimum full-scan distance. In practice, an overall image of the target object can be acquired, and the overall image can be converted to three-dimensional space to obtain the overall height of the target object. Alternatively, the overall height of the target object can be calculated using a three-dimensional model built based on the first scan information. The process of building a three-dimensional model based on the first scan information can be found in step 203 below.

[0162] Additionally, after obtaining the supplementary scan trajectory, it can be displayed in segments, and text can be used to guide the user in scanning the target area of ​​the target object. For example, see... Figure 22 The "scan the whole body from a distance" and Figure 23 The "follow the arrow" function guides the user.

[0163] After obtaining the second scan information corresponding to the target area, if the target object includes other unscanned areas besides the local area and the target area, a third scan information can be obtained. This third scan information is obtained by the user freely scanning the unscanned areas. In implementation, the UI can provide stop buttons labeled "Complete," "End," or "STOP" to guide the user to click the stop button after completing the scan. If the stop button is selected, it indicates that the user's actual scanning process has ended, and subsequent processing can be triggered.

[0164] Step 203: Establish a three-dimensional model of the target object based on the first scan information.

[0165] As explained above, once the user begins scanning the target object, the first scan information is obtained. In this embodiment, a 3D model of the target object can be simultaneously built based on the first scan information. Taking the upper body of the target object as an example, when the face of the target object is scanned, a 3D model of the face is built. When the scan moves from the face to the top of the head, a 3D model of the top of the head is added to the already built 3D facial model. After scanning all positions of the upper body of the target object, a half-body 3D model of the target object is formed. Alternatively, the 3D model of the target object can be built after all the first scan information is obtained. For example, after scanning the face and top of the head of the target object in sequence, all the first scan information is obtained, and then a half-body 3D model of the target object is built based on all the first scan information.

[0166] It should be noted that the first scan information includes RGB (or RGBD) information acquired by the camera, as well as rotation angle and acceleration information acquired by the IMU. During the process of building the 3D model, the terminal pose is estimated based on the information acquired by the IMU, and then the RGB (or RGBD) information and the estimated terminal pose are fused using a 3D model fusion algorithm to complete the construction of the 3D model.

[0167] In the process of establishing a 3D model of the target object based on the first scan information, the method provided in this example also includes: determining the scanned and unscanned areas of the target object; displaying the projected image of the target object; and marking the scanned and unscanned areas on the projected image of the target object, respectively.

[0168] If a 3D model is created simultaneously during the scanning process, the area with the created 3D model is the scanned area. Therefore, the scanned area can be determined by projecting the area with the created 3D model onto the terminal using a ray casting algorithm. Alternatively, if the 3D model is created after the scanning process is complete, the scanned area can be determined based on the actual scanning trajectory. Afterward, the outline of the target object can be projected onto the terminal, thereby displaying the projected image of the target object on the UI. In the projected image of the target object, all areas except the scanned areas are unscanned areas, thus determining the unscanned areas. It should be noted that the projected image of the target object can be a projected image of a local area of ​​the target object. For example, if the local area is the upper body of a human body, then the scanned and unscanned areas are marked on the projected image of the upper body. Alternatively, the projected image of the target object can also be a projected image of a local area of ​​the target object and the projected image of the target area; this embodiment does not limit this.

[0169] After determining the scanned and unscanned areas, see Figure 24 Both scanned and unscanned areas can be marked on the projected image of the target object. In practice, the scanned and unscanned areas can be marked with different colors, different grid densities, or different types of fill patterns. This embodiment does not limit the marking method, as long as the marked scanned and unscanned areas are distinguishable.

[0170] For annotations of scanned areas in related technologies, please refer to [link to relevant documentation]. Figure 25 As can be seen, for any given moment during the scanning process, related technologies only annotate the scanned areas from the corresponding scanning perspective at that moment. Therefore, users can only see the modeling results at the current moment and are unaware of which areas of the target object have not yet been scanned. Compared to related technologies, the annotation method provided in this embodiment more intuitively displays the areas of the target object that have not yet been scanned. This not only helps users understand the current scanning progress but also triggers users to spontaneously move their devices to unscanned areas, thus achieving a better annotation effect.

[0171] As explained in step 202, after obtaining the first scan information of the target object, the second scan information corresponding to the target area within the target object can also be obtained. Therefore, accordingly, establishing a 3D model of the target object based on the first scan information includes: establishing a 3D model of the target object based on the first scan information and the second scan information. In implementation, taking the lower body of the target object as an example, the 3D model of the lower body can be simultaneously established during the acquisition of the second scan information. Again, according to the explanation in step 202, since the scanned area of ​​the second scan information partially overlaps with the local area corresponding to the first scan information, the 3D model of the lower body established based on the second scan information also has overlapping model parts with the 3D model of the upper body. Therefore, the 3D models of the upper and lower bodies can be merged based on the overlapping model parts, that is, the 3D model of the target object is supplemented based on the second scan information. Alternatively, after scanning the local area and the target area, obtaining the first scan information and the second scan information, the 3D model of the target object can be established based on the first scan information and the second scan information.

[0172] In addition, if the model obtained by combining the first scan information, the second scan information and the third scan information obtained by the user's free scanning mentioned in step 202 is a three-dimensional model with missing regions, this embodiment can also call a reconstruction algorithm to fill in the missing regions as needed.

[0173] In an exemplary embodiment, after establishing a 3D model of the target object based on the first scanning information, the method further includes: post-processing the 3D model to obtain a processed 3D model, and displaying the processed 3D model. The processing of the 3D model includes, but is not limited to, model filtering, meshing, texture mapping, and skin smoothing steps. Since the pose information of the terminal relative to the target object is determined to meet the requirements in step 202 before the formal scanning, texture mapping of the 3D model can be performed more effectively. That is, corresponding textures are applied to each face of the 3D model, resulting in a more aesthetically pleasing and realistic 3D model.

[0174] It should be noted that processing the 3D model takes time; therefore, in this embodiment, the UI displays content related to the user's actual scanning process to alleviate user anxiety during waiting. See the exemplary embodiment for details. Figure 26When the reference scan information includes a reference scan trajectory, this embodiment can display both the actual scan trajectory and the reference scan trajectory of the scanned target object, thereby intuitively showing the user the difference between the actual scan trajectory and the reference scan trajectory, and helping the user improve their scanning skills. In implementation, the actual scan trajectory and the reference scan trajectory can be displayed in different colors for easy differentiation. Alternatively, the actual scan trajectory and the reference scan trajectory can be displayed as different line types, such as displaying the actual scan trajectory as a solid line and the reference scan trajectory as a dashed line. During the display process, the complete actual scan trajectory and the reference scan trajectory can be displayed directly, or the actual scan trajectory and the reference scan trajectory can be replayed separately according to the direction from the trajectory start point to the trajectory end point. This embodiment does not limit the display method of the actual scan trajectory and the reference scan trajectory.

[0175] In addition to displaying the actual scan trajectory and the reference scan trajectory, in this exemplary embodiment, this embodiment can also obtain scan evaluation information for evaluating the scan situation based on the deviation between the actual scan information and the reference scan information, and display the scan evaluation information. The scan evaluation information may include trajectory evaluation information for evaluating the deviation between the actual scan trajectory and the reference scan trajectory, and may also include speed evaluation information for evaluating the deviation between the actual scan speed and the reference scan speed. The method for obtaining trajectory evaluation information and speed evaluation information can be found in the following formula:

[0176]

[0177]

[0178] The symbols in the formula are explained as follows:

[0179] G i —The score of the actual scan trajectory in the i-th lap, i = 1, 2

[0180] n i —Total number of actual scanning trajectory sampling points in the i-th cycle

[0181] P Real (i, j) — Position of the j-th sampling point on the actual scanning trajectory of the i-th circle.

[0182] P sugg (i, j) — Position of the j-th sampling point on the i-th reference scan trajectory.

[0183] S i —Trajectory evaluation information on the actual scan trajectory of the i-th cycle

[0184] S v —Speed ​​evaluation information

[0185] In the above explanation, the first loop refers to the scanning process around a local area of ​​the target object, and the second loop refers to the scanning process around the target area of ​​the target object. It can be seen that when the user's actual scanning trajectory includes more than two loops, a trajectory evaluation information can be determined for each loop of the actual scanning trajectory. See also... Figure 27 The acquired trajectory evaluation information and speed evaluation information can be displayed in the UI interface using a star rating system. Alternatively, they can be displayed numerically or as a progress bar. This embodiment does not limit the display format of the trajectory evaluation information and speed evaluation information.

[0186] It should be noted that this implementation does not limit the execution order of the processes of displaying the actual scan trajectory and reference scan trajectory, displaying scan evaluation information, and the post-processing process of the 3D model. In addition to the synchronous execution as described above, one or two of the above displays can be performed first, followed by the post-processing process of the 3D model, or the post-processing process of the 3D model can be performed first, followed by one or two of the above displays.

[0187] In summary, after detecting the target object to be modeled, this embodiment first determines whether the target object has feature information through feature location detection. If the target object is found to have feature information, a 3D model of the target object is then built based on the scan information of the target object. Therefore, the 3D model built in this embodiment closely matches the actual shape of the target object, has high accuracy, and achieves good object reconstruction results.

[0188] Furthermore, after obtaining the first scan information corresponding to a local area of ​​the target object, this embodiment can continue to obtain the second scan information corresponding to the target area outside the local area. Therefore, the local 3D model can be supplemented based on the second scan information to obtain a more complete 3D model, such as a full-body 3D model of the target object. It can be seen that the object reconstruction method of this embodiment is quite flexible.

[0189] Furthermore, this embodiment outputs real-time prompts to the user during the actual scanning process and visually displays the scanned and unscanned areas using projection. Therefore, it provides strong guidance for the user's scanning process, ensuring the accuracy of the scanned information and thus improving the completeness and accuracy of the reconstructed 3D model. In addition, while the user waits for the terminal to process the 3D model, it displays a comparison between the actual scanning trajectory and the reference scanning trajectory, as well as scanning evaluation information for the actual scanning process. This not only alleviates the user's waiting anxiety but also helps the user improve their scanning skills.

[0190] Based on the same concept, such as Figure 28 As shown in the illustration, this application also provides an apparatus for object reconstruction, the apparatus comprising:

[0191] The detection module 2801 is used to perform feature location detection on the target object when the terminal detects the target object. The feature location detection is used to obtain the feature information of the target object.

[0192] The acquisition module 2802 is used to scan a local area around the target object if feature information of the target object is detected, and obtain the first scan information of the target object.

[0193] Module 2803 is used to create a three-dimensional model of the target object based on the first scan information.

[0194] In an exemplary embodiment, if the target object is a human body, the feature location is the face.

[0195] In an exemplary embodiment, the apparatus further includes: a first determining module, configured to determine the pose information of the terminal relative to the target object based on the feature information if feature information of the target object is detected, wherein the pose information includes one or both of position information and posture information; and an acquiring module, configured to perform a scan around a local area of ​​the target object to obtain first scan information of the target object if the pose information satisfies the conditions.

[0196] In an exemplary embodiment, the apparatus further includes: a second determining module, configured to determine a target region of the target object based on a local region of the target object, the target region including at least a portion of other regions on the target object besides the local region; acquire second scanning information corresponding to the target region; and a building module 2803, configured to build a three-dimensional model of the target object based on the first scanning information and the second scanning information.

[0197] In an exemplary embodiment, the device further includes: a first display module, configured to acquire reference scanning information, the reference scanning information including one or more of reference scanning trajectory, reference scanning speed, and reference scanning posture; and output reference scanning information, the reference scanning information being used to guide scanning of a local area of ​​a target object.

[0198] In an exemplary embodiment, the first display module includes: an establishment unit, configured to determine the core region of a target object based on feature information and establish a feature position model of the target object; a first determination unit, configured to determine the completeness information of the feature position model based on the core region; and a second determination unit, configured to determine reference scanning information based on the completeness information.

[0199] In an exemplary embodiment, the first determining unit is configured to determine an upper region, a lower region, a left region, and a right region on the feature location model, with the core region as the center; obtain the normalized completeness corresponding to each region, and use the normalized completeness corresponding to each region as the completeness information of the feature location model.

[0200] In an exemplary embodiment, the second determining unit is configured to, when the reference scanning information includes a reference scanning trajectory, determine the scanning distance between the terminal and the target object based on the normalized completeness corresponding to the core region, determine the scanning height and the number of scans based on the normalized completeness corresponding to the upper and lower regions, and determine the scanning direction based on the normalized completeness corresponding to the left and right regions; and determine the reference scanning trajectory based on the scanning distance, scanning height, number of scans, and scanning direction.

[0201] In an exemplary embodiment, when the reference scan information includes a reference scan trajectory, the reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories. The first display module is used to sequentially display each sub-scan trajectory according to the position of the terminal.

[0202] In an exemplary embodiment, the device further includes: a second display module, configured to detect the actual scanning information of the terminal, the actual scanning information including one or more of the following: actual scanning trajectory, actual scanning speed, and actual scanning posture; if the deviation between the actual scanning information and the reference scanning information is greater than a reference threshold, displaying a prompt message based on the deviation, the prompt message being used to prompt correction of the actual scanning information.

[0203] In an exemplary embodiment, the apparatus further includes: a labeling module, configured to determine the scanned and unscanned areas of the target object; display a projected image of the target object, and label the scanned and unscanned areas on the projected image of the target object respectively.

[0204] In an exemplary embodiment, the apparatus further includes: a processing module for post-processing the three-dimensional model of the target object to obtain a processed three-dimensional model; and displaying the processed three-dimensional model.

[0205] In an exemplary embodiment, the apparatus further includes a third display module, configured to display the actual scan trajectory of the scanned target object and the reference scan trajectory when the reference scan information includes a reference scan trajectory.

[0206] In an exemplary embodiment, the apparatus further includes: a fourth display module, configured to obtain scan evaluation information based on the deviation between actual scan information and reference scan information, wherein the scan evaluation information is used to evaluate the scan status; and to display the scan evaluation information.

[0207] In summary, after detecting the target object to be modeled, this embodiment first determines whether the target object has feature information through feature location detection. If the target object is found to have feature information, a 3D model of the target object is then built based on the scan information of the target object. Therefore, the 3D model built in this embodiment closely matches the actual shape of the target object, has high accuracy, and achieves good object reconstruction results.

[0208] Furthermore, after obtaining the first scan information corresponding to a local area of ​​the target object, this embodiment can continue to obtain the second scan information corresponding to the target area outside the local area. Therefore, the local 3D model can be supplemented based on the second scan information to obtain a more complete 3D model, such as a full-body 3D model of the target object. It can be seen that the object reconstruction method of this embodiment is quite flexible.

[0209] Furthermore, this embodiment outputs real-time prompts to the user during the actual scanning process and visually displays the scanned and unscanned areas using projection. Therefore, it provides strong guidance for the user's scanning process, ensuring the accuracy of the scanned information and thus improving the completeness and accuracy of the reconstructed 3D model. In addition, while the user waits for the terminal to process the 3D model, it displays a comparison between the actual scanning trajectory and the reference scanning trajectory, as well as scanning evaluation information for the actual scanning process. This not only alleviates the user's waiting anxiety but also helps the user improve their scanning skills.

[0210] It should be understood that the above Figure 28 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0211] Based on the same concept, see Figure 29 This embodiment also provides an object reconstruction device 2900, which is used to perform the operations involved in the object reconstruction method described above. The device 2900 includes a memory 2901, a processor 2902, and an interface 2903, which are connected via a bus 2904.

[0212] The memory 2901 stores at least one instruction, which is loaded and executed by the processor 2902 to implement any of the object reconstruction methods described above. The interface 2903 is used to communicate with other devices on the network. This interface 2903 can be implemented wirelessly or via a wired connection; exemplarily, the interface 2903 can be a network interface card (NIC). For example, device 2900 can communicate with a server through this interface 2903.

[0213] It should be understood that, Figure 28 Only a simplified design of device 2900 is shown. In practical applications, device 2900 can contain any number of memories 2901, processors 2902, or interfaces 2903.

[0214] Based on the same concept, this embodiment also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method in any possible implementation of this embodiment.

[0215] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0216] Furthermore, in an alternative embodiment, the memory described above may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0217] The memory can be volatile or non-volatile, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0218] This application provides a computer program that, when executed by a computer, causes a processor or computer to perform the corresponding steps and / or processes in the above method embodiments.

[0219] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0220] The above description is merely an embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for object reconstruction, characterized in that, The method is applied to a terminal, and the method includes: When the terminal detects a target object, it performs feature location detection on the target object, and the feature location detection is used to obtain feature information of the target object; If feature information of the target object is detected, the core region of the target object is determined based on the feature information; Obtain scanning information of the feature positions of the target object, and establish a feature position model of the target object based on the scanning information of the feature positions; Centered on the core region, an upper region, a lower region, a left region, and a right region are determined on the feature position model; Obtain the normalized completeness corresponding to each region, and use the normalized completeness corresponding to each region as the completeness information of the feature location model; The scanning distance between the terminal and the target object is determined based on the normalized completeness corresponding to the core region, and the scanning height and number of scans are determined based on the normalized completeness corresponding to the upper region and the lower region. The scanning direction is determined based on the normalized completeness corresponding to the left region and the right region, and the starting side of the scanning direction is the region with the lower normalized completeness corresponding to the left region and the right region. A reference scanning trajectory is determined based on the scanning interval, the scanning height, the number of scans, and the scanning direction; Based on the normalized completeness of each region, the reference scan speed and reference scan pose for each region are determined respectively. Output reference scanning information, which includes the reference scanning trajectory, the reference scanning speed, and the reference scanning attitude. The reference scanning information is used to guide the scanning of a local area of ​​the target object. A scan is performed around a local area of ​​the target object to obtain the first scan information of the target object; A three-dimensional model of the target object is established based on the first scan information.

2. The method according to claim 1, characterized in that, If the target object is a human body, the feature location is the face.

3. The method according to claim 1 or 2, characterized in that, If the feature information of the target object is detected, the method further includes: The pose information of the terminal relative to the target object is determined based on the feature information, wherein the pose information includes one or both of position information and posture information; If the pose information meets the conditions, then the scanning of the local region around the target object is performed to obtain the first scan information of the target object.

4. The method according to claim 1 or 2, characterized in that, After scanning a local region around the target object to obtain the first scan information of the target object, the method further includes: The target region of the target object is determined based on the local region of the target object, and the target region includes at least a portion of other regions on the target object besides the local region. Obtain the second scan information corresponding to the target area; The step of establishing a three-dimensional model of the target object based on the first scanning information includes: A three-dimensional model of the target object is established based on the first scan information and the second scan information.

5. The method according to claim 1 or 2, characterized in that, The reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories, and the output reference scan information includes: The reference number of sub-scan trajectories are displayed sequentially according to the location of the terminal.

6. The method according to claim 1 or 2, characterized in that, After outputting the reference scan information, the method further includes: The actual scanning information of the terminal is detected, and the actual scanning information includes one or more of the following: actual scanning trajectory, actual scanning speed, and actual scanning posture. If the deviation between the actual scan information and the reference scan information is greater than a reference threshold, a prompt message is displayed based on the deviation, and the prompt message is used to prompt the correction of the actual scan information.

7. The method according to claim 1 or 2, characterized in that, After scanning a local region around the target object to obtain the first scan information of the target object, the method further includes: Determine the scanned and unscanned areas of the target object; Display the projected image of the target object, and mark the scanned area and the unscanned area on the projected image of the target object respectively.

8. The method according to claim 1 or 2, characterized in that, After establishing the three-dimensional model of the target object based on the first scanning information, the method further includes: The three-dimensional model of the target object is post-processed to obtain the processed three-dimensional model. The processed 3D model is displayed.

9. The method according to claim 6, characterized in that, After detecting the actual scanning information of the terminal, the method further includes: The display shows the actual scan trajectory of the target object and the reference scan trajectory.

10. The method according to claim 6, characterized in that, After scanning a local region around the target object to obtain the first scan information of the target object, the method further includes: Scan evaluation information is obtained based on the deviation between the actual scan information and the reference scan information, and the scan evaluation information is used to evaluate the scan situation. The scan evaluation information is displayed.

11. An apparatus for object reconstruction, characterized in that, The device includes: The detection module is used to perform feature location detection on the target object when the terminal detects the target object, and the feature location detection is used to obtain feature information of the target object; The first display module is configured to: if feature information of the target object is detected, determine the core region of the target object based on the feature information; acquire scanning information of the feature positions of the target object, and establish a feature position model of the target object based on the scanning information of the feature positions; determine an upper region, a lower region, a left region, and a right region on the feature position model with the core region as the center; acquire the normalized completeness corresponding to each region, and use the normalized completeness corresponding to each region as the completeness information of the feature position model; determine the scanning distance between the terminal and the target object based on the normalized completeness corresponding to the core region, and determine the scanning distance based on the normalized completeness corresponding to the upper region and the lower region. The scanning height and number of scans are determined; the scanning direction is determined based on the normalized completeness corresponding to the left and right regions, with the starting side of the scanning direction being the region with the lower normalized completeness corresponding to the left and right regions; a reference scanning trajectory is determined based on the scanning interval, the scanning height, the number of scans, and the scanning direction; a reference scanning speed and reference scanning posture are determined for each region based on the normalized completeness corresponding to each region; reference scanning information is output, including the reference scanning trajectory, the reference scanning speed, and the reference scanning posture, which is used to guide the scanning of a local region of the target object. The acquisition module is used to scan a local area around the target object to obtain first scan information of the target object; A module is established to create a three-dimensional model of the target object based on the first scan information.

12. The apparatus according to claim 11, characterized in that, If the target object is a human body, the feature location is the face.

13. The apparatus according to claim 11 or 12, characterized in that, The device further includes: The first determining module is configured to, if feature information of the target object is detected, determine the pose information of the terminal relative to the target object based on the feature information, wherein the pose information includes one or both of position information and posture information. The acquisition module is used to perform a scan of the local region surrounding the target object if the pose information meets the conditions, so as to obtain the first scan information of the target object.

14. The apparatus according to claim 11 or 12, characterized in that, The device further includes: The second determining module is used to determine the target region of the target object based on the local region of the target object, wherein the target region includes at least a portion of other regions on the target object besides the local region; and to obtain second scanning information corresponding to the target region; the establishing module is used to establish a three-dimensional model of the target object based on the first scanning information and the second scanning information.

15. The apparatus according to claim 11 or 12, characterized in that, The reference scan trajectory includes a number of non-overlapping reference sub-scan trajectories, and the first display module is used to sequentially display each sub-scan trajectory according to the position of the terminal.

16. The apparatus according to claim 11 or 12, characterized in that, The device further includes: The second display module is used to detect the actual scanning information of the terminal, which includes one or more of the following: actual scanning trajectory, actual scanning speed, and actual scanning posture. If the deviation between the actual scanning information and the reference scanning information is greater than a reference threshold, a prompt message is displayed based on the deviation. The prompt message is used to prompt correction of the actual scanning information.

17. The apparatus according to claim 11 or 12, characterized in that, The device further includes: The annotation module is used to determine the scanned and unscanned areas of the target object; display the projected image of the target object, and annotate the scanned and unscanned areas on the projected image of the target object respectively.

18. The apparatus according to claim 11 or 12, characterized in that, The device further includes: The processing module is used to perform post-processing on the three-dimensional model of the target object to obtain the processed three-dimensional model; and to display the processed three-dimensional model.

19. The apparatus according to claim 16, characterized in that, The device further includes: The third display module is used to display the actual scanning trajectory of the target object and the reference scanning trajectory.

20. The apparatus according to claim 16, characterized in that, The device further includes: The fourth display module is used to obtain scanning evaluation information based on the deviation between the actual scanning information and the reference scanning information, the scanning evaluation information being used to evaluate the scanning situation; and to display the scanning evaluation information.

21. An object reconstruction device, characterized in that, The device includes a memory and a processor; the memory stores at least one instruction, which is loaded and executed by the processor to implement the object reconstruction method according to any one of claims 1-10.

22. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement the object reconstruction method as described in any one of claims 1-10.

Images