User guidance method and device based on object point cloud data scanning, equipment, medium and product

Abstract

The application discloses a user guidance method and device based on object point cloud data scanning, equipment, medium and product. The method comprises the following steps: obtaining current point cloud data and current object image information of a to-be-scanned object in a current time period, which are obtained by a mobile device under the control of historical guidance prompt information of a target user in a historical time period; if it is determined that there is a hollow region in the current point cloud data according to the object image information of the to-be-scanned object, determining a supplementary scanning path according to the hollow region; obtaining current inertial measurement unit data of the to-be-scanned object in the current time period, determining the moving speed and attitude angle of the mobile device, and generating current guidance prompt information according to the moving speed and attitude angle of the mobile device; and feeding back the current guidance prompt information and the supplementary scanning path to the target user, so that the target user controls the mobile device to perform point cloud data scanning on the to-be-scanned object again based on the current guidance prompt information and the supplementary scanning path.

Description

Technical Field

[0001] This invention relates to the field of point cloud data scanning and processing, and in particular to a user-guided method, apparatus, device, medium, and product based on object point cloud data scanning. Background Technology

[0002] In the field of point cloud data scanning and processing, the method by which users control mobile devices to acquire point cloud data of the object to be scanned is a key process supporting the integrity of point cloud data and scanning quality. Its execution effect determines the efficiency of point cloud data acquisition and the reliability of subsequent data processing. As 3D scanning application scenarios become increasingly complex, users are prone to problems such as data loss, unreasonable poses, and non-standard acquisition paths when autonomously scanning the 3D point cloud data of the object to be scanned.

[0003] Existing scanning guidance processes suffer from inaccurate data parsing, insufficient identification of void areas, and inadequate path planning. They easily overlook conditions such as point cloud integrity, device posture, and motion status, leading to defects in point cloud data such as voids and insufficient density. Furthermore, traditional point cloud data scanning guidance methods lack mobile device posture assessment, making it difficult to adapt to the guidance needs of different object structures and acquisition scenarios. Moreover, guidance strategies rely on simple prompts based on a single state, resulting in point cloud data scanning guidance processes failing to meet the demands for high-quality and efficient point cloud data acquisition. Summary of the Invention

[0004] This invention provides a user guidance method, apparatus, device, medium, and product based on object point cloud data scanning, to optimize the execution process of point cloud data scanning guidance in the field of point cloud data scanning and processing, thereby ensuring the integrity of point cloud data acquisition and improving point cloud data scanning efficiency.

[0005] According to one aspect of the present invention, a user guidance method based on object point cloud data scanning is provided, the method comprising:

[0006] The target user obtains historical guidance prompts based on historical time periods to control the mobile device to collect current point cloud data and current object image information of the object to be scanned in the current time period;

[0007] If it is determined from the object image information of the object to be scanned that there is a hole region in the current point cloud data, then a supplementary scanning path is determined based on the hole region;

[0008] Obtain the current inertial measurement unit data of the object to be scanned in the current time period;

[0009] Based on the current inertial measurement unit data, determine the moving speed and attitude angle of the mobile device, and generate the current guidance prompt information based on the moving speed and attitude angle of the mobile device;

[0010] The current guidance prompt and the supplementary scanning path are fed back to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt and the supplementary scanning path.

[0011] According to another aspect of the present invention, a user guidance device based on object point cloud data scanning is provided, the device comprising:

[0012] The point cloud data acquisition module is used to acquire the current point cloud data and current object image information of the object to be scanned, which are collected by the target user based on historical guidance prompts and information collected by the mobile device in the current time period.

[0013] The supplementary path determination module is used to determine, based on the object image information of the object to be scanned, that there are empty regions in the current point cloud data, and then determine a supplementary scanning path based on the empty regions;

[0014] The measurement data acquisition module is used to acquire the current inertial measurement unit data of the object to be scanned in the current time period;

[0015] The guidance information generation module is used to determine the moving speed and attitude angle of the mobile device based on the current inertial measurement unit data, and generate current guidance prompt information based on the moving speed and attitude angle of the mobile device.

[0016] The guidance information feedback module is used to feed back the current guidance prompt information and the supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and the supplementary scanning path.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory that is communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the user-guided method based on object point cloud data scanning according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a user-guided method based on object point cloud data scanning according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a user-guided method for scanning object point cloud data according to any embodiment of the present invention.

[0023] The technical solution of this invention obtains the current point cloud data and current object image information of the object to be scanned by the mobile device based on the historical guidance prompt information of the target user under a historical time period. If it is determined that there are holes in the current point cloud data based on the object image information of the object to be scanned, a supplementary scanning path is determined based on the holes. The current inertial measurement unit data of the object to be scanned under the current time period is obtained. Based on the current inertial measurement unit data, the moving speed and attitude angle of the mobile device are determined, and the current guidance prompt information is generated based on the moving speed and attitude angle of the mobile device. The current guidance prompt information and the supplementary scanning path are fed back to the target user so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and the supplementary scanning path. The above technical solution can improve the integrity and acquisition efficiency of point cloud data scanning, avoid scanning defects caused by holes in point cloud data, unreasonable device posture, and non-standard acquisition paths, and ensure that the scanned point cloud data can truly reflect the actual structure of the object to be scanned. On the other hand, by accurately identifying hole areas and generating supplementary scanning paths, invalid scanning operations can be reduced, saving scanning time and labor costs. At the same time, reasonable guidance prompts can be generated based on the motion status of mobile devices, reducing the risk of data loss during rescanning, improving the reliability and standardization of the scanning process, and better meeting the requirements of high-quality and high-efficiency data acquisition in the field of point cloud data scanning and processing.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a user guidance method based on object point cloud data scanning according to Embodiment 1 of the present invention;

[0027] Figure 2 This is a flowchart of a user guidance method based on object point cloud data scanning according to Embodiment 2 of the present invention;

[0028] Figure 3 This is a flowchart of a user guidance method based on object point cloud data scanning according to Embodiment 3 of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of a user guidance device based on object point cloud data scanning according to Embodiment 4 of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of an electronic device that implements a user guidance method based on object point cloud data scanning according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1This is a flowchart illustrating a user guidance method based on object point cloud data scanning according to Embodiment 1 of the present invention. This embodiment is applicable to application scenarios that provide real-time guidance to users to perform complete and accurate scanning of the point cloud data of the object to be scanned. This method can be executed by a user guidance device based on object point cloud data scanning. This user guidance device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S101. Obtain the target user's historical guidance prompts based on historical time periods and control the mobile device to collect the current point cloud data and current object image information of the object to be scanned in the current time period.

[0036] S102. If it is determined from the object image information of the object to be scanned that there are hollow areas in the current point cloud data, then a supplementary scanning path is determined based on the hollow areas.

[0037] S103. Obtain the current inertial measurement unit data of the object to be scanned in the current time period.

[0038] S104. Based on the current inertial measurement unit data, determine the moving speed and attitude angle of the mobile device, and generate the current guidance prompt information based on the moving speed and attitude angle of the mobile device.

[0039] S105. Feedback the current guidance prompts and supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompts and supplementary scanning path.

[0040] The target user can be any operator performing point cloud data scanning. The historical time period can be any scanning period prior to the object to be scanned. Historical guidance prompts can be scanning operation guidance information output within the historical time period. The object to be scanned can be any object requiring 3D point cloud scanning. The current time period can be the currently executing scanning period. The current point cloud data can be the 3D point cloud coordinate data of the object to be scanned collected by the mobile device within the current time period. The current object image information can be the semantic information of the image of the object to be scanned collected by the mobile device within the current time period. The mobile device can be a handheld smart scanning device used to scan the 3D point cloud data of the object to be scanned.

[0041] For example, when a target user scans an object, the preset continuous scanning period before this scan is set as the historical time period, and the various scanning guidance information output within the historical time period is the historical guidance prompt information. The currently executing continuous scanning period is set as the current time period. The mobile device collects the three-dimensional point cloud coordinate data of the object to be scanned within this period to obtain the current point cloud data. At the same time, the image semantic information of the object to be scanned is collected and integrated into the current object image information.

[0042] The void region can be an area within the target area of ​​the object to be scanned that is not covered by the current point cloud data. The supplementary scan path can be a mobile device-specific scan path generated by a pre-defined path planning algorithm based on the location and shape of the void region.

[0043] For example, semantic information is extracted from the current object image information of the object to be scanned. Areas not covered by the current point cloud data within the target area of ​​the object to be scanned are identified as "hole regions." Based on the position and shape features of these hole regions on the object to be scanned, a path planning algorithm is invoked, using the hole regions as target path points, to generate a supplementary scanning path that the mobile device must follow. The path planning algorithm can employ a breadth-first search algorithm.

[0044] Among them, the current inertial measurement unit data can be the acceleration and angular velocity data of the mobile device collected in real time within the current time period, reflecting the mobile device's acceleration and angular velocity.

[0045] For example, during the current time period of scanning the object to be scanned, the mobile device continuously collects and uploads instantaneous acceleration and instantaneous angular velocity data of the device along the X / Y / Z axes in three-dimensional space as the current inertial measurement unit data.

[0046] The mobile device's movement speed can be the real-time motion rate of the mobile device calculated and output based on the current inertial measurement unit (IMU) data. The mobile device's attitude angle can be the real-time rotation angle of the mobile device calculated and output based on the current IMU data. The current guidance prompt information can be the scanning operation guidance information generated based on the calculated mobile device movement speed and attitude angle.

[0047] For example, the current inertial measurement unit data of the object to be scanned, collected within the current time period, can be input into a synchronous positioning and mapping algorithm. The algorithm processes the data to obtain the real-time motion rate of the mobile device in three-dimensional space, i.e., the device's speed. Simultaneously, the algorithm calculates the device's pitch, roll, and yaw angles, i.e., its attitude angles. Based on the calculated speed and attitude angle values, a corresponding current guidance prompt is generated. For instance, if the mobile device's speed is too fast, the generated prompt could be: "The mobile device is moving too fast; please reduce your speed."

[0048] For example, after receiving the current guidance prompts and supplementary scanning path, the target user can control the mobile device according to the current guidance prompts and supplementary scanning path, and then scan the point cloud data of the object to be scanned again, and finally obtain the complete point cloud data of the object to be scanned.

[0049] Furthermore, to improve the accuracy of calculating the mobile device's moving speed and attitude angle, in one optional embodiment, the current inertial measurement unit (IMU) data includes the mobile device's acceleration and angular velocity; correspondingly, determining the mobile device's moving speed and attitude angle based on the current IMU data includes:

[0050] Step a1: Determine the moving speed of the mobile device based on its acceleration.

[0051] Step a2: Determine the attitude angle of the mobile device based on the angular velocity of the mobile device.

[0052] Among them, the acceleration of the mobile device can be the instantaneous acceleration data of the three-dimensional space X / Y / Z axes collected by the mobile device within the current time period.

[0053] For example, with the initial moving speed of the mobile device as 0, the instantaneous acceleration data of the mobile device in the X / Y / Z axes collected in the current time period is integrated over time to calculate the velocity increment at this moment relative to the previous moment. The velocity increments at each moment are accumulated to obtain the instantaneous motion speed of the mobile device in the X, Y, and Z axes. Then, the instantaneous motion speed of the three axes is vector synthesized to obtain the real-time moving speed of the mobile device in three-dimensional space.

[0054] Among them, the angular velocity of the mobile device can be the instantaneous angular velocity data of the three-dimensional space X / Y / Z axes collected by the mobile device within the current time period.

[0055] For example, with the initial attitude angle of the mobile device being 0, the instantaneous angular velocity data of the mobile device's X / Y / Z axes collected within the current time period are integrated over time at a preset collection frequency. The rotation angle increment of the three-axis angular velocity at each collection moment relative to the previous moment is calculated. The angle increments at each moment are accumulated to obtain the cumulative rotation angle of the mobile device around the X-axis, Y-axis, and Z-axis, which is the attitude angle of the mobile device.

[0056] The above technical solution extracts acceleration and angular velocity data from the inertial measurement unit data, and calculates velocity and attitude angles separately through time integration. This simplifies the calculation process, reduces interference from invalid data, improves the efficiency and accuracy of calculating the device's motion state parameters, and makes the generation of guidance prompts more in line with the actual scanning operation requirements.

[0057] The technical solution of this invention obtains the current point cloud data and current object image information of the object to be scanned by the mobile device based on the historical guidance prompt information of the target user under a historical time period. If it is determined that there are holes in the current point cloud data based on the object image information of the object to be scanned, a supplementary scanning path is determined based on the holes. The current inertial measurement unit data of the object to be scanned under the current time period is obtained. Based on the current inertial measurement unit data, the moving speed and attitude angle of the mobile device are determined, and the current guidance prompt information is generated based on the moving speed and attitude angle of the mobile device. The current guidance prompt information and the supplementary scanning path are fed back to the target user so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and the supplementary scanning path. The above technical solution can improve the integrity and acquisition efficiency of point cloud data scanning, avoid scanning defects caused by holes in point cloud data, unreasonable device posture, and non-standard acquisition paths, and ensure that the scanned point cloud data can truly reflect the actual structure of the object to be scanned. On the other hand, by accurately identifying hole areas and generating supplementary scanning paths, invalid scanning operations can be reduced, saving scanning time and labor costs. At the same time, reasonable guidance prompts can be generated based on the motion status of mobile devices, reducing the risk of data loss during rescanning, improving the reliability and standardization of the scanning process, and better meeting the requirements of high-quality and high-efficiency data acquisition in the field of point cloud data scanning and processing.

[0058] Example 2

[0059] Figure 2This is a flowchart of a user guidance method based on object point cloud data scanning provided in Embodiment 2 of the present invention. This embodiment optimizes and improves upon the above-mentioned technical solutions. The step "generating current guidance prompt information based on the moving speed and attitude angle of the mobile device" is refined to "if the moving speed of the mobile device is greater than a preset speed threshold, then the current guidance prompt information is a preset first guidance prompt information; if the attitude angle of the mobile device is greater than a preset attitude angle threshold, then the current guidance prompt information is a preset second guidance prompt information." This improves the process of generating guidance prompt information.

[0060] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments. For example... Figure 2 As shown, the method includes the following specific steps:

[0061] S201. Obtain the target user's historical guidance prompts based on historical time periods and control the mobile device to collect the current point cloud data and current object image information of the object to be scanned in the current time period.

[0062] S202. If it is determined from the object image information of the object to be scanned that there are hollow areas in the current point cloud data, then a supplementary scanning path is determined based on the hollow areas.

[0063] S203. Obtain the current inertial measurement unit data of the object to be scanned in the current time period.

[0064] S204. Based on the current inertial measurement unit data, determine the moving speed and attitude angle of the mobile device.

[0065] S205. Determine the relationship between the moving speed of the mobile device and the speed threshold. If the moving speed of the mobile device is greater than the preset speed threshold, then execute S206. If the attitude angle of the mobile device is greater than the preset attitude angle threshold, then execute S207.

[0066] S206. The current guidance prompt is the preset first guidance prompt.

[0067] S207. The current guidance prompt is the preset second guidance prompt.

[0068] S208. Feedback the current guidance prompts and supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompts and supplementary scanning path.

[0069] The first guidance message can be a prompt message preset by relevant technicians to guide users to reduce the device's movement speed when the mobile device's movement speed exceeds a preset speed threshold. The speed threshold is a critical value for determining whether the mobile device's current scanning speed is compliant. The speed threshold can be preset by relevant technicians according to actual needs, or it can be flexibly adjusted according to the scanning accuracy requirements of the object to be scanned. The higher the scanning accuracy requirement, the lower the speed threshold should be set.

[0070] For example, the calculated real-time moving speed of the mobile device (0.8 m / s) is compared with the preset speed threshold (0.5 m / s). If the real-time moving speed of the mobile device is determined to be greater than the preset speed threshold, the mobile device outputs the first guidance prompt message "Scanning speed is too fast, please slow down to within 0.5 m / s". At the same time, the difference between the current moving speed and the threshold can be displayed on the mobile device in real time, intuitively showing the speed exceeding the standard.

[0071] The second guidance message can be a prompt message preset by relevant technicians to guide users to adjust the device's scanning posture when the mobile device's posture angle is greater than a preset posture angle threshold. The posture angle threshold is a critical angle value used to determine whether the mobile device's current scanning posture is compliant. The posture angle threshold can be preset by relevant technicians according to actual needs and can be adapted to posture settings for different scanning directions.

[0072] For example, the calculated real-time attitude angle of the mobile device (35 degrees) is compared with a preset attitude angle threshold of 25 degrees. If the attitude angle is greater than the threshold, the mobile device outputs a second guidance message: "The device attitude angle is too large. Please adjust it to within 25 degrees." If the mobile device simultaneously exceeds both the movement speed and attitude angle thresholds, the mobile device will prioritize broadcasting both types of prompts and highlight the exceeding values ​​of speed and attitude angle in different areas of the operation interface, guiding the user to first adjust the attitude angle to the compliant range and then reduce the movement speed.

[0073] Furthermore, in order to optimize the guidance and prompting strategy based on the overall quality of point cloud data and improve the integrity and quality of the point cloud data after rescanning, in an optional embodiment, before feeding back the current guidance and prompting information and the rescanning path to the target user, the following is also included:

[0074] Step b1: Determine the quality score of the point cloud data of the object to be scanned.

[0075] Step b2: If the point cloud data quality score is less than the preset quality score threshold, the current guidance prompt message is the preset third guidance prompt message.

[0076] Among them, the point cloud data quality score can be a quantitative score that reflects the quality of point cloud acquisition of the object to be scanned.

[0077] Furthermore, in order to standardize the calculation of point cloud data quality scores and improve the objectivity and accuracy of point cloud data quality score determination, in one optional embodiment, determining the point cloud data quality score of the object to be scanned includes:

[0078] Step c1: Divide the current point cloud data of the object to be scanned into regions to obtain at least one point cloud data sub-region.

[0079] Step c2: Determine the regional point cloud density corresponding to each point cloud data sub-region, and determine the point cloud data quality score of the object to be scanned based on the regional point cloud density of each point cloud data sub-region.

[0080] Among them, the point cloud data sub-region can be a voxel space sub-region obtained by dividing the space where the point cloud data is located according to a preset voxel size.

[0081] For example, the point cloud data of the object to be scanned is spatially divided into cubic voxels of a preset size by relevant technicians, with the point cloud data within each voxel forming a point cloud data sub-region. Simultaneously, a voxel downsampling algorithm can be used to spatially divide and lightweight the current point cloud data of the object to be scanned: using cubic voxels of a preset size as the basic unit, the entire point cloud is traversed and voxel meshing is completed. Each cubic voxel containing point cloud data is directly taken as a point cloud data sub-region. Simultaneously, the geometric center point of the original point cloud data within each sub-region is calculated as the feature point of that sub-region, thus completing the division of point cloud data sub-regions and the simplification of sub-region point clouds.

[0082] Among them, the regional point cloud density can be the point cloud quantity density within a single point cloud data sub-region.

[0083] For example, the number of point clouds in a single voxel sub-region is counted, and the ratio of this number of point clouds to a preset maximum number of point clouds is processed to obtain the normalized regional point cloud density. The average value of the normalized density of all voxel sub-regions is taken, and the average value is mapped to a score of 0-100. The final score can be used as the point cloud data quality score.

[0084] The above technical solution divides point cloud data into sub-regions by voxels, normalizes the point cloud density of each sub-region, and comprehensively calculates the point cloud data quality score of the object to be scanned. This enables a refined evaluation of point cloud quality, avoids ignoring local low-quality areas in the overall evaluation, and makes the point cloud data quality score of the object to be scanned more accurately reflect the actual acquisition situation of each local area, providing a reliable basis for subsequent refined guidance.

[0085] Furthermore, in order to achieve dynamic generation of the mass fraction threshold, velocity threshold, and attitude angle threshold, and to make the threshold parameters more closely match the actual needs of different scanning scenarios, in an optional embodiment, the mass fraction threshold, velocity threshold, and attitude angle threshold are determined as follows:

[0086] Step d1: Determine the object scanning scene and object environment information of the object to be scanned, and determine the object scanning complexity and scene scanning cases based on the object scanning scene.

[0087] Step d2: Obtain the object scanning requirements for the object to be scanned.

[0088] Step d3: Generate target prompts based on object scanning requirements, object environment information, object scanning complexity, and scene scanning cases.

[0089] Step d4: Call the pre-selected large model to generate the mass fraction threshold, velocity threshold, and attitude angle threshold of the object to be scanned based on the target prompt words.

[0090] Among them, the object scanning scene can be the type of application scenario for conducting point cloud scanning of the object to be scanned. The object environment information can be environmental feature information such as lighting, occlusion, and spatial range of the scanning site. The object scanning complexity can be the level of difficulty of the scanning operation determined based on the structural shape and detailed features of the object to be scanned. The scene scanning case can be a historical 3D point cloud data scanning case that matches the current object scanning scene and complexity.

[0091] For example, firstly, the application scenario type for conducting point cloud scanning of the object to be scanned is determined to obtain the object scanning scenario. At the same time, environmental feature information such as illumination, occlusion, and spatial range of the scanning site is collected to obtain the object environment information. Then, the difficulty level of the scanning operation is determined based on the structural shape and detailed features of the object to be scanned to obtain the object scanning complexity. Finally, historical point cloud scanning operation cases that match the current scanning scenario and complexity are retrieved from the system's historical database to obtain the scene scanning case.

[0092] Among them, the object scanning requirements can be the specific requirements put forward by the target user for the point cloud data of the object to be scanned, such as the acquisition accuracy, completeness, and point cloud density. For example, the acquisition accuracy should be less than or equal to 0.05 mm, the point cloud should be free of holes and have 100% completeness, and the point cloud density should be greater than or equal to 300 points / square centimeter.

[0093] The target prompts can be the core features that integrate object scanning requirements, object environment information, object scanning complexity, and scene scanning cases. For example, the scanning environment should be well-lit and unobstructed, the object to be scanned should have a medium structure, and the point cloud acquisition should have an accuracy of less than or equal to 0.05 mm, no holes, and a density of greater than or equal to 300 points / square centimeter. Matching scanning cases of objects with the same type of structure can generate appropriate point cloud quality scores, device scanning speeds, and device attitude angle thresholds.

[0094] For example, core feature information from the scanning requirements of the object to be scanned, the object's environmental information, the object's scanning complexity, and scene scanning cases is extracted, integrated and edited according to a preset text format, and standardized text prompts are generated as target prompt words to guide the large model to output adaptation thresholds. For example, the feature information from the scanning requirements of the object to be scanned, the object's environmental information, the object's scanning complexity, and scene scanning cases is extracted, and integrated according to a preset text format as: "Sufficient indoor lighting, simple structure of the object to be scanned, point cloud requires complete and uniformity, please generate corresponding target prompt words for point cloud data quality score threshold, mobile device speed threshold, and mobile device attitude angle threshold."

[0095] The pre-selected large model can be used to dynamically generate quality score thresholds, velocity thresholds, and pose angle thresholds adapted to the current scanning scenario, based on the input target prompt words and combined with training data from the point cloud scanning domain. The quality score threshold can be a quantitative score threshold used to determine whether the quality of the current point cloud data meets the standard. The pre-selected large model can be a general language large model that has been fine-tuned specifically for the point cloud scanning domain. The fine-tuning dataset contains threshold parameters, environmental features, and correlation data between complexity and scanning quality for massive point cloud scanning scenarios.

[0096] For example, the generated target prompts are input into a pre-selected large model. The large model then analyzes and calculates the data using point cloud scanning training data and historical case data, ultimately outputting quality score thresholds, velocity thresholds, and pose angle thresholds adapted to the current scanning scene of the object to be scanned. The pre-selected large model can be a general-purpose language model.

[0097] The above technical solution generates target prompt words by integrating multi-dimensional information such as scanning scene, environment, and needs, and dynamically generates thresholds by calling a large model. This avoids the defect that fixed thresholds cannot adapt to different scanning scenes, and makes the threshold parameters more in line with the structural features of the object to be scanned and the user's actual scanning needs, thereby improving the rationality and adaptability of threshold determination and optimizing the accuracy of the current guidance prompt information generated subsequently.

[0098] The third guidance message can be a prompt message preset by relevant technicians to guide the user to rescan the current area when the point cloud data quality score of the object being scanned by the mobile device is lower than a preset quality score threshold. The quality score threshold can also be preset by relevant technicians according to actual needs.

[0099] For example, if the calculated quality score of the point cloud data of the object to be scanned, which is 75, is compared with the preset quality score threshold of 80, and the quality score is determined to be less than the quality score threshold, the mobile device can output a voice prompt, "The current area's point cloud quality is substandard, please rescan," as a third guiding prompt, and highlight the substandard scanned area on the mobile device's operating interface. After the rescan is completed, the mobile device can also re-evaluate the point cloud data of that area in real time. If the point cloud data quality score is greater than or equal to the quality score threshold, the highlighting on the interface will be automatically removed, and a voice prompt will indicate that the current area's point cloud data quality score meets the standard. If it still does not meet the standard, the specific reasons for the substandard point cloud data quality will be further indicated, such as: low point cloud density, the presence of small holes in some areas, etc., and targeted adjustment suggestions will be given until the quality of the point cloud data in that area meets the requirements.

[0100] The above technical solution, by adding point cloud data quality score judgment conditions before the current guidance prompt information, and outputting targeted guidance prompts when the point cloud data quality score is not up to standard, can make up for the shortcomings of guidance based solely on the motion status of mobile devices and empty areas. It can control the scanning effect from the overall quality level, avoid the problem of overall quality not meeting the standard after rescanning, and improve the overall pass rate of point cloud data acquisition of the object to be scanned.

[0101] This embodiment adds threshold judgments for the current mobile device speed and attitude angle during the generation of current guidance prompt information. For cases that do not meet the thresholds, corresponding first and second guidance prompt information are output respectively. This can achieve refined and targeted guidance for the user's scanning operation, and can promptly correct non-standard operations such as excessive device speed and unreasonable attitude. From the perspective of mobile device motion state, it reduces the generation of defects such as hole and insufficient density in point cloud data, and improves the efficiency of supplementary scanning and the quality of point cloud data acquisition.

[0102] Example 3

[0103] Figure 3 This is a flowchart of a user guidance method based on object point cloud data scanning provided in Embodiment 3 of the present invention. This embodiment provides a preferred example based on the above embodiments.

[0104] S301. Obtain the target user's historical guidance prompts based on historical time periods and control the mobile device to collect the current point cloud data and current object image information of the object to be scanned in the current time period.

[0105] S302. If it is determined from the object image information of the object to be scanned that there are hollow areas in the current point cloud data, then a supplementary scanning path is determined based on the hollow areas.

[0106] S303. Obtain the current inertial measurement unit data of the object to be scanned in the current time period.

[0107] S304. Determine the moving speed of the mobile device based on its acceleration, and determine the attitude angle of the mobile device based on its angular velocity.

[0108] S305. Determine the relationship between the moving speed of the mobile device and the speed threshold. If the moving speed of the mobile device is greater than the preset speed threshold, then execute S306. If the attitude angle of the mobile device is greater than the preset attitude angle threshold, then execute S307.

[0109] S306. The current guidance prompt is the preset first guidance prompt.

[0110] S307. The current guidance prompt is the preset second guidance prompt.

[0111] S308. Divide the current point cloud data of the object to be scanned into regions to obtain at least one point cloud data sub-region.

[0112] S309. Determine the regional point cloud density corresponding to each point cloud data sub-region, and determine the point cloud data quality score of the object to be scanned based on the regional point cloud density of each point cloud data sub-region.

[0113] S310. If the point cloud data quality score is less than the preset quality score threshold, the current guidance prompt message is the preset third guidance prompt message.

[0114] The mass fraction threshold, velocity threshold, and attitude angle threshold are determined as follows:

[0115] Determine the object scanning scene and environment information of the object to be scanned, and determine the object scanning complexity and scene scanning cases based on the object scanning scene; obtain the object scanning requirements of the object to be scanned; generate target prompt words based on the object scanning requirements, object environment information, object scanning complexity and scene scanning cases; call the pre-selected large model to generate the mass score threshold, velocity threshold and attitude angle threshold of the object to be scanned based on the target prompt words.

[0116] S311. Feedback the current guidance prompts and supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompts and supplementary scanning path.

[0117] The information collected in the above embodiments of the present invention is all information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0118] Example 4

[0119] Figure 4 This is a schematic diagram of a user guidance device based on object point cloud data scanning provided in Embodiment 4 of the present invention. The user guidance device based on object point cloud data scanning provided in this embodiment of the present invention is applicable to application scenarios in the field of point cloud data scanning and processing where users continuously guide data acquisition while scanning objects in real time. This user guidance device based on object point cloud data scanning can be implemented in hardware and / or software. It can be applied to a user guidance method based on object point cloud data scanning and can be specifically configured in a controller, such as... Figure 4 As shown, the device includes: a point cloud data acquisition module 401, a supplementary path determination module 402, a measurement data acquisition module 403, a guidance information generation module 404, and a guidance information feedback module 405. Wherein:

[0120] The point cloud data acquisition module 401 is used to acquire the current point cloud data and current object image information of the object to be scanned, which are collected by the target user based on historical guidance prompts and control of the mobile device under the current time period.

[0121] The supplementary path determination module 402 is used to determine the existence of a hole region in the current point cloud data based on the object image information of the object to be scanned, and then determine the supplementary scanning path based on the hole region.

[0122] The measurement data acquisition module 403 is used to acquire the current inertial measurement unit data of the object to be scanned in the current time period;

[0123] The guidance information generation module 404 is used to determine the moving speed and attitude angle of the mobile device based on the current inertial measurement unit data, and generate the current guidance prompt information based on the moving speed and attitude angle of the mobile device.

[0124] The guidance information feedback module 405 is used to feed back the current guidance prompt information and supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and supplementary scanning path.

[0125] The technical solution of this invention obtains the current point cloud data and current object image information of the object to be scanned by the mobile device based on the historical guidance prompt information of the target user under a historical time period. If it is determined that there are holes in the current point cloud data based on the object image information of the object to be scanned, a supplementary scanning path is determined based on the holes. The current inertial measurement unit data of the object to be scanned under the current time period is obtained. Based on the current inertial measurement unit data, the moving speed and attitude angle of the mobile device are determined, and the current guidance prompt information is generated based on the moving speed and attitude angle of the mobile device. The current guidance prompt information and the supplementary scanning path are fed back to the target user so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and the supplementary scanning path. The above technical solution can improve the integrity and acquisition efficiency of point cloud data scanning, avoid scanning defects caused by holes in point cloud data, unreasonable device posture, and non-standard acquisition paths, and ensure that the scanned point cloud data can truly reflect the actual structure of the object to be scanned. On the other hand, by accurately identifying hole areas and generating supplementary scanning paths, invalid scanning operations can be reduced, saving scanning time and labor costs. At the same time, reasonable guidance prompts can be generated based on the motion status of mobile devices, reducing the risk of data loss during rescanning, improving the reliability and standardization of the scanning process, and better meeting the requirements of high-quality and high-efficiency data acquisition in the field of point cloud data scanning and processing.

[0126] Optionally, the boot information generation module 404 includes:

[0127] The first guidance prompt information generation unit is used to determine if the moving speed of the mobile device is greater than a preset speed threshold, and then the current guidance prompt information is the preset first guidance prompt information;

[0128] The second guidance prompt information generation unit is used to determine if the attitude angle of the mobile device is greater than the preset attitude angle threshold, and then the current guidance prompt information is the preset second guidance prompt information.

[0129] Optionally, the device further includes:

[0130] The quality score determination module is used to determine the point cloud data quality score of the object to be scanned before feeding back the current guidance prompt information and the supplementary scanning path to the target user.

[0131] The third guidance prompt information generation module is used to determine if the point cloud data quality score is less than the preset quality score threshold, and then the current guidance prompt information is the preset third guidance prompt information.

[0132] Optional, a quality score determination module, specifically used for:

[0133] Divide the current point cloud data of the object to be scanned into regions to obtain at least one point cloud data sub-region;

[0134] Determine the regional point cloud density corresponding to each point cloud data sub-region, and determine the point cloud data quality score of the object to be scanned based on the regional point cloud density of each point cloud data sub-region.

[0135] Optionally, the device further includes:

[0136] The scanning environment determination module is used to determine the object scanning scene and object environment information of the object to be scanned, and to determine the object scanning complexity and scene scanning cases based on the object scanning scene.

[0137] The scanning requirement acquisition module is used to acquire the object scanning requirements of the object to be scanned.

[0138] The target prompt word generation module is used to generate target prompt words based on object scanning requirements, object environment information, object scanning complexity, and scene scanning cases.

[0139] The threshold generation module is used to call a pre-selected large model to generate the mass fraction threshold, velocity threshold, and attitude angle threshold of the object to be scanned based on the target prompt words.

[0140] Optionally, the current inertial measurement unit data includes the acceleration and angular velocity of the mobile device; correspondingly, the guidance information generation module 404 further includes:

[0141] A moving speed determination unit is used to determine the moving speed of the mobile device based on the acceleration of the mobile device;

[0142] The attitude angle determination unit is used to determine the attitude angle of the mobile device based on the angular velocity of the mobile device.

[0143] The user guidance device based on object point cloud data scanning provided in this embodiment of the invention can execute a user guidance method based on object point cloud data scanning provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0144] Example 5

[0145] Figure 5A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0146] like Figure 5 As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded from storage unit 58 into the RAM 53. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0147] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0148] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as user-guided methods based on object point cloud data scanning.

[0149] In some embodiments, a user-guided method based on object point cloud data scanning can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the user-guided method based on object point cloud data scanning described above can be performed. Alternatively, in other embodiments, processor 51 can be configured for the user-guided method based on object point cloud data scanning by any other suitable means (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0155] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0156] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0157] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A user guidance method based on object point cloud data scanning, characterized in that, include: The target user obtains historical guidance prompts based on historical time periods to control the mobile device to collect current point cloud data and current object image information of the object to be scanned in the current time period; If it is determined from the object image information of the object to be scanned that there is a hole region in the current point cloud data, then a supplementary scanning path is determined based on the hole region; Obtain the current inertial measurement unit data of the object to be scanned in the current time period; Based on the current inertial measurement unit data, determine the moving speed and attitude angle of the mobile device, and generate the current guidance prompt information based on the moving speed and attitude angle of the mobile device; The current guidance prompt and the supplementary scanning path are fed back to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt and the supplementary scanning path.

2. The method according to claim 1, characterized in that, The step of generating the current guidance prompt information based on the mobile device's moving speed and attitude angle includes: If the mobile device's movement speed is greater than a preset speed threshold, then the current guidance prompt information is a preset first guidance prompt information; and, If the attitude angle of the mobile device is greater than a preset attitude angle threshold, then the current guidance prompt information is a preset second guidance prompt information.

3. The method according to claim 2, characterized in that, Before sending the current guidance prompt information and the supplementary scan path to the target user, the method further includes: Determine the quality score of the point cloud data of the object to be scanned; If the point cloud data quality score is less than a preset quality score threshold, then the current guidance prompt information is the preset third guidance prompt information.

4. The method according to claim 3, characterized in that, Determining the point cloud data quality score of the object to be scanned includes: The current point cloud data of the object to be scanned is divided into regions to obtain at least one point cloud data sub-region; The point cloud density corresponding to each of the point cloud data sub-regions is determined, and the point cloud data quality score of the object to be scanned is determined based on the point cloud density of each of the point cloud data sub-regions.

5. The method according to claim 3, characterized in that, The mass fraction threshold, velocity threshold, and attitude angle threshold are determined as follows: Determine the object scanning scene and object environment information of the object to be scanned, and determine the object scanning complexity and scene scanning cases based on the object scanning scene; Obtain the object scanning requirements for the object to be scanned; Based on the object scanning requirements, object environment information, object scanning complexity, and scene scanning cases, generate target prompt words; The pre-selected large model is invoked to generate the mass fraction threshold, velocity threshold, and attitude angle threshold of the object to be scanned based on the target prompt words.

6. The method according to claim 1, characterized in that, The current inertial measurement unit (IMU) data includes the acceleration and angular velocity of the mobile device; correspondingly, determining the moving speed and attitude angle of the mobile device based on the current IMU data includes: The moving speed of the mobile device is determined based on the acceleration of the mobile device; The attitude angle of the mobile device is determined based on the angular velocity of the mobile device.

7. A user guidance device based on object point cloud data scanning, characterized in that, include: The point cloud data acquisition module is used to acquire the current point cloud data and current object image information of the object to be scanned, which are collected by the target user based on historical guidance prompts and information collected by the mobile device in the current time period. The supplementary path determination module is used to determine, based on the object image information of the object to be scanned, that there are empty regions in the current point cloud data, and then determine a supplementary scanning path based on the empty regions; The measurement data acquisition module is used to acquire the current inertial measurement unit data of the object to be scanned in the current time period; The guidance information generation module is used to determine the moving speed and attitude angle of the mobile device based on the current inertial measurement unit data, and generate current guidance prompt information based on the moving speed and attitude angle of the mobile device. The guidance information feedback module is used to feed back the current guidance prompt information and the supplementary scanning path to the target user, so that the target user can control the mobile device to scan the point cloud data of the object to be scanned again based on the current guidance prompt information and the supplementary scanning path.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the user-guided method based on object point cloud data scanning as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the user-guided method based on object point cloud data scanning as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the user-guided method for scanning object point cloud data according to any one of claims 1-6.

Images