Handheld point cloud scanning blind area detection method based on acquisition area and boundary interaction

Abstract

This invention discloses a blind zone detection method for handheld point cloud scanning based on the interaction between the acquisition area and the boundary, relating to the fields of 3D scanning and point cloud data processing technology. The method first acquires the device pose and local point cloud, preprocesses it, and then transfers it to a global horizontal coordinate system. A boundary feature extraction engine is used to identify room boundary lines and project them to form a 2D boundary contour map. Simultaneously, an effective acquisition loop is dynamically constructed and projected based on the effective measurement range parameters of the laser scanner and the current ambient lighting conditions. Boolean operations are performed on the effective acquisition loop and the boundary contour map to identify uncovered boundary segments as potential blind zones. The occlusion state is determined using the temporal information of continuous frame point clouds and a ray casting algorithm to identify the effective blind zone. This invention reduces the 3D integrity determination to 2D planar geometric intersection, eliminating the need for prior models and enabling real-time identification of dynamic occlusion and acquisition blind zones. This results in high computational efficiency and significantly improves the success rate and coverage integrity of handheld scanning operations.

Description

Technical Field

[0001] This invention belongs to the field of 3D scanning and point cloud data processing technology, specifically, it relates to a handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary. Background Technology

[0002] Handheld 3D laser scanning technology, with its advantages of flexible operation, convenient deployment, and efficient data acquisition, has become a core data acquisition method in key scenarios such as architectural surveying, interior space modeling, industrial facility reverse engineering, and digital twin construction. In these applications, the completeness and sufficiency of point cloud data directly determine the accuracy of subsequent modeling, the reliability of analysis, and even the success or failure of the entire digitization process. However, in actual operation, due to factors such as blind spots of human vision, complex interior layouts, and interference from dynamic obstacles, operators find it difficult to grasp the actual coverage of the target space by the current scanning action in real time and comprehensively. This is especially true for typical geometric shadow areas such as corners, behind furniture, and areas obscured by equipment, which are prone to data loss. Such blind spots are often not immediately apparent from a single scan and require subsequent point cloud stitching and manual verification to discover. This not only significantly extends the project cycle but also results in a double waste of manpower and time costs due to repeated rework, severely restricting the large-scale application of handheld scanning technology in scenarios requiring high efficiency and high reliability.

[0003] To address these challenges, the industry has developed two main technical approaches. The first is a completeness verification method based on full model comparison. This approach uses a pre-built Building Information Model (BIM) or computer-aided design drawings as an ideal reference benchmark. During the scanning process, it continuously spatially registers the real-time collected point cloud data with the preset model and identifies uncovered areas through voxel differencing and surface deviation analysis. This method theoretically possesses high accuracy and is particularly suitable for standardized building environments with regular structures and few changes. The second approach shifts to a lightweight strategy, employing simplified geometric rules for blind zone inference. For example, it sets a distance threshold between the scanner and the nearest wall, or determines whether a valid observation has been completed based on the scanning angle. This approach significantly reduces computational complexity, achieving near real-time response on mobile terminals and alleviating the problem of delayed on-site feedback to some extent.

[0004] However, with increasingly complex application scenarios and continuously improving operational standards, the two aforementioned technical approaches have revealed deep-seated and irreconcilable inherent contradictions at the principle level. Specifically, while the method based on full-scale model comparison is logically rigorous, its implementation heavily relies on the completeness and on-site consistency of a high-precision prior model. In real-world operational environments, dynamic factors such as temporary structures, movable furniture, and equipment additions and removals are common, causing structural deviations between the pre-set model and the actual physical space, resulting in distorted comparison results and frequent misjudgments or missed detections. More importantly, real-time registration and difference analysis between full-scale point clouds and complex 3D models involve massive floating-point operations and nonlinear optimization, with a computational load far exceeding the processing capabilities of current mainstream handheld devices. Even with downsampling or local comparison strategies, it is difficult to achieve millisecond-level feedback while ensuring accuracy, thus losing the core value of real-time guidance. Correspondingly, while lightweight solutions based on simple geometric rules have achieved breakthroughs in computational efficiency, they introduce systematic errors due to oversimplification of the environmental model. They fail to consider the inherent effective measurement range of laser scanners—that is, the existence of a minimum effective distance to avoid near-field noise and a maximum effective distance to ensure point cloud density and accuracy, forming a dynamically changing effective acquisition loop. Furthermore, they cannot perceive the local occlusion effects caused by physical obstacles such as furniture and pillars, relying solely on line-of-sight accessibility to determine coverage status, which is clearly severely disconnected from the physical acquisition process. In addition, these methods generally lack a quantitative evaluation mechanism for coverage adequacy, equating instantaneous scanning with effective acquisition and ignoring the dependence of point cloud quality on multi-dimensional factors such as observation duration, angle diversity, and stability. Ultimately, their functionality stops at static alarms, failing to generate intelligent guidance strategies for operators, such as optimal movement trajectories, suggested dwell positions, or scanning posture adjustment prompts, resulting in the system only being able to detect problems but not solve them.

[0005] At its core, existing technologies are caught in a classic performance trade-off: pursuing high-fidelity integrity assessment inevitably comes with unbearable computational overhead, while pursuing low-latency response necessitates sacrificing the precision of environmental modeling and the intelligence of decision-making logic. This contradiction, coupled with the limitations of mobile resources, highly unstructured operating environments, and increasingly stringent requirements for single-scan success rates, has become a core bottleneck hindering the evolution of handheld point cloud scanning technology towards intelligence and autonomy. Therefore, how to construct a blind spot detection mechanism that accurately characterizes the interaction between the scanner's effective acquisition capabilities and environmental boundaries, while strictly adapting to the computational constraints of mobile terminals, without relying on high-precision prior models, has become a critical technical challenge that urgently needs to be overcome by those skilled in the art. Summary of the Invention

[0006] The present invention provides a handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary, in order to solve the problems of poor real-time performance, weak environmental adaptability and lack of guidance capability caused by the reliance on high-precision prior models or simplified geometric rules in the prior art.

[0007] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0008] A handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary includes:

[0009] Step S1: After the handheld scanning device is started and enters the dynamic scanning mode, the current spatial pose information of the device is obtained synchronously with the global navigation satellite system receiver through the built-in inertial measurement unit, and the local point cloud data stream at the current moment is continuously captured by the depth sensor array.

[0010] Step S2: Preprocess the local point cloud data stream and transform the point cloud coordinates from the sensor local coordinate system to the global horizontal reference coordinate system with the current position of the device as the origin. The Z-axis of the global horizontal reference coordinate system is vertically upward, and the XY plane forms a horizontal reference plane.

[0011] Step S3: Use the boundary feature extraction engine to identify the set of room boundary lines composed of walls and static structures from the preprocessed point cloud, and project the set of room boundary lines onto the horizontal reference plane to construct a two-dimensional boundary contour map;

[0012] Step S4: Dynamically construct an effective acquisition ring based on the effective measurement range parameters of the laser scanner and the current ambient lighting conditions, and project the effective acquisition ring onto the horizontal reference plane to form a ring-shaped area with dynamically adjustable boundaries centered on the current position of the device;

[0013] Step S5: Perform geometric Boolean operations on the effective acquisition ring and the two-dimensional boundary contour map in the horizontal reference plane to identify boundary segments not covered by the effective acquisition ring and form a potential blind zone candidate set;

[0014] Step S6: Input the potential blind zone candidate set into the occlusion state analysis module. Utilize the point cloud temporal information between consecutive frames and combine it with the ray casting algorithm to determine whether the line of sight of each candidate boundary segment is blocked by an obstacle. If it is blocked, the boundary segment is determined to be in a true occlusion state and is marked as an effective blind zone.

[0015] Furthermore, before step S1, the method of the present invention may also include an environmental idle detection stage: keeping the device in a stationary state, collecting background point clouds for ten seconds through the depth sensor array, establishing an initial noise baseline and constructing a static obstacle distribution map, which is used to eliminate fixed structure interference in the occlusion state analysis;

[0016] In step S2, the specific preprocessing operations include: setting the voxel side length of the voxel downsampling to 5mm; performing statistical outlier removal to remove noise using a parameter configuration of a neighborhood point threshold of 20 and a standard deviation multiple of 1.5; and aligning the local point cloud data stream with the timestamp of the inertial measurement unit through a hard synchronization circuit to ensure spatial consistency.

[0017] Furthermore, the specific execution logic of the boundary feature extraction engine in step S3 is as follows:

[0018] First, calculate the normal vector of each point, use K-nearest neighbor search to determine the neighborhood point set, with K value set to 30, and use principal component analysis to solve for the eigenvector corresponding to the smallest eigenvalue of the local covariance matrix as the direction of the normal vector.

[0019] Subsequently, the point cloud was clustered based on the normal vector direction using the DBSCAN algorithm, with an angle tolerance of 15° and a distance tolerance of 50mm. In the clustering results, clusters with a normal vector direction and a Z-axis angle of less than 25° were identified as candidate wall areas, and clusters with a normal vector direction and a Z-axis angle of greater than 65° were identified as candidate ground or ceiling areas.

[0020] Finally, RANSAC plane fitting is performed on each candidate wall area, with an upper limit of 1000 iterations and a distance threshold of 20mm. The successfully fitted plane is projected onto the horizontal reference plane, and its intersection with the ground plane is extracted as a two-dimensional boundary line segment. After endpoint connection and closure verification, the two-dimensional boundary contour map is formed.

[0021] Furthermore, the effective measurement range parameter mentioned in step S4 includes the minimum effective measurement distance of the laser scanner. With the maximum effective measurement distance ;

[0022] The construction rule for the effective acquisition loop is as follows: under standard illumination conditions, set... , When the ambient lighting conditions indicate that the light intensity is below a preset threshold of 100 lx, the system will automatically... Reduced to 6 meters;

[0023] The shape of the effective acquisition ring is based on the current pitch angle of the device. With yaw angle After ellipticization correction, within the horizontal reference plane, the outer ring boundary of the effective acquisition ring is described by the following parametric equation:

[0024]

[0025] in, For parameter variables, semi-major axis semi-short shaft And the inner ring boundary adopts Alternative Perform the same calculations; the effective acquisition ring is the projection of the annular region defined by the inner ring boundary and the outer ring boundary onto the horizontal reference plane.

[0026] Furthermore, the geometric Boolean operation in step S5 employs the Weiler-Atherton polygon clipping algorithm to accurately identify boundary segments in the two-dimensional boundary contour map that are not covered by the effective acquisition ring.

[0027] The specific determination logic of the occlusion state analysis module in step S6 is as follows: using historical point cloud data of three consecutive frames, a ray is emitted from the current pose position of the device to each sampling point on each candidate boundary segment in the potential blind zone candidate set, and it is checked whether the ray intersects with non-boundary points in the historical point cloud before reaching the target point; if more than 80% of the sampling points on a certain boundary segment have at least one instance of ray occlusion in three consecutive frames, then the boundary segment is determined to be in a true occlusion state and is marked as the effective blind zone.

[0028] Furthermore, the method of the present invention also includes assessing the coverage quality of the effective blind area, and using a coverage quality assessment unit to calculate a coverage deficiency score based on the historical dwell time of the device in the vicinity of the area, the scanning angle diversity index, and the point cloud density gradient change rate. The calculation formula is as follows:

[0029]

[0030] in, This refers to the cumulative dwell time of the device within the effective blind zone projection area over the past sixty seconds, expressed in seconds. To prevent division by zero of small constants; The angular diversity index is defined as the ratio of the number of sectors in the blind zone that the device can successfully observe from different azimuth angles to the total number of sectors, wherein the azimuth angles are divided into 12 sectors at 30° intervals. The point cloud density gradient variance is obtained by performing a one-dimensional discrete Fourier transform on the density distribution of the point cloud at the blind zone edge along the normal direction and then taking the standard deviation of the low-frequency components; the weighting coefficients satisfy... , , .

[0031] Furthermore, the method of the present invention also includes supplementary scanning path planning:

[0032] The central collaborative controller receives the list of effective blind spots and their corresponding coverage insufficiency scores, and drives the trajectory planning engine to generate the optimal supplementary scanning path. The trajectory planning engine first divides the working area into a grid map with a side length of 20cm, and marks the grids occupied by known obstacles as impassable.

[0033] Subsequently, a fast exploratory random tree algorithm, incorporating an information gain function as a heuristic factor for node expansion, is employed for path search. In position The definition of a location is:

[0034]

[0035] in, For A set of blind spots that can be simultaneously covered within a radius of two meters centered on the target area. Blind spot The coverage insufficiency score, For its geometric center, It is a smoothing constant; during path planning, an upper limit constraint is imposed on the path curvature to ensure that the turning angle formed by any three adjacent points does not exceed 30°, and the final generated guide trajectory is decomposed into a series of discrete target dwell points.

[0036] Furthermore, the method of the present invention also includes augmented reality visualization guidance: assigning suggested scanning attitude parameters to each of the target dwell points, including suggested pitch angle, suggested yaw angle, and suggested dwell time; sending the target dwell points and suggested scanning attitude parameters into the augmented reality rendering module; the augmented reality rendering module acquires real-time environmental video stream through the device's front-facing camera and overlays a semi-transparent virtual guide icon on it, including: drawing a dynamic arrow with the starting point being the screen projection of the device's current position and the ending point pointing to the screen projection of the next target dwell point; drawing a semi-transparent annular prompt box with a diameter mapped to a physical size of 30cm around the target dwell point; and displaying an attitude adjustment indicator in the form of a concentric circle dial, with the outer circle being the yaw angle scale and the inner circle being the pitch angle scale, the current device attitude being represented by a red pointer and the suggested attitude being represented by a green pointer, and when the deviation between the two exceeds five degrees, the corresponding scale area is highlighted and flashed.

[0037] Furthermore, the method of the present invention also includes a voice interaction guidance step, wherein the voice synthesis unit synchronously generates voice commands based on the relative state between the device and the target point: when the Euclidean distance between the device and the target dwell point is less than one meter, a voice prompt of "Approaching the target point, please slow down and prepare to stop" is triggered; when the device is at the target dwell point and the deviation of the current pitch angle or yaw angle from the recommended value exceeds 8° and lasts for more than 0.5 seconds, a voice command of orientation correction is triggered; when the device stays at the target dwell point for the recommended dwell time, and the real-time point cloud quality assessment module confirms that the point cloud density in the area exceeds 5,000 points per square meter and the normal vector consistency is higher than 90%, a voice prompt of "The data collection in this area is complete, and we are heading to the next target point" is triggered.

[0038] Furthermore, the method of the present invention also includes adaptive adjustment based on physiological state: the physiological state analysis engine is used to analyze the operator's state in real time, and specific indicators include: grip stability, movement speed fluctuation and gaze focus distribution; if the standard deviation of the X, Y, and Z axis accelerations of the inertial measurement unit exceeds 0.5 m / s² within a 1-second window, or the average rate of change of the displacement vector direction exceeds 45° per second between five consecutive frames, or two indicators of the proportion of gaze dwelling in the augmented reality guidance area exceed preset thresholds, then the operator is determined to be in a high-load state; when the operator is detected to be in a high-load state, the system automatically executes a simplified mode: only dynamic arrows and circular prompt boxes are retained in the augmented reality rendering module; the voice prompt interval is extended to twice the normal level; and the target dwell points are reordered, prioritizing guidance to the three blind spots with the highest coverage insufficiency scores, while other low-priority blind spots are temporarily deferred.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] (1) This invention innovatively transforms the complex problem of determining the integrity of 3D point cloud coverage into a geometric intersection problem of the "effective acquisition loop" and the "room boundary contour" in a 2D horizontal reference plane. This dimensionality reduction significantly reduces computational complexity, enabling millisecond-level real-time feedback even on handheld embedded devices with limited computing power. At the same time, by combining ray casting algorithm with the temporal information of continuous frame point clouds for occlusion state analysis, it can accurately identify real blind spots formed by physical occlusion, such as corners of walls and behind furniture, without any external model reference, thus solving the adaptability problem in unstructured environments.

[0041] (2) This invention does not use a fixed scanning range, but dynamically constructs an "effective acquisition ring" based on ambient lighting conditions (such as automatically narrowing the range under low light) and the real-time attitude of the device (elliptical projection caused by pitch and yaw angles). This mechanism ensures that the coverage area determined by the system is a truly effective high-quality data area, rather than just the line-of-sight area, avoiding invalid acquisition caused by low signal-to-noise ratio or excessive incident angle, and improving the accuracy of blind zone determination.

[0042] (3) Unlike a simple "present / absent" judgment, this invention introduces a coverage insufficiency scoring model, which integrates three dimensions: historical dwell time, angle diversity index, and point cloud density gradient variance. This not only identifies where there are blind spots, but also quantifies and sorts them according to the risk level of missing data, ensuring that subsequent guidance strategies can prioritize the most critical missing areas and improve the completeness of the final model.

[0043] (4) This invention generates a smooth guide trajectory through an improved RRT algorithm and combines augmented reality (AR) visual icons with voice interaction, which lowers the operating threshold and enables non-professionals to intuitively complete high-quality scans. In addition, the system can monitor the operator's physiological state (grip shaking, gaze focus, etc.) and automatically simplify the AR interface and optimize the guidance strategy when fatigue or high load is detected, effectively ensuring work safety and stability during long-term work.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, embodiments of the present invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic block diagram of the overall system architecture of the handheld point cloud scanning blind zone detection method described in this invention.

[0047] Figure 2 This is a schematic diagram illustrating the geometric relationship between the effective acquisition ring and the two-dimensional boundary contour map in the horizontal reference plane in this invention.

[0048] Figure 3 This is a flowchart illustrating the process of generating a candidate set of potential blind zones and resolving occlusion states in this invention.

[0049] Figure 4This is a schematic diagram of the supplementary sweep path planning and target dwell point generation based on the coverage deficiency score in this invention.

[0050] Figure 5 This is a schematic diagram illustrating the usage state of the augmented reality guidance interface and voice interaction working together in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0052] This invention provides a handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary. The overall system architecture and process are as follows: Figure 1 As shown, the specific implementation process is as follows. After the handheld scanning device is started, the method first performs an environmental idle detection phase. During this phase, the device is stationary, and the depth sensor array continuously acquires background point cloud data streams for ten seconds. The acquired point cloud data undergoes denoising and filtering by a preprocessing module to generate an initial noise baseline, which is then used to construct a static obstacle distribution map. This distribution map is used to eliminate interference from fixed structures on the line-of-sight path during subsequent occlusion state analysis.

[0053] After completing the environmental idle detection, the system enters dynamic scanning mode. In this mode, the system cyclically executes the complete blind zone detection and guidance process every twenty milliseconds. At the beginning of each cycle, the inertial measurement unit (IMU) and the global navigation satellite system (GNSS) receiver synchronously acquire the device's current spatial pose information, including three-dimensional position coordinates, pitch angle, yaw angle, and roll angle. Simultaneously, the depth sensor array captures the local point cloud data stream at the current moment. This local point cloud data stream is aligned with the IMU's timestamp through a hard synchronization circuit to ensure spatial consistency.

[0054] After receiving the aforementioned local point cloud data stream, the preprocessing module sequentially performs voxel downsampling, statistical outlier removal, and coordinate system transformation operations. The voxel side length for voxel downsampling is set to 5mm to reduce data volume while preserving geometric details; statistical outlier removal uses a neighborhood point threshold of 20 and a standard deviation factor of 1.5; the coordinate system transformation converts the point cloud from the sensor's local coordinate system to a global horizontal reference coordinate system with the device's current position as the origin, where the Z-axis points vertically upwards and the XY plane forms the horizontal reference plane.

[0055] The preprocessed point cloud is fed into the boundary feature extraction engine. This engine first calculates the normal vector for each point, uses K-nearest neighbor search to determine the neighborhood point set (K = 30), and then uses principal component analysis to find the eigenvector corresponding to the smallest eigenvalue of the local covariance matrix as the normal vector direction. Subsequently, the point cloud is clustered based on the normal vector direction. The clustering algorithm used is DBSCAN, with an angle tolerance of 15° and a distance tolerance of 50mm. In the clustering results, clusters with nearly vertical normal vector directions (i.e., an angle less than 25° with the Z-axis) are identified as candidate wall regions, while clusters with nearly horizontal normal vector directions (i.e., an angle greater than 65° with the Z-axis) are identified as candidate ground or ceiling regions.

[0056] For each candidate wall region, the boundary feature extraction engine performs RANSAC plane fitting with an upper limit of 1000 iterations and a distance threshold of 20mm. Successfully fitted planes are projected onto a horizontal reference plane, and their intersection with the ground plane forms the two-dimensional boundary segments. All such boundary segments are then connected at their endpoints and their closure is verified to form a complete two-dimensional boundary profile. This profile consists of several closed or multi-segment continuous polylines, representing the geometric boundaries of the room on the horizontal plane.

[0057] Simultaneously, the system dynamically constructs an effective acquisition ring based on the physical characteristics of the laser scanner (e.g., effective measurement range parameters) and the current ambient lighting conditions. The inner diameter of this effective acquisition ring... With outer diameter These are determined by the minimum and maximum effective measurement distances of the laser scanner, respectively. Under standard lighting conditions, , When the ambient light intensity is below a preset threshold (e.g., 100 lux), the system automatically reduces its size. The effective detection range is reduced to 6 meters to compensate for the decrease in signal-to-noise ratio.

[0058] The effective acquisition loop is not an ideal circle, but rather depends on the current pitch angle of the device. With yaw angle Ellipticization correction is performed. Specifically, within the horizontal reference plane, the outer ring boundary of the effective acquisition loop is described by the following parametric equation:

[0059]

[0060] in, For parameter variables, semi-major axis semi-short shaft The same applies to the inner ring. Alternative The resulting inner and outer elliptical boundaries together define a ring-shaped region, which is the projection of the effective acquisition ring onto the horizontal reference plane.

[0061] like Figure 2 The geometric relationship diagram shown illustrates that the 2D boundary contour map output by the boundary feature extraction engine and the projection of the effective acquisition ring are subjected to Boolean intersection operation on the same horizontal reference plane. This operation employs a polygon clipping algorithm from the computational geometry library, specifically the Weiler-Atherton algorithm, to identify boundary segments not covered by the effective acquisition ring. These uncovered boundary segments constitute a potential blind zone candidate set. Each candidate segment is stored as an ordered list of points, along with an identifier of its original boundary line.

[0062] The potential blind zone candidate set is then sent to the occlusion state resolution module. The specific generation and resolution process is as follows: Figure 3 As shown, this module uses three consecutive frames of historical point cloud data to perform ray casting detection on each candidate boundary segment. Specifically, a ray is emitted from the device's current pose position to each sampling point on the candidate segment, and it is checked whether the ray intersects with a non-boundary point in the historical point cloud before reaching the target point. If, in three consecutive frames, more than 80% of the sampling points on a certain boundary segment have a ray occluded at least once, the segment is determined to be in a true occlusion state and marked as a valid blind zone.

[0063] Effective blind spot information is transmitted to the coverage quality assessment unit. This unit calculates a coverage deficiency score for each effective blind spot. The calculation formula is as follows:

[0064]

[0065] in, This is the cumulative dwell time (in seconds) of the device within the projected area of ​​the blind zone over the past sixty seconds. To prevent division by zero of small constants; The angular diversity index is defined as the ratio of the number of sectors in the blind zone that the device successfully observed from different azimuth angles (divided into 12 sectors at 30° intervals) to the total number of sectors. The point cloud density gradient variance is obtained by performing a one-dimensional discrete Fourier transform on the density distribution of the point cloud at the blind zone edge along the normal direction and then taking the standard deviation of the low-frequency components; weighting coefficients. , , ,satisfy .

[0066] After receiving all valid blind spots and their corresponding coverage deficiency scores, the central collaborative controller drives the trajectory planning engine to generate the optimal rescan path. The detailed process of path planning and dwell point generation is as follows: Figure 4As shown, the engine first divides the working area into a grid map with sides of 20cm, marking grids occupied by known obstacles as impassable. Then, an improved Fast Random Tree Exploration (RRT) algorithm is used for pathfinding. The improvement lies in introducing an information gain function as a heuristic factor for node expansion. Information gain In position The definition of a location is:

[0067]

[0068] in, For A set of blind spots that can be simultaneously covered within a radius of two meters centered on the target area. Blind spot Coverage inadequacy score, For its geometric center, This is a smoothing constant. In this formula, the Euclidean distance between path nodes... Used as a metric to measure the cost of equipment movement, shorter distances mean less movement for the operator. When expanding new nodes, RRT prioritizes directions with high information gain and imposes upper limits on path curvature to ensure that the turning angle formed by any three adjacent points does not exceed 30°.

[0069] The guiding trajectory output by the trajectory planning engine is decomposed into a series of discrete target dwell points. Each target dwell point... The selection follows the principle of maximum information gain, meaning that while ensuring accessibility, locations that cover the most high-scoring blind spots are chosen. Simultaneously, suggested scanning attitude parameters, including suggested pitch angles, are assigned to each target dwell point. Recommended yaw angle and recommended length of stay It is recommended that the pitch angle be calculated based on the vertical height and horizontal distance of the target blind zone to ensure that the laser beam is directly facing the target area; it is recommended that the yaw angle be directed towards the center of the blind zone; it is recommended that the dwell time be proportional to the coverage insufficiency score, with a base value of 1.5 seconds, and the score is extended by 0.1 seconds for each additional 0.1 points, with an upper limit of 4 seconds.

[0070] like Figure 5The diagram illustrates the usage state. The target dwell point and suggested scanning attitude parameters are sent to the augmented reality rendering module. This module acquires a real-time environmental video stream through the front-facing camera and overlays a virtual guide icon on it. Specifically, a dynamic arrow is drawn in the video frame, starting from the screen projection of the device's current position and ending at the screen projection of the next target dwell point. A semi-transparent circular prompt box is drawn around the target dwell point, with a physical size of 30cm in diameter mapped to pixels according to the current depth ratio. The attitude adjustment indicator is displayed in the upper right corner of the screen as a concentric circle scale, with the outer circle for yaw angle and the inner circle for pitch angle. The current device attitude is indicated by a red pointer, and the suggested attitude is indicated by a green pointer. When the deviation between the two exceeds five degrees, the corresponding scale area is highlighted and flashes.

[0071] The speech synthesis unit synchronously generates voice commands matching the current guidance phase. When the Euclidean distance between the device and the target dwell point is less than one meter, a voice prompt "Approaching the target point, please slow down and prepare to stop" is triggered; when the device's current pitch or yaw angle deviates from the recommended value by more than 8° and the duration exceeds 0.5 seconds, a voice prompt "Please adjust the device angle left / right / up / down" is triggered; when the device stays at the target point for the recommended duration and the real-time point cloud quality assessment module confirms that the point cloud density in the area exceeds 5,000 points per square meter and the normal vector consistency is higher than 90%, a voice prompt "Data acquisition in this area is complete, proceeding to the next target point" is triggered.

[0072] The physiological state analysis engine assesses the operator's cognitive load by analyzing grip stability, movement speed fluctuations, and gaze focus distribution. Grip stability is measured by the standard deviation of acceleration from the inertial measurement unit; if the standard deviation of acceleration on the X, Y, and Z axes all exceeds 0.5 m / s² within a 1-second window, it is considered unstable. Movement speed fluctuations are obtained by calculating the rate of change of displacement vector direction over five consecutive frames; if the average angular velocity exceeds 45 degrees per second, it is considered hasty operation. Gaze focus distribution is estimated by combining the front-facing camera with a lightweight facial landmark detection model, and the proportion of time the gaze stays within the AR-guided area is statistically analyzed. If two of the above three indicators exceed the threshold, the operator is determined to be fatigued or distracted.

[0073] When the system detects that the operator is under high load, it automatically simplifies the AR interface elements, retaining only the dynamic arrows and circular prompts, and hiding the posture adjustment indicator; the voice prompt interval is extended to twice the normal interval; at the same time, the trajectory planning engine reorders the target dwell points, prioritizing the guidance to the three blind spots with the highest coverage insufficiency scores, while other low-priority blind spots are temporarily deferred to reduce operational complexity.

[0074] All software modules are deployed on the embedded computing platform of the handheld scanning device. This platform uses a 2.4GHz quad-core ARM Cortex-A76 processor, paired with a Mali-G77 GPU, and runs a custom operating system based on the Linux 5.10 kernel. To ensure efficient collaboration between the sensors and processing units, the depth sensor array and the inertial measurement unit are connected via a hard synchronization circuit to ensure microsecond-level spatiotemporal alignment; the global navigation satellite system receiver is connected to the central processing unit via a serial communication interface (UART); the graphics processing unit communicates directly with the central processing unit via the PCIe bus to meet the high bandwidth requirements of AR rendering; the audio output device is connected to the digital signal processing unit via the I2S bus; the front-facing camera is directly connected to the image signal processor using a MIPI CSI-2 interface; and the user interface (touchscreen and buttons) is managed by a GPIO controller. Each module runs as an independent process, and inter-process communication is achieved through POSIX shared memory and semaphore mechanisms. The boundary feature extraction engine and the occlusion state analysis module share the same point cloud buffer. This buffer adopts a ring buffer structure with a capacity of the most recent ten frames of point cloud data. Each frame of point cloud is stored in PCL format. The trajectory planning engine and the augmented reality rendering module exchange guiding trajectory data through a double buffering mechanism. The write buffer is updated by the trajectory planning engine, and the read buffer is read by the rendering module. The exchange is controlled by atomic flag bits. The speech synthesis unit and the physiological state analysis engine adopt an event-driven model and receive state change notifications from other modules through the kernel event queue.

[0075] In one specific embodiment, the operator uses the method described in this invention to perform point cloud scanning on an office with an area of ​​40 square meters. The office contains temporary obstacles such as desks, filing cabinets, and sofas. The system completes environmental vacancy detection and establishes an initial static obstacle distribution map within ten seconds of startup. After entering dynamic scanning mode, the system identifies six potential blind spot candidate sets within the first minute. After occlusion status analysis, four of these are confirmed as effective blind spots, located behind the filing cabinet, on the side of the sofa, at the corner of the wall, and inside the door frame, respectively. The coverage quality assessment unit calculates their coverage deficiency scores to be 0.78, 0.65, 0.82, and 0.59, respectively. The guidance path generated by the trajectory planning engine passes sequentially through the center points of two blind spots with scores of 0.82 and 0.78, with suggested dwell times of 3.3 seconds and 3.1 seconds, respectively. Guided by AR arrows and voice prompts, the operator successfully completes the supplementary scanning. At the end of the third minute, the system detects that all blind spot scores have dropped below 0.2, and no new effective blind spots have been added for five consecutive seconds, thus triggering the task completion confirmation process.

[0076] To verify the technical effectiveness of this invention, a comparative experiment was conducted using a scale comparison method. The scale comparison employed a traditional blind spot detection method based on a complete prior model, which involves pre-importing a building BIM model and determining blind spots by comparing the current scanned point cloud with the residual area after model registration. The experiment was conducted in the same office setting, with the same operator performing the same scanning task.

[0077] The experimental results are shown in Table 1:

[0078] Table 1

[0079] index Method of the present invention Traditional methods First-time blind zone detection delay (milliseconds) 42 210 Total time for a single scan task (seconds) 182 245 Final point cloud completeness (percentage) 98.7 92.3 Number of missed detection blind spots caused by temporary obstacles 0 3 Average CPU utilization (percentage) 68 89 Operator's subjective difficulty rating (1-5 points, 1 being extremely easy) 1.8 3.6

[0080] Data shows that this invention significantly outperforms traditional methods that rely on prior models in terms of real-time performance, completeness, environmental adaptability, and ease of operation. Especially in scenarios with temporary obstacles, this invention can accurately identify and guide the scanning of real blind spots, while the comparative method fails to detect multiple blind spots due to discrepancies between the model and the actual environment.

[0081] Furthermore, the method described in this invention also demonstrates good adaptability in industrial facility inspection scenarios. In a chemical plant pump room, the dense network of pipes and complex lighting conditions render traditional methods ineffective due to the inability to establish reliable prior models. This invention, through real-time boundary extraction and dynamic occlusion perception, successfully identifies multiple narrow blind spots formed by pipe occlusion and, through optimized guidance trajectories, enables operators to complete high-quality supplementary cleaning without touching the high-temperature pipes.

[0082] In summary, this invention transforms the problem of determining the integrity of 3D point cloud coverage into a geometric intersection problem between an effective acquisition loop and the room boundary line in a 2D plane, achieving millisecond-level blind zone detection with low computational overhead. Combining real-time boundary extraction, dynamic occlusion perception, coverage quality quantification assessment, and predictive path planning, the system possesses adaptive understanding capabilities for unstructured environments. Through the deep integration of augmented reality visualization and voice interaction, a natural and intuitive human-computer collaboration interface is constructed, effectively reducing the operational threshold and improving the success rate of single scans and overall work efficiency. All algorithm processes are deeply optimized for the computing power characteristics of mobile terminals, and can stably run at a processing rate of over thirty frames per second on typical embedded platforms, meeting the stringent real-time requirements of field operations.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary, characterized in that, include: Step S1: After the handheld scanning device is started and enters the dynamic scanning mode, the current spatial pose information of the device is obtained synchronously with the global navigation satellite system receiver through the built-in inertial measurement unit, and the local point cloud data stream at the current moment is continuously captured by the depth sensor array. Step S2: Preprocess the local point cloud data stream and transform the point cloud coordinates from the sensor local coordinate system to the global horizontal reference coordinate system with the current position of the device as the origin. The Z-axis of the global horizontal reference coordinate system is vertically upward, and the XY plane forms a horizontal reference plane. Step S3: Use the boundary feature extraction engine to identify the set of room boundary lines composed of walls and static structures from the preprocessed point cloud, and project the set of room boundary lines onto the horizontal reference plane to construct a two-dimensional boundary contour map; Step S4: Dynamically construct an effective acquisition ring based on the effective measurement range parameters of the laser scanner and the current ambient lighting conditions, and project the effective acquisition ring onto the horizontal reference plane to form a ring-shaped area with dynamically adjustable boundaries centered on the current position of the device; Step S5: Perform geometric Boolean operations on the effective acquisition ring and the two-dimensional boundary contour map in the horizontal reference plane to identify boundary segments not covered by the effective acquisition ring and form a potential blind zone candidate set; Step S6: Input the potential blind zone candidate set into the occlusion state analysis module. Utilize the point cloud temporal information between consecutive frames and combine it with the ray casting algorithm to determine whether the line of sight of each candidate boundary segment is blocked by an obstacle. If it is blocked, the boundary segment is determined to be in a true occlusion state and is marked as an effective blind zone.

2. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 1, characterized in that, Before step S1, the method further includes an environmental idle detection stage: keeping the device in a stationary state, collecting background point cloud data for ten seconds using the depth sensor array, establishing an initial noise baseline and constructing a static obstacle distribution map to exclude fixed structure interference in occlusion state analysis; In step S2, the specific preprocessing operations include: setting the voxel side length of the voxel downsampling to 5mm; and performing statistical outlier removal to remove noise using a parameter configuration of a neighborhood point threshold of 20 and a standard deviation multiple of 1.

5. Furthermore, spatial consistency is ensured by aligning the local point cloud data stream with the timestamp of the inertial measurement unit through a hard synchronization circuit.

3. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 1, characterized in that, The specific execution logic of the boundary feature extraction engine in step S3 is as follows: First, calculate the normal vector of each point, use K-nearest neighbor search to determine the neighborhood point set, with K value set to 30, and use principal component analysis to solve for the eigenvector corresponding to the smallest eigenvalue of the local covariance matrix as the direction of the normal vector. Subsequently, the point cloud was clustered based on the normal vector direction using the DBSCAN algorithm, with an angle tolerance of 15° and a distance tolerance of 50mm. In the clustering results, clusters with a normal vector direction and a Z-axis angle of less than 25° were identified as candidate wall areas, and clusters with a normal vector direction and a Z-axis angle of greater than 65° were identified as candidate ground or ceiling areas. Finally, RANSAC plane fitting is performed on each candidate wall area, with an upper limit of 1000 iterations and a distance threshold of 20mm. The successfully fitted plane is projected onto the horizontal reference plane, and its intersection with the ground plane is extracted as a two-dimensional boundary line segment. After endpoint connection and closure verification, the two-dimensional boundary contour map is formed.

4. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 1, characterized in that, The effective measurement range parameter mentioned in step S4 includes the minimum effective measurement distance of the laser scanner. With the maximum effective measurement distance ; The construction rule for the effective acquisition loop is as follows: under standard illumination conditions, set... , ; When the ambient lighting conditions indicate that the light intensity is below a preset threshold of 100 lx, automatically... Reduced to 6 meters; The shape of the effective acquisition ring is based on the current pitch angle of the device. With yaw angle After ellipticization correction, within the horizontal reference plane, the outer ring boundary of the effective acquisition ring is described by the following parametric equation: in, For parameter variables, semi-major axis semi-short shaft And the inner ring boundary adopts Alternative Perform the same calculations; the effective acquisition ring is the projection of the annular region defined by the inner ring boundary and the outer ring boundary onto the horizontal reference plane.

5. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 1, characterized in that, The geometric Boolean operation in step S5 uses the Weiler-Atherton polygon clipping algorithm to accurately identify the boundary segments in the two-dimensional boundary contour map that are not covered by the effective acquisition ring. The specific determination logic of the occlusion state analysis module in step S6 is as follows: using three consecutive frames of historical point cloud data, a ray is emitted from the current pose position of the device to each sampling point on each candidate boundary segment in the potential blind zone candidate set, and it is checked whether the ray intersects with non-boundary points in the historical point cloud before reaching the target point. If, in three consecutive frames, more than 80% of the sampling points on a certain boundary segment have at least one instance of ray occlusion, then the boundary segment is determined to be in a true occlusion state and is marked as the effective blind zone.

6. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 1, characterized in that, It also includes a coverage quality assessment of the effective blind area, using a coverage quality assessment unit to calculate a coverage deficiency score based on the device's historical dwell time in the vicinity of the area, the scanning angle diversity index, and the point cloud density gradient change rate. The calculation formula is as follows: in, This refers to the cumulative dwell time of the device within the effective blind zone projection area over the past sixty seconds, expressed in seconds. To prevent division by zero of small constants; The angular diversity index is defined as the ratio of the number of sectors in the blind zone that the device can successfully observe from different azimuth angles to the total number of sectors, wherein the azimuth angles are divided into 12 sectors at 30° intervals. The point cloud density gradient variance is obtained by performing a one-dimensional discrete Fourier transform on the density distribution of the point cloud at the blind zone edge along the normal direction and then taking the standard deviation of the low-frequency components; the weighting coefficients satisfy... , , .

7. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 6, characterized in that, It also includes supplementary cleaning path planning: The central collaborative controller receives the list of effective blind spots and their corresponding coverage insufficiency scores, and drives the trajectory planning engine to generate the optimal supplementary scanning path. The trajectory planning engine first divides the working area into a grid map with a side length of 20cm, and marks the grids occupied by known obstacles as impassable. Subsequently, a fast exploratory random tree algorithm, incorporating an information gain function as a heuristic factor for node expansion, is employed for path search. In position The definition of a location is: in, For A set of blind spots that can be simultaneously covered within a radius of two meters centered on the target area. Blind spot The coverage insufficiency score, For its geometric center, It is a smoothing constant; during path planning, an upper limit constraint is imposed on the path curvature to ensure that the turning angle formed by any three adjacent points does not exceed 30°, and the final generated guide trajectory is decomposed into a series of discrete target dwell points.

8. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 7, characterized in that, It also includes augmented reality visualization guidance: assigning suggested scanning attitude parameters to each of the target dwell points, including suggested pitch angle, suggested yaw angle and suggested dwell time; The target dwell point and the suggested scanning posture parameters are sent to the augmented reality rendering module; The augmented reality rendering module acquires real-time environmental video streams through the device's front-facing camera and overlays semi-transparent virtual guide icons on them, including: drawing dynamic arrows with the screen projection starting from the device's current position and the screen projection ending at the next target dwell point; A semi-transparent ring-shaped prompt box with a diameter mapped to a physical size of 30cm is drawn around the target dwell point; and an attitude adjustment indicator in the form of a concentric circle dial is displayed, with the outer circle being the yaw angle scale and the inner circle being the pitch angle scale. The current attitude of the device is indicated by a red pointer, and the recommended attitude is indicated by a green pointer. When the deviation between the two exceeds five degrees, the corresponding scale area is highlighted and flashed.

9. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 8, characterized in that, It also includes a voice interaction guidance step, where the voice synthesis unit synchronously generates voice commands based on the relative state between the device and the target point: when the Euclidean distance between the device and the target dwell point is less than one meter, a voice prompt is triggered that the device is about to reach the target point and should slow down to prepare to stop; when the device is at the target dwell point and the deviation of the current pitch angle or yaw angle from the recommended value exceeds 8° and lasts for more than 0.5 seconds, a voice command for orientation correction is triggered; when the device stays at the target dwell point for the recommended dwell time, and the real-time point cloud quality assessment module confirms that the point cloud density in the area exceeds 5,000 points per square meter and the normal vector consistency is higher than 90%, a voice prompt is triggered that the data acquisition in the area is complete and the device is moving to the next target point.

10. The handheld point cloud scanning blind spot detection method based on the interaction between the acquisition area and the boundary as described in claim 8, characterized in that, It also includes adaptive adjustment based on physiological state: the physiological state analysis engine analyzes the operator's state in real time, with specific indicators including grip stability, movement speed fluctuation, and gaze focus distribution; if the standard deviation of the X, Y, and Z axis accelerations of the inertial measurement unit exceeds 0.5 m / s² within a 1-second window, or the average rate of change of the displacement vector direction exceeds 45° per second between five consecutive frames, or two indicators of the proportion of gaze dwelling in the augmented reality guidance area exceed preset thresholds, then the operator is determined to be in a high-load state; when the operator is detected to be in a high-load state, the system automatically executes a simplified mode: only dynamic arrows and circular prompt boxes are retained in the augmented reality rendering module; the voice prompt interval is extended to twice the normal level; and the target dwell points are reordered, prioritizing guidance to the three blind spots with the highest coverage insufficiency scores, while other low-priority blind spots are temporarily deferred.

Images