Baggage detection method and system based on point cloud graph plane projection mixed bounding box

Abstract

This invention discloses a hybrid bounding box method and system for baggage detection based on point cloud image planar projection. The method includes: acquiring 3D point cloud data of a target scene; establishing a dynamic quality benchmark based on the extreme value distribution of the 3D point cloud data in a preset direction; segmenting the 3D point cloud data using a cascaded segmentation strategy, wherein the cascaded segmentation strategy includes at least two segmentation algorithms with different segmentation characteristics; after each segmentation algorithm is executed, performing real-time quality evaluation on the current segmentation result based on the dynamic quality benchmark, and generating a quality evaluation result; determining whether to trigger the next level segmentation algorithm or output the current segmentation result based on the quality evaluation result; and generating the minimum bounding box of the target baggage based on the final output segmentation result. This invention, by constructing a dynamic quality benchmark and a cascaded segmentation strategy, achieves efficient and robust segmentation of baggage point clouds in complex scenes, significantly improving the accuracy and adaptability of baggage detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D point cloud processing technology, and more specifically to a hybrid bounding box baggage detection method and system based on point cloud map planar projection. Background Technology

[0002] In automated sorting and transportation systems, the rapid and accurate identification and positioning of luggage in scenarios such as conveyor belts, pallets, and check-in counters is a key link in improving operational efficiency and ensuring operational safety.

[0003] In the field of 3D point cloud processing, point cloud segmentation is fundamental to advanced tasks such as target recognition and scene understanding. Especially in industrial vision applications requiring precise extraction and localization of specific targets (such as airport baggage sorting), segmentation accuracy, robustness, and adaptability are key evaluation metrics. However, current mainstream segmentation methods, when faced with complex and variable real-world point clouds, have inherent limitations that make it difficult for a single algorithm to achieve ideal results in complex and ever-changing baggage sorting system scenarios.

[0004] Therefore, there is an urgent need in this field for a more flexible and adaptive segmentation scheme that can dynamically allocate or fuse different processing strategies according to the specific characteristics of the input point cloud, thereby achieving a better balance between accuracy, speed and practicality. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a hybrid bounding box baggage detection method based on point cloud map planar projection that overcomes or at least partially solves the above problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] S1. Acquire the three-dimensional point cloud data of the target scene, and establish a dynamic quality benchmark based on the extreme value distribution of the three-dimensional point cloud data in a preset direction. S2. The 3D point cloud data is segmented using a cascaded segmentation strategy, wherein the cascaded segmentation strategy includes at least two segmentation algorithms with different segmentation characteristics; S3. After each level of segmentation algorithm is executed, the current segmentation result is evaluated in real time based on the dynamic quality benchmark, and a quality evaluation result is generated. S4. When the quality evaluation result meets the preset conditions, output the current segmentation result and terminate the subsequent segmentation algorithm; When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is not the last level, the next level segmentation algorithm is triggered. When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is already the last level, the current segmentation result is forcibly output. S5. Generate the minimum bounding box of the target baggage based on the segmentation results.

[0008] Preferably, step S1 further includes: Based on the maximum coordinate value of the three-dimensional point cloud data in the preset direction max(P y ) with minimum coordinate value min(P y ) Calculate the point cloud span in the preset direction. Y range As the dynamic quality benchmark:

[0009] The preset direction is the direction in which the luggage travels on the conveyor belt.

[0010] Preferably, step S3 further includes: Extract the local maximum coordinate values ​​of point cloud clusters obtained by any level segmentation algorithm in the preset direction. max (P sub_y ) with local minimum coordinates min(P sub_y ) Calculate longitudinal coverage y ratio As the quality evaluation result: .

[0011] Preferably, step S3 further includes: Calculate the number of point clouds in the current point cloud cluster. Count(P sub ) The total number of points in the three-dimensional point cloud data Count(P total ) The ratio of the points yields the percentage of points. P ratio As a result of the aforementioned quality evaluation: .

[0012] Preferably, step S4 specifically includes: When the longitudinal coverage y ratio The percentage of points is greater than the first preset threshold. P ratio When the value is greater than the second preset threshold, the point cloud cluster obtained by the current segmentation is determined to meet the quality requirements, the current point cloud cluster is output and the subsequent segmentation algorithm is terminated. Otherwise, if a next-level segmentation algorithm exists, it will be triggered; if no next-level segmentation algorithm exists, the current segmentation result will be forcibly output.

[0013] Preferably, the cascaded segmentation strategy includes: The first-level segmentation uses a connected component segmentation algorithm based on spatial connectivity to extract structurally continuous point cloud clusters; The second-level segmentation is triggered when the quality evaluation result of the point cloud clusters obtained from the first-level segmentation does not meet the preset conditions. The adaptive DBSCAN segmentation algorithm based on density clustering is used to cluster targets with sparse or broken point clouds. The third-level segmentation is triggered when the quality evaluation result of the point cloud cluster obtained from the second-level segmentation does not meet the preset conditions. A forced segmentation algorithm based on statistical features is used to extract the target core region.

[0014] Preferably, the first-level segmentation algorithm includes: Using a neighborhood search radius R and a minimum point count threshold MinPts, spatial topological clustering is performed on the 3D point cloud data based on Euclidean distance. Point clouds with a distance less than or equal to R are grouped into the same cluster, and clusters with a point count greater than MinPts are selected as candidate point cloud clusters. The point cloud cluster with the most points is then selected as the current segmentation result.

[0015] Preferably, the second-level segmentation algorithm includes: Calculate the average distance of the k-th nearest neighbor for each point in the 3D point cloud data. avg_dist , where k is a preset value; According to the average distance avg_dist Multiple sets of neighborhood search radii, ranging from small to large, are generated according to a preset scaling factor. Eps i And configure a corresponding density threshold for the search radius of each neighborhood group. MinPts i This constitutes the parameter search space. M= {(Eps i ,MinPts i )|i=1,2,…,n} ; A set of parameters is selected sequentially from the parameter search space, and DBSCAN clustering is performed to obtain candidate point cloud clusters; Extract the point cloud cluster with the most points and whose position conforms to the preset spatial constraints from the candidate point cloud clusters, and use it as the point cloud cluster obtained by the current segmentation. Call steps S3 and S4 to perform a quality evaluation on the current point cloud cluster. If the quality evaluation result meets the preset conditions, terminate the search and output the current point cloud cluster. If the quality evaluation result does not meet the preset conditions and there are untried parameter combinations in the parameter search space, then continue to select the next set of parameters to try; If all parameter combinations have been tried and the quality evaluation results do not meet the preset conditions, the third-level segmentation algorithm is triggered.

[0016] Preferably, the third-level segmentation algorithm includes: Calculate the geometric center of all points in the 3D point cloud data. C global ; Calculate the distance from each point in the 3D point cloud data to the geometric center. C global The Euclidean distance is used to obtain the distance set. {d i } ; The distance set {d i } Sort the data in ascending order and use the distance value at the preset quantile as the distance threshold. T limit ; Extraction distance less than or equal to the distance threshold T limit The point cloud is used as the point cloud cluster obtained by segmentation.

[0017] Based on the same inventive concept, this invention also discloses a hybrid bounding box baggage detection system based on point cloud map planar projection, used to execute any of the above-mentioned hybrid bounding box baggage detection methods based on point cloud map planar projection, including: The data acquisition module is used to acquire 3D point cloud data of the target scene; The benchmark establishment module is used to establish a dynamic quality benchmark based on the extreme value distribution of the three-dimensional point cloud data in a preset direction. The cascaded segmentation module includes at least two segmentation units with different segmentation characteristics, used to segment the three-dimensional point cloud data to obtain point cloud clusters; The quality evaluation module is used to perform real-time quality evaluation on the point cloud clusters obtained by the current segmentation based on the dynamic quality benchmark after each segmentation unit is executed, and generate quality evaluation results. The scheduling and control module is used to make decisions based on the quality evaluation results: When the quality evaluation result meets the preset conditions, the current segmentation result is output and the subsequent segmentation unit is terminated; When the quality evaluation result does not meet the preset conditions and the current segmentation unit is not the last level, the next level segmentation unit is triggered; When the quality evaluation result does not meet the preset conditions and the current segmentation unit is already the last level, the current segmentation result is forcibly output. The bounding box generation module is used to generate the minimum bounding box of the target luggage based on the segmentation results of the final output.

[0018] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following: 1. Improve segmentation robustness in complex scenarios By employing a cascaded segmentation strategy that includes at least two different segmentation characteristics and introducing a real-time quality evaluation and scheduling mechanism based on dynamic quality benchmarks, this invention can adaptively select or switch segmentation algorithms according to point cloud quality, effectively overcoming the limitations of a single algorithm in dealing with complex conditions such as different materials (e.g., light absorption, light reflection), occlusion, or point cloud fragmentation, and significantly improving the success rate and stability of baggage detection.

[0019] 2. Achieve adaptive closed-loop control of segmentation quality. This invention performs real-time evaluation based on a dynamic quality benchmark after each level of segmentation, and determines whether to terminate or continue subsequent segmentation based on the evaluation results, forming a closed-loop feedback mechanism of "segmentation-evaluation-decision". This mechanism can dynamically adapt to luggage of different sizes and shapes, avoiding misjudgments or missed detections caused by fixed thresholds, and improving the geometric integrity and structural representativeness of the segmentation results.

[0020] 3. Optimize computational efficiency and reduce system overhead. By employing an "early exit" mechanism in the cascaded structure, subsequent complex algorithms are terminated promptly when the early segmentation results meet quality requirements, avoiding unnecessary consumption of computational resources. Experimental data shows that the processing speed can be improved by approximately 60% under normal operating conditions, and the operator energy efficiency advantage of over 50% can still be maintained in complex material scenarios, balancing detection accuracy and real-time performance.

[0021] 4. Enhance system availability under extreme operating conditions When all conventional segmentation methods fail to meet quality requirements, this invention uses a final-level forced segmentation algorithm based on statistical features to still extract the core area of ​​the target and output the positioning result. This ensures the sorting system's continuous operation capability under extreme conditions such as severe occlusion and extremely sparse point clouds, achieving the engineering goal of "never running idle" in industrial applications.

[0022] 5. Improve the applicability and generalization ability of baggage inspection. The dynamic quality benchmark is established based on the extreme value distribution of point cloud in a preset direction, which enables the evaluation criteria to automatically scale with the target size. It is applicable to various targets from large checked baggage to small carry-on bags, enhancing the versatility and generalization ability of the method in different sorting scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 This is a flowchart of the hybrid bounding box baggage detection method based on point cloud map planar projection provided in this embodiment of the invention; Figure 2 This is a framework diagram of a hybrid bounding box baggage detection system based on point cloud map planar projection provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, this embodiment discloses a hybrid bounding box baggage detection method based on point cloud map planar projection, including: S1. Acquire the 3D point cloud data of the target scene, and establish a dynamic quality benchmark based on the extreme value distribution of the 3D point cloud data in a preset direction. S2. A cascaded segmentation strategy is used to segment the 3D point cloud data. The cascaded segmentation strategy includes at least two segmentation algorithms with different segmentation characteristics. S3. After each level of the segmentation algorithm is executed, the current segmentation result is evaluated in real time based on the dynamic quality benchmark, and a quality evaluation result is generated. S4. When the quality evaluation result meets the preset conditions, output the current segmentation result and terminate the subsequent segmentation algorithm; When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is not the last level, the next level segmentation algorithm is triggered. When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is already the last level, the current segmentation result is forcibly output. S5. Generate the minimum bounding box of the target baggage based on the segmentation results.

[0027] The method of the present invention will now be described in detail.

[0028] First, 3D point cloud data of the target scene is acquired using 3D sensors. To overcome the problem that a fixed threshold cannot adapt to luggage of different sizes, this invention pre-analyzes the spatial distribution of the point cloud before performing segmentation, establishing an adaptive quality assessment benchmark.

[0029] In one embodiment, after acquiring the raw baggage point cloud data P, a lightweight pre-inspection mechanism is first initiated. This is done by counting the total number of points in the input frames. N Compare it with the preset minimum confidence threshold. N min Compare: like N < N min It immediately determines that there is severe occlusion, light interference, or target missing in the current acquisition environment, and automatically triggers the anomaly protection mechanism to stop subsequent complex algorithm calculations.

[0030] Technical significance: This step effectively avoids blind calculation of low-quality point clouds, reduces the system's ineffective computing power overhead, and improves the overall response speed of the automated pipeline.

[0031] In one embodiment, step S1 further includes: The point cloud span Yrange in the preset direction is calculated based on the maximum and minimum coordinate values ​​max(Py) of the 3D point cloud data in the preset direction, and is used as a dynamic quality benchmark.

[0032] The preset direction is the direction in which the luggage travels on the conveyor belt.

[0033] This parameter, serving as the core reference quantity of this invention, is used to quantify the original physical length of luggage in the direction of conveyor belt travel. Instead of using a fixed absolute length value as the judgment criterion, the system dynamically generates a structural integrity definition for the current target by calculating the ratio between the segment cluster span and the original envelope span. This ratio-based dynamic benchmark provides a quantifiable self-checking threshold for each stage of the cascaded algorithm, effectively solving the problem that a single static threshold cannot adapt to luggage of different sizes.

[0034] In one embodiment, step S3 further includes: Extract the local maximum coordinate value max(Psub_y) and local minimum coordinate value min(Psub_y) of the point cloud cluster obtained from the current segmentation in a preset direction, and calculate the longitudinal coverage yratio as the quality evaluation result using the following formula:

[0035] Here, Yrange, which is the dynamic quality benchmark yratio established based on the original point cloud, has the physical meaning of measuring whether the continuity of the point cloud of the target in the direction of the conveyor belt movement has reached the logical expectation. If yratio approaches 1.0, it indicates that the current algorithm has completely restored the longitudinal geometric outline of the luggage; if yratio is lower than 0.3, it is judged as "target fragmentation", that is, the algorithm has divided the complete luggage into multiple discrete segments.

[0036] In one embodiment, step S3 further includes: Calculate the number of point clouds in the current point cloud cluster. Count(P sub ) Total number of points in 3D point cloud data Count (P total ) The ratio of the points yields the percentage of points. P ratio As a result of quality evaluation:

[0037] Specifically, to improve fault tolerance, this assessment center not only monitors longitudinal coverage but also introduces auxiliary verification based on the percentage of points, forming a dual-judgment branch: Branch A (pass condition): when satisfied y ratio > (where the threshold) (Set to 0.6) and local point count P ratio When the number of points reaches more than 30% of the original total number of points N, the system determines that the current segmentation result is structurally representative. At this point, the evaluation center sends a "stop search" signal to the system, marks the current point cloud cluster as the "optimal result," and outputs it.

[0038] Branch B (Jump Condition): If any of the above conditions are not met, the system determines that the current algorithm has over-segmented or that point cloud drift has occurred due to material interference. At this time, the evaluation center will generate a "logical degradation rollback" instruction, directing the data flow to enter the next level of more powerful segmentation operator (such as jumping from connected components to adaptive clustering in the following embodiment).

[0039] Existing technologies typically use fixed distance thresholds, which are prone to false alarms when processing stacked or small items. This invention addresses this issue by... y ratio The proportional design makes: For oversized luggage: The system automatically increases the absolute length of the judgment threshold to prevent grabbing deviation caused by only picking up the edge of the luggage.

[0040] For very small luggage: due toY range Despite the relatively small base number, the system is still able to accurately capture its simplified outline at a rate of 60%.

[0041] This dynamic evaluation mechanism, which adapts to the specific item, forms the decision-making center of the entire cascaded architecture, enabling digital and closed-loop control of baggage segmentation quality.

[0042] In one embodiment, step S4 specifically includes: When vertical coverage y ratio Greater than the first preset threshold and the percentage of points P ratio When the value is greater than the second preset threshold, the point cloud cluster obtained by the current segmentation is determined to meet the quality requirements, the current point cloud cluster is output and the subsequent segmentation algorithm is terminated. Otherwise, if a next-level segmentation algorithm exists, it is triggered; if no next-level segmentation algorithm exists, the current segmentation result is forcibly output. (This has been described in detail in the above embodiment and will not be repeated here).

[0043] It should be noted that different quality thresholds are used for each level of the segmentation algorithm: the first-level connected component segmentation requires... y ratio >0.6 and P ratio >0.3; Second-level improved DBSCAN clustering requirements y ratio >0.5 and P ratio >0.3; The third-level spatial range segmentation serves as a safety net, eliminating quality checks and forcing the output of the result. This tiered threshold design allows the system to appropriately relax geometric integrity requirements during gradual degradation, maximizing accuracy while ensuring availability.

[0044] In one embodiment, the cascaded segmentation strategy includes: The first-level segmentation uses a connected component segmentation algorithm based on spatial connectivity to extract structurally continuous point cloud clusters; The second-level segmentation is triggered when the quality evaluation result of the point cloud clusters obtained from the first-level segmentation does not meet the preset conditions. The adaptive DBSCAN segmentation algorithm based on density clustering is used to cluster targets with sparse or broken point clouds. The third-level segmentation is triggered when the quality evaluation result of the point cloud cluster obtained from the second-level segmentation does not meet the preset conditions. A forced segmentation algorithm based on statistical features is used to extract the target core region.

[0045] 1. The first-level segmentation algorithm includes: Using the neighborhood search radius R and the minimum number of points threshold MinPts, spatial topological clustering of 3D point cloud data is performed based on Euclidean distance. Point clouds with a distance less than or equal to R are grouped into the same cluster, and clusters with a number of points greater than MinPts are selected as candidate point cloud clusters. The point cloud cluster with the most points is selected as the current segmentation result.

[0046] 2. The second-level segmentation algorithm includes: Calculate the average distance of the k-th nearest neighbor for each point in the 3D point cloud data. avg_dist , where k is a preset value; Based on average distance avg_dist Multiple sets of neighborhood search radii, ranging from small to large, are generated according to a preset scaling factor. Eps i And configure a corresponding density threshold for the search radius of each neighborhood group. MinPts i This constitutes the parameter search space. M={(Eps i , MinPts i )|i=1,2,…,n} ; A set of parameters is selected sequentially from the parameter search space, and DBSCAN clustering is performed to obtain candidate point cloud clusters; Extract the point cloud cluster with the most points and whose position meets the preset spatial constraints from the candidate point cloud clusters, and use it as the point cloud cluster obtained by the current segmentation. Call steps S3 and S4 to evaluate the quality of the current point cloud cluster. If the quality evaluation result meets the preset conditions, terminate the search and output the current point cloud cluster. If the quality evaluation result does not meet the preset conditions and there are untried parameter combinations in the parameter search space, then continue to select the next set of parameters to try; If all parameter combinations have been tried and the quality evaluation results do not meet the preset conditions, the third-level segmentation algorithm is triggered.

[0047] 3. The third-level segmentation algorithm includes: Calculate the geometric center of all points in a 3D point cloud dataset. C global ; Calculate the distance from each point in a 3D point cloud to its geometric center. C global The Euclidean distance is used to obtain the distance set. {d i } ; Distance set {d i } Sort the data in ascending order and use the distance value at the preset quantile as the distance threshold. Tlimit ; Extract the point cloud with a distance less than or equal to the distance threshold T limit as the point cloud clusters obtained by segmentation.

[0048] Specifically, 1. The structure-guided improved connected component guiding strategy, as the first-level execution unit of the cascaded system, this algorithm aims to utilize the spatial topological characteristics of the point cloud to quickly strip the luggage with normal reflectivity and continuous structure in milliseconds.

[0049] (1) The core algorithm logic and parameter modeling algorithm are based on the principle of spatial topological connectivity. By constructing a local search neighborhood, the adjacent point clouds in physical space are aggregated into clusters. Its core mathematical model is as follows: R (neighborhood search radius): Defines the maximum spatial distance threshold for determining whether two points are connected. For any two points P i and P j , calculate their Euclidean distance Dist( P i , P j ).

[0050] Connectivity determination criterion: If Dist( P i , P j ) ≤R, then logically P i is classified into P j 's topological subset. In the luggage sorting scenario, the value of R is usually dynamically set according to the resolution of the lidar and the conveyor belt background noise level to ensure that the point cloud on the surface of the luggage box can be completely recognized as a whole.

[0051] This invention introduces a dynamic density threshold MinPoints (minimum number of connected points). The physical meaning of MinPoints: Defines the minimum point cloud scale that an effective luggage cluster must have.

[0052] Filtering logic: After the preliminary topological aggregation is completed, all generated candidate clusters will be traversed. If the number of points in a certain cluster < MinPoints, then it is determined that this cluster is discrete random noise generated by the sensor or a slight interference at the conveyor belt edge, and it is excluded. ]>

[0053] This method effectively solves the problem that traditional algorithms are sensitive to isolated noise points, ensuring that the point cloud data entering the second-level segmentation algorithm subsequent has a high confidence level.

[0054] (3) Execution process and jump decision The system utilizes a spatial index structure to accelerate neighborhood retrieval, quickly locating the point cloud cluster with the most points in the scene, which is then denoted as the segmentation result for that level. P sub .

[0055] Extract the point cloud cluster in Y maximum value in the axial direction max(P) sub_y Minimum value min(P) sub_y ), combined with a pre-established global longitudinal benchmark Y range Calculate the longitudinal coverage using the aforementioned formula:

[0056] Decision-making logic: Success path: If y ratio A value >0.6 indicates that the luggage structure is complete and clear, and the system directly terminates the subsequent algorithm and outputs the current segmentation result.

[0057] rollback path: If y ratio If the value is ≤0.6 (e.g., black material causes point cloud fragmentation, only segmenting part of the luggage), the system determines that the structure is "discontinuous" and transfers the point cloud to the second-level adaptive clustering algorithm for enhanced processing.

[0058] 2. Adaptive Clustering Optimization Based on Multi-Strategy Parameter Search When the first-level connected component algorithm is affected by factors such as luggage material (e.g., dark light-absorbing rubber, frosted surface causing extremely sparse point clouds), y ratio When verification fails, it automatically switches to the second-level execution unit with parameter self-healing capability.

[0059] (1) The mathematical construction system of the multi-strategy parameter matrix M pre-sets a dynamic parameter set. M={(Eps i , MinPts i )|i=1,2,…,n} The physical meaning of each parameter pair is as follows: Eps (Neighborhood Search Radius): Defines the maximum physical search circle radius when cluster core points are searching for "neighbors".

[0060] MinPts (density threshold): Defines the minimum number of sampling points that must be included within the Eps radius to determine whether the region is a valid structure.

[0061] (2) The grid search process based on "trial-verification-iteration" differs from the one-time calculation of traditional algorithms. This invention adopts an iterative search mechanism with a feedback chain: Parameter sequential extraction: The system enters a loop and extracts each group of parameters in the parameter matrix M in sequence. Eps i ,MinPts i ).

[0062] Clustering feature calculation: Using the current parameters, the DBSCAN operator is executed to extract the candidate cluster with the largest size and closest to the center of the conveyor belt from all generated clusters for real-time quality benchmarking. Immediately invoke on the candidate cluster y ratio calculate.

[0063] Intelligent early termination: Once a certain combination of parameters is calculated... y ratio If the value is greater than 0.5, the system immediately recognizes that the current parameters are perfectly suited to the luggage material and stops all subsequent unnecessary calculations.

[0064] This segmentation algorithm addresses the segmentation blind spots under heterogeneous materials, and the depth extension step demonstrates the system's robustness when facing heterogeneous targets. When the black luggage point cloud is broken, a larger Eps can act as a physical bridge, forcibly clustering the two broken parts. It also solves the over-segmentation problem: through multiple rounds of parameter trials, it can automatically find the optimal parameters that ensure integrity without introducing excessive background noise, thus avoiding the engineering challenge of luggage being mistakenly cut into fragments.

[0065] 3. Robust minimum guarantee algorithm based on statistical centroid distribution When the system fails to obtain a geometrically complete baggage cluster after traversing all parameter combinations (usually due to the baggage surface absorbing extreme light or being severely occluded, resulting in a fragmented point cloud distribution), a minimum segmentation strategy based on global statistical features is initiated.

[0066] (1) The dynamic extraction system of global centroid Cglobal breaks away from the local search logic and first performs centroid calculation on all questionable point clouds in the current region of interest to determine the geometric center reference of the target:

[0067] in, P i These are the coordinates of discrete points in the scene. This centroid represents the most likely distribution core of the heterogeneous targets in physical space and does not drift drastically due to the absence or fragmentation of the local point cloud.

[0068] (2) Statistical modeling based on radial Euclidean distance In order to forcibly separate the main body of the luggage from the messy background noise, the system calculates the statistical model of each sampling point. P i To the global center of mass C global radial distance D i Subsequently, a quantile filtering mechanism from statistics is introduced. This filters all distances... D i Sort by numerical value in ascending order and calculate the distance threshold at the 95th quantile. T limit :

[0069] The core of this segmentation algorithm lies in the fact that it does not rely on whether the point cloud is connected, but rather on the probability distribution to determine that the 95% of points closest to the center are extremely likely to belong to the main body of the luggage.

[0070] (3) The mandatory partitioning execution system based on envelope inclusiveness retains all those that satisfy the condition. D i T limit The point cloud is used as the final segmentation output.

[0071] It's important to understand that even if the baggage point cloud is fragmented and scattered, it can still be completely extracted as long as the whole remains within the centroid's radiation range. Simultaneously, this algorithm ensures that the sorting system can still provide location coordinates even under the most extreme conditions, achieving the engineering goal of "never running idle" in industrial applications.

[0072] like Figure 2 As shown, based on the same inventive concept, this invention also discloses a hybrid bounding box baggage detection system based on point cloud map planar projection, used to execute any of the above-mentioned hybrid bounding box baggage detection methods based on point cloud map planar projection, including: The data preprocessing module is used to acquire 3D point cloud data of the target scene; The benchmark establishment module is used to establish a dynamic quality benchmark based on the extreme value distribution of 3D point cloud data in a preset direction. The cascaded segmentation module includes at least two segmentation units with different segmentation characteristics, used to segment 3D point cloud data to obtain point cloud clusters; The quality evaluation module is used to perform real-time quality evaluation on the point cloud clusters obtained from the current segmentation based on the dynamic quality benchmark, and generate quality evaluation results. The scheduling and control module is used to determine whether to trigger the next level segmentation unit or output the point cloud cluster obtained from the current segmentation based on the quality evaluation results. The bounding box generation module is used to generate the minimum bounding box of the target luggage based on the final output point cloud cluster.

[0073] The following section provides a detailed description of a hybrid bounding box baggage detection method and system based on point cloud map planar projection, in conjunction with specific application scenarios.

[0074] Example 1: High-efficiency cascaded splitting under conventional standard operating conditions This embodiment takes a typical regular rectangular cardboard box as an example on the sorting line (size: 400mm). 300mm (200mm, material: light-colored corrugated paper), demonstrating the rapid processing logic of this invention under ideal working conditions.

[0075] Step S101: Point cloud data preprocessing and global environment modeling 1. Data Access: The computing device receives 3D point cloud data from the 3D sensor through the data acquisition module.

[0076] 2. Spatial cropping (ROI): Based on the robotic arm's operational envelope, the system performs spatial cropping on the point cloud, retaining valid data within the coordinate range of X∈[-500,500], Y∈[0,600], Z∈[0,500] (unit: mm).

[0077] 3. Confidence verification: The full point cloud size within the ROI after cropping is N=1200. Since N>Nmin(100), it is determined that there is an entity target in the current field of view rather than instantaneous noise, triggering the algorithm chain.

[0078] 4. Reference Extraction: The system automatically identifies the current sensing window length of the conveyor belt and determines the longitudinal physical reference reference for this frame. Y range =600mm.

[0079] Step S102: First-level cascade attempt – Fast connected component splitting The scheduling control module acts as the intelligent scheduling center, prioritizing the use of connected component segmentation units.

[0080] 1. Parametric clustering: using a set of preset fixed parameters ( Eps =0.15, MinPts =5) Perform preliminary clustering on the input point cloud.

[0081] 2. Feature Extraction: Filter the results to find the largest connected component P containing the most points. sub (Points Count(P) sub =1150). Quantitative index calculation: Traversing P sub Extract the maximum value of the coordinate point along the Y-axis.y max and minimum value y min Calculate the geometric integrity index of this component: y ratio It is approximately 0.642.

[0082] 3. Points Percentage Calculation: Calculate the points confidence level. P ratio =1150 / 1200 95.8%.

[0083] Step S103: Quality Decision and Operator Avoidance Logic 1. Criterion Determination: Obtained from the Quality Evaluation Module y ratio (0.642) and P ratio (95.8%). In comparison, due to... y ratio >0.6 and P ratio >30%, meeting the criteria for high-quality segmentation.

[0084] 2. Performance gain: Once the decision center confirms that the current result has geometric completeness, it immediately issues an instruction to terminate the subsequent second-level and third-level segmentation algorithms.

[0085] Actual test data: In this embodiment, the system completed the effective segmentation output in 22ms. Comparative experiments show that if full depth search clustering is forced for such simple targets (taking about 55ms), the present invention improves the processing speed by about 60% while ensuring the same accuracy.

[0086] Step S104: Generate and output the final detection result bounding box. 1. The bounding box generation module receives P sub Perform minimum bounding rectangle fitting.

[0087] 2. Results output: Calculate and output the center coordinates C(x,y,z), length, width, and rotation angle of the rectangle relative to the conveyor belt axis.

[0088] Feedback: Actual testing showed that the system's segmentation accuracy remained above 99.5% under these conditions, achieving optimal allocation of operator energy efficiency.

[0089] Example 2: Adaptive Search and Logic Reconstruction of Heterogeneous Material Wrapping This embodiment simulates a typical black matte suitcase on a sorting line. Due to the strong absorption of laser light by this material and the reflection deviation at the surface corners, the point cloud data exhibits severe "discrete fragmentation" characteristics in spatial distribution.

[0090] Step S201: Fast failure detection for the first-stage cascade attempt 1. First round of clustering: After point cloud data preprocessing, it enters the cascaded segmentation module. The scheduling and control module first calls the first-level connected component operator for preliminary segmentation.

[0091] 2. Quality monitoring and interception: Due to physical loss points, the luggage was divided into two fragmented clusters that were not connected to each other. P sub1 and P sub2 The decision center calculates the maximum cluster. P sub1 The geometric integrity index was measured. y ratio Only within the range [0.2, 0.3].

[0092] 3. Downgrade decision: because y ratio If the value is far below the preset threshold of 0.6, the system determines that the first-level attempt cannot obtain the complete target, and automatically triggers a process downgrade after approximately [8ms, 12ms].

[0093] Step S202: Second-level attempt – Improving DBSCAN's dynamic self-healing logic The decision module invokes the improved DBSCAN unit to initiate an adaptive parameter search for heterogeneous materials: 1. Online density sensitivity calculation: Construct a KDTree spatial index for the point cloud and calculate the average distance of the 5th nearest neighbor of the global point cloud. avg_dist In this example, due to the sparse point cloud, avg_dist It is dynamically locked within the range of [0.02m, 0.04m].

[0094] 2. Gradient parameter space construction: The system is based on avg_dist Four sets of parameter sets M, ranging from fine to coarse, are automatically derived.

[0095] 3. Iterative bridging attempts: Initial attempt: Using a smaller neighborhood radius Eps still failed to eliminate spatial holes. Gradient stepping: The system automatically steps to subsequent parameter sets (e.g., Eps in the range [0.06, 0.10]). Under this expanded radius, the originally physically discrete... P sub1 and P sub2Logical connections were established, enabling point cloud bridging across gaps.

[0096] 4. Quality Selection Judgment: Real-time evaluation of the bridged point cloud clusters. Measurement of the merged... y ratio Increase to the range of [0.7, 0.8], and the point ratio P ratio It remains stable between [60%, 80%].

[0097] Step S203: Performance Gain and Technical Effect Verification 1. Improved Recognition Rate: Through the self-repair mechanism of the second-level algorithm, the system successfully recovered the geometric features lost due to material light absorption. Experiments show that for this type of black, light-absorbing luggage, this invention significantly improves the segmentation success rate from approximately 55% using traditional methods to over 88%.

[0098] 2. Processing latency control: Although parameter search is triggered, thanks to the fast retry logic, the total processing time in this embodiment is controlled between [20ms, 30ms]. Comparative data: Compared to blindly performing full deep clustering (which takes approximately [40ms, 60ms]), this invention maintains an operator energy efficiency advantage of approximately 50% while solving the problem of complex material identification.

[0099] Step S204: Output the final result The bounding box generation module generates detection results containing precise center coordinates and rotation angles based on the reconstructed complete point cloud cluster, ensuring the accuracy of subsequent grasping actions.

[0100] Example 3: Statistical baseline handling in extreme occlusion scenarios. This example simulates an extreme case of severe occlusion on a sorting line. For example, a suitcase runs close to the edge of the conveyor belt, causing more than 50% of its outline to be obscured by the guardrail; or the surface of the suitcase is covered with special packaging with extremely high reflectivity / absorption (such as a fully transparent stretch film), causing the point cloud to appear as irregular, random fragments.

[0101] Step S301: Successive failure detection of the first two levels of intelligent strategies 1. First-level fast failure: Because the point cloud cannot form basic spatial connectivity, the connected component operator cannot extract any valid clusters.

[0102] 2. Second-level search exhaustion: The system enters the improved DBSCAN unit. Although it performs a full search of 4 sets of adaptive parameter sets M (taking approximately [25ms, 40ms]), the quality indicators of all candidate clusters remain low due to the low effective point cloud density and extremely scattered spatial distribution. y ratio <0.2, P ratio <15%.

[0103] 3. Final Degradation Trigger: The scheduling control module determines that the current scene does not have the geometric conditions to perform accurate clustering. In order to avoid the algorithm from falling into a meaningless infinite loop, it immediately forces a jump to the third-level spatial range segmentation unit.

[0104] Step S302: Robust core extraction based on statistical centroids 1. Global Centroid Calculation: This unit abandons the search for "local blocks" and instead calculates the geometric center coordinates of all point clouds within the ROI region. C global ( , , The physical basis for this is that although point clouds are sparse and fragmented, their centers of mass can still roughly correspond to the physical location of the target.

[0105] 2. Distance distribution statistics: Calculate the distance distribution of each point within the region. P i arrive C global Euclidean distance d i .

[0106] 3. Adaptive Threshold Filtering: The system sorts the distance array in descending order and uses the 95th percentile as the dynamic rejection threshold Dthreshold. Action: Keep d i T limit All points were statistically analyzed to forcibly remove the 5% of outliers that were furthest from the centroid and most discrete.

[0107] 4. Core region formation: The resulting point cloud set is the "core perception region" that envelops the target.

[0108] Step S303: Performance Assurance and Reliability Data Verification Experimental verification shows that even in extreme conditions where the clustering algorithm completely fails due to environmental factors, the backup mechanism of this invention ensures that the system still outputs positioning coordinates with a 100% response rate. In this embodiment, although the backup segmentation is slightly lower in geometric precision than the preceding clustering algorithm, its computational overhead is extremely low, ensuring that the total time of the entire process can still be maintained within the preset real-time sorting response cycle even under extremely harsh conditions. Actual testing proves that the center position coordinates provided by this mechanism are sufficient to drive the actuator to complete basic grasping or sorting actions, effectively solving the "perception blind spot" problem of the sorting system in complex environments and significantly improving the system's continuous operation capability.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0110] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hybrid bounding box baggage detection method based on point cloud map planar projection, characterized in that, include: S1. Acquire the three-dimensional point cloud data of the target scene, and establish a dynamic quality benchmark based on the extreme value distribution of the three-dimensional point cloud data in a preset direction. S2. The 3D point cloud data is segmented using a cascaded segmentation strategy, wherein the cascaded segmentation strategy includes at least two segmentation algorithms with different segmentation characteristics; S3. After each level of segmentation algorithm is executed, the current segmentation result is evaluated in real time based on the dynamic quality benchmark, and a quality evaluation result is generated. S4. When the quality evaluation result meets the preset conditions, output the current segmentation result and terminate the subsequent segmentation algorithm; When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is not the last level, the next level segmentation algorithm is triggered. When the quality evaluation result does not meet the preset conditions and the current segmentation algorithm is already the last level, the current segmentation result is forcibly output. S5. Generate the minimum bounding box of the target baggage based on the segmentation results.

2. The method according to claim 1, characterized in that, Step S1 further includes: Based on the maximum coordinate value of the three-dimensional point cloud data in the preset direction max(P y ) with minimum coordinate value min(P y ) Calculate the point cloud span in the preset direction. Y range As the dynamic quality benchmark: The preset direction is the direction in which the luggage travels on the conveyor belt.

3. The method according to claim 2, characterized in that, Step S3 further includes: Extract the local maximum coordinate values ​​of point cloud clusters obtained by any level segmentation algorithm in the preset direction. max (P sub_y ) with local minimum coordinates min(P sub_y ) Calculate longitudinal coverage y ratio As the quality evaluation result: 。 4. The method according to claim 3, characterized in that, Step S3 further includes: Calculate the number of point clouds in the current point cloud cluster. Count(P sub ) The total number of points in the three-dimensional point cloud data Count(P total ) The ratio of the points yields the percentage of points. P ratio As a result of the aforementioned quality evaluation: 。 5. The method according to claim 4, characterized in that, Step S4 specifically includes: When the longitudinal coverage y ratio The percentage of points is greater than the first preset threshold. P ratio When the value is greater than the second preset threshold, the point cloud cluster obtained by the current segmentation is determined to meet the quality requirements, the current point cloud cluster is output and the subsequent segmentation algorithm is terminated. Otherwise, if a next-level segmentation algorithm exists, it will be triggered; if no next-level segmentation algorithm exists, the current segmentation result will be forcibly output.

6. The method according to claim 1, characterized in that, The cascaded segmentation strategy includes: The first-level segmentation uses a connected component segmentation algorithm based on spatial connectivity to extract structurally continuous point cloud clusters; The second-level segmentation is triggered when the quality evaluation result of the point cloud clusters obtained from the first-level segmentation does not meet the preset conditions. The adaptive DBSCAN segmentation algorithm based on density clustering is used to cluster targets with sparse or broken point clouds. The third-level segmentation is triggered when the quality evaluation result of the point cloud cluster obtained from the second-level segmentation does not meet the preset conditions. A forced segmentation algorithm based on statistical features is used to extract the target core region.

7. The method according to claim 6, characterized in that, The first-level segmentation algorithm includes: Using a neighborhood search radius R and a minimum point count threshold MinPts, spatial topological clustering is performed on the 3D point cloud data based on Euclidean distance. Point clouds with a distance less than or equal to R are grouped into the same cluster, and clusters with a point count greater than MinPts are selected as candidate point cloud clusters. The point cloud cluster with the most points is then selected as the current segmentation result.

8. The method according to claim 6, characterized in that, The second-level segmentation algorithm includes: Calculate the average distance of the k-th nearest neighbor for each point in the 3D point cloud data. avg_dist , where k is a preset value; According to the average distance avg_dist Multiple sets of neighborhood search radii, ranging from small to large, are generated according to a preset scaling factor. Eps i And configure a corresponding density threshold for the search radius of each neighborhood group. MinPts i This constitutes the parameter search space. M={(Eps i , MinPts i )|i=1,2,…,n} ; A set of parameters is selected sequentially from the parameter search space, and DBSCAN clustering is performed to obtain candidate point cloud clusters; Extract the point cloud cluster with the most points and whose position conforms to the preset spatial constraints from the candidate point cloud clusters, and use it as the point cloud cluster obtained by the current segmentation. Call steps S3 and S4 to perform a quality evaluation on the current point cloud cluster. If the quality evaluation result meets the preset conditions, terminate the search and output the current point cloud cluster. If the quality evaluation result does not meet the preset conditions and there are untried parameter combinations in the parameter search space, then continue to select the next set of parameters to try; If all parameter combinations have been tried and the quality evaluation results do not meet the preset conditions, the third-level segmentation algorithm is triggered.

9. The method according to claim 6, characterized in that, The third-level segmentation algorithm includes: Calculate the geometric center of all points in the 3D point cloud data. C global ; Calculate the distance from each point in the 3D point cloud data to the geometric center. C global The Euclidean distance is used to obtain the distance set. {d i } ; The distance set {d i } Sort the data in ascending order and use the distance value at the preset quantile as the distance threshold. T limit ; Extraction distance less than or equal to the distance threshold T limit The point cloud is used as the point cloud cluster obtained by segmentation.

10. A hybrid bounding box baggage detection system based on point cloud map planar projection, used to execute the hybrid bounding box baggage detection method based on point cloud map planar projection as described in any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire 3D point cloud data of the target scene; The benchmark establishment module is used to establish a dynamic quality benchmark based on the extreme value distribution of the three-dimensional point cloud data in a preset direction. The cascaded segmentation module includes at least two segmentation units with different segmentation characteristics, used to segment the three-dimensional point cloud data to obtain point cloud clusters; The quality evaluation module is used to perform real-time quality evaluation on the point cloud clusters obtained by the current segmentation based on the dynamic quality benchmark after each segmentation unit is executed, and generate quality evaluation results. The scheduling and control module is used to make decisions based on the quality evaluation results: When the quality evaluation result meets the preset conditions, the current segmentation result is output and the subsequent segmentation unit is terminated; When the quality evaluation result does not meet the preset conditions and the current segmentation unit is not the last level, the next level segmentation unit is triggered; When the quality evaluation result does not meet the preset conditions and the current segmentation unit is already the last level, the current segmentation result is forcibly output. The bounding box generation module is used to generate the minimum bounding box of the target luggage based on the segmentation results of the final output.

Images