A non-priori detection method and device for a vegetation canopy shielding target based on a multi-spectral point cloud of a drone
By using an adaptive spherical shell model and multi-scale spatial-spectral feature extraction, the problem of detecting targets obscured by vegetation canopy in UAV multispectral point cloud data was solved, achieving efficient detection of targets obscured by vegetation canopy in UAV multispectral point cloud data and improving detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2025-05-26
- Publication Date
- 2026-06-30
AI Technical Summary
Under the conditions of UAV multispectral point cloud data, the occlusion of vegetation canopy reduces the probability of target exposure, making it difficult to obtain samples. Existing methods rely on a large number of samples for training, making it difficult to achieve effective detection.
An adaptive spherical shell model-based approach is adopted. By extracting multi-scale spatial-spectral features and constructing local background point sets, an adaptive spherical shell model is built. Using UAV multispectral point cloud data, non-prior detection is performed to achieve robust detection of targets obscured by canopies.
Without the need for target sample training, it can effectively detect targets partially occluded and exposed by vegetation canopy, improving the accuracy and robustness of detection.
Smart Images
Figure CN120472354B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of three-dimensional target detection technology in remote sensing, and relates to a method and device for detecting targets obscured by vegetation canopy. Background Technology
[0002] In recent years, multispectral point cloud data collected by drones has gradually become a hot application in agriculture and forestry due to its high-precision spatial information and rich spectral information, and has begun to be applied in the field of target detection. However, in target detection applications, the reduced probability of target exposure caused by vegetation canopy occlusion and the difficulty in obtaining samples due to missing spatial morphology limit the accuracy of sample-based deep learning methods that can be trained with a large number of samples.
[0003] Current research on the detection of targets obscured by vegetation canopy is limited. Most existing methods rely on training with large datasets, using the spatial-spectral information of existing samples to train detection models. This places high demands on both the quantity and quality of the samples. However, in applications using unmanned aerial vehicle (UAV) platforms in natural environments, canopy occlusion often leads to problems such as the lack of prior samples and incomplete spatial morphology of the samples, increasing the difficulty of detection. Therefore, it is essential to develop a non-prior detection method for targets obscured by vegetation canopy under the condition of UAV-borne multispectral point cloud data. Summary of the Invention
[0004] This invention addresses the problem that target detection methods based on multispectral point cloud data suffer from a lack of target samples due to vegetation canopy obstruction, making target detection difficult.
[0005] A non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds includes:
[0006] Step 1: Cluster the non-ground points in the UAV multispectral point cloud and count the set of radii R formed by the circumcircle radii of each cluster of non-ground points;
[0007] Step 2: Determine R1 = min(R). This involves sorting the c radius statistics results from smallest to largest, indexing the value at the σ% position, and rounding it up; for any point p in the multispectral point cloud data P... i Obtain spatial features with radius R1 and spatial features with radius R2; use the spatial features with radius R1 and spatial features with radius R2 as point p. i Multi-scale spatial features P spat (p i );
[0008] Get p i The corresponding spectral eigenvector P spec (p i), thus obtaining the spatial-spectral eigenvector P i =(P spec (p i ),P spat (p i ));
[0009] Step 3: Construct a radius-adaptive spherical shell model. The radius-adaptive spherical shell model is a concentric double-sphere model, based on the inner sphere radius R. inner =R2 constructs an adaptive outer sphere radius R outer The spherical shell model yields the local background point set S. i ={p j ∈P|R inner <||p j -p i ||2≤R outer (p i )};
[0010] Step 4, based on p i Local background point set S i The spatial-spectral eigenvectors of corresponding points determine p i Local background point set S i The spatial-spectral characteristic matrix is Seeking and The weight α that takes the minimum value is denoted as Calculate any point p in the multispectral point cloud P i Error E(P) i ), E(P i If the value is less than the threshold, it is a background pixel; otherwise, it is a target pixel.
[0011] Furthermore, the process of clustering non-ground points in the UAV multispectral point cloud and calculating the set R of radii formed by the circumcircles of each cluster of non-ground points includes:
[0012] For any UAV-borne multispectral point cloud data P, assuming there are l points in the multispectral point cloud data, first use cloth filtering to remove ground points, and denote the non-ground points as P. cover ; Use density clustering algorithm to analyze P cover Perform clustering, calculate the radius of the circumsphere of each cluster, and obtain the radius statistics of c clusters R = (r1, r2, ..., r...). c ).
[0013] Furthermore, σ% is taken as 75%.
[0014] Furthermore, the process of obtaining spatial features with radius R1 and spatial features with radius R2 includes:
[0015] For any point p in multispectral point cloud data P i Spatial features with radius R1 include flatness. Surface change rate linearity Verticality λ1, λ2, and λ3 represent the eigenvalues of the first three directional vectors obtained by performing principal component analysis on the coordinate vectors of all points within radius R1, arranged in descending order, where n is the value of point p. i Normal vector, k is the elevation vector;
[0016] The same method is used to obtain spatial features with radius R2.
[0017] Furthermore, point p i Multiscale spatial features
[0018] Furthermore, the p i The corresponding spectral eigenvector P spec (p i ) = (band1,...,band k ), band1, band2, ..., band k Record the spectral features of k bands in the spectral point cloud.
[0019] Furthermore, the adaptive outer sphere radius R outer satisfy denoted as the number of points on the inner sphere, and m as the dimension of the spatial-spectral eigenvector.
[0020] Furthermore, seeking to make and The process of obtaining the minimum value of weight α is as follows:
[0021] Determine the objective function β is the Lagrange multiplier, and I is the identity matrix; retain 95% of the characteristics, namely Where λ i yes The eigenvalues of the covariance matrix are then obtained.
[0022] Furthermore, any point p in the multispectral point cloud P i error
[0023] A non-prior detection device for vegetation canopy occlusion targets based on UAV multispectral point clouds, the device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the aforementioned non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds.
[0024] This invention proposes a non-prior stereo target detection method for UAV-borne multispectral point clouds based on an adaptive spherical shell model. It does not require target sample training and utilizes multi-scale spatial-spectral features to construct an adaptive spherical shell model to represent the local background points obtained from the detected points. This method can robustly detect targets obscured by canopies and effectively solve the problem of difficult detection of targets obscured by canopies. Attached Figure Description
[0025] Figure 1 A schematic diagram illustrating the implementation process of a non-prior detection method for targets obscured by vegetation canopy.
[0026] Figure 2 The data collection area for HIT Campus 2022 includes top-down views of two target exposure conditions and environmental maps and top-down views of targets obscured by vegetation canopy.
[0027] Figure 3 This image shows the data acquisition area of HIT Campus 2023, the target deployment, and the top-down view of the targets in the image data.
[0028] Figure 4 This is a schematic diagram of an adaptive spherical shell model.
[0029] Figure 5 The images show the detection results from the two sets of data. Detailed Implementation
[0030] To address the problems existing in the background technology, this invention proposes a target detection algorithm based on no prior knowledge. This algorithm achieves target detection in multispectral point clouds without sample training, and it can simultaneously and accurately detect both partially occluded targets under vegetation canopies and exposed targets, exhibiting strong robustness. The following detailed description, in conjunction with specific implementation methods, provides further insights.
[0031] Specific implementation method one: Combining Figure 1 This implementation method is described below.
[0032] This embodiment is a non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds, including:
[0033] Step 1: Cluster the non-ground points in the UAV multispectral point cloud and calculate the circumradius R of the non-ground point clusters:
[0034] For any UAV-borne multispectral point cloud data P, assuming there are l points in the multispectral point cloud data, first use Cloth Simulation Filtering (CSF) to remove ground points, and denote the non-ground points as P. cover ;
[0035] Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm was used to analyze P cover Perform clustering, calculate the radius of the circumsphere of each cluster, and obtain the radius statistics of c clusters R = (r1, r2, ..., r...). c ), where r1, r2, ..., r c These are all radius values.
[0036] Step 2: Extract multi-scale spatial features of the multispectral point cloud based on the clustering results, and construct the spatial-spectral feature matrix of the multispectral point cloud:
[0037] S201. Perform multi-scale spatial feature extraction:
[0038] In the spatial feature extraction process, to ensure the robustness of the algorithm, we use two spheres of different scales, R1 and R2, to extract the spatial features of the multispectral point cloud data P. To ensure that the spatial features of the smallest cluster can be correctly extracted without being submerged in the features of surrounding points, the minimum scale radius is defined as R1 = min(R). This is further validated using empirical values obtained from multiple datasets. At that time, the target and the background can be better separated. This involves sorting the c radius statistics results from smallest to largest, indexing the value at the 75th percentile, and rounding it up; here... It is actually the value at the 75th percentile of R.
[0039] Therefore, the above two scales are used to extract the spatial features of the UAV-borne multispectral point cloud data P. Among them, the spatial features that are more sensitive to occluded targets include flatness (P), surface change rate (SV), linearity (L), and verticality (V). For each test point p i The spatial features derived with radius R1 are represented as follows:
[0040]
[0041] Where λ1, λ2, and λ3 represent the eigenvalues of the first three direction vectors obtained by performing principal component analysis on the coordinate vectors of all points within radius R1, arranged in descending order. λ1(R1) = max{λ|Cv = λv}, λ2(R1) = max{λ|Cv = λv, v⊥v1}, λ3(R1) = max{λ|Cv = λv, v⊥v1, v⊥v2}, where C is the covariance matrix of the coordinates of all points within radius R1, and v is the eigenvector, v1, v2, ..., v i-1 Let n be the eigenvectors obtained from the 1st to the (i-1th)th eigenvector, and n be the eigenpoint of point p. i Normal vector, k is elevation vector.
[0042] For multispectral point cloud data P, the results of its multi-scale spatial feature extraction can be recorded as follows:
[0043]
[0044] S202. Obtain the spatial-spectral eigenvector P of the multispectral point cloud. i :
[0045] For any point p in the multispectral point cloud data P i The original data is as follows:
[0046] p i =(x i ,y i ,z i ,band1,band2,...,band k )
[0047] Where, x i ,y i ,z i For p i Spatial coordinates; band1, band2, ..., band k Record the spectral features of k bands in the spectral point cloud;
[0048] The spectral eigenvector is denoted as P. spec (p i ) = (band1,...,band k The spatial-spectral eigenvector of each point can be denoted as:
[0049] P i =(P spec (p i ),P spat (p i ))
[0050] Step 3: Construct a radius-adaptive spherical shell model and obtain any point p. iThe corresponding local background point set S in the spherical shell i :
[0051] The radius-adaptive spherical shell model is an optimization of the concentric double-sphere model, where the inner sphere radius R... inner and outer sphere radius R outer By utilizing the spherical shell region between the inner and outer spheres, adaptive local segmentation of the multispectral point cloud is performed. For any point p i It is necessary to obtain its local background point set S. i Avoid local background point set S i Addressing the issue of reduced accuracy due to target contamination, and improving algorithm robustness.
[0052] At the same time, in order to ensure the local background point set S i The spatial characteristics P of the detected point are not contaminated. spat We directly define R inner =R2, based on the inner sphere radius R inner Construct an adaptive outer sphere radius R outer The spherical shell model, R outer The following conditions must be met for the settings to be met:
[0053] To avoid local background point set S i The covariance matrix M composed of spatial-spectral features i The problem of false inversion caused by dissatisfaction with rank requires M i If the rank is full, then the spatial-spectral eigenvector dimension m and the number of local background points n = N. shell (p i It is necessary to satisfy n > m;
[0054] Assuming the point cloud is uniformly distributed, then the number of points is directly proportional to the volume, N. shell (p i ) and number of inner balls It should meet the following requirements:
[0055]
[0056] Summarized as follows:
[0057]
[0058] Then R outer The following conditions must be met:
[0059]
[0060] Summarized as follows:
[0061]
[0062] The set of local background points S obtained through a spherical shell model with an adaptive radiusi It can be expressed as:
[0063] S i ={p j ∈P|R inner <||p j -p i ||2≤R outer (p i )}
[0064] Step 4, using S i For p i Spatial-spectral features are jointly expressed, and the residuals between all expressed results and the original features are calculated as the detection result of the occluded target:
[0065] For any point p i Its spatial-spectral eigenvector is P i =(P spec (p i ),P spat (p i Based on p i Local background point set S i The spatial-spectral eigenvectors of corresponding points determine p i Local background point set S i The spatial-spectral characteristic matrix is Based on the assumption that the spatial-spectral characteristics of the target differ from those of the background, it can be deduced that when p i When P is the background point i It can be obtained from the spatial-spectral feature matrix of the surrounding background points Linear expression, while when p i When the target point is occluded, it cannot be detected because its spatial-spectral characteristics differ from the background. To accurately represent the local background, a spatial-spectral joint representation model is constructed.
[0066] n is S i The number of points in S, m is the number of points in S. i Given the number of feature dimensions, find the weight α such that... and All values are minimized, therefore the objective function is:
[0067]
[0068] Therefore, it can be simplified to:
[0069]
[0070] Where β is the Lagrange multiplier and I is the identity matrix, we minimize the above equation and set it to zero. Meanwhile, to further reduce the target's interference with the background, we retain... 95% of the characteristics, namely Where λ i yes The eigenvalues of the covariance matrix are obtained, thus deriving an estimate of α. This can be represented by the following expression:
[0071]
[0072] Calculate any point p in the multispectral point cloud P i Error E(P) i As the detection result, the linear representation in the point cloud P is evaluated by calculating the l2 norm. and P i Difference E(p) i ), can be expressed as:
[0073]
[0074] Occluded target detection is essentially a binary classification problem. The result is compared with a preset threshold τ. If the residual E(P) is positive, the result is positive. i If the value is less than the threshold, it means that the spatial-spectral features of the point are similar to the background and it is a background pixel; otherwise, it is a target point.
[0075] Example:
[0076] The experimental data used were multispectral point cloud data of woodlands at different densities in the Harbin Institute of Technology Science Park. The HIT Campus 2022 dataset was generated by fusing 3D reconstructed data from 10-band images captured by a drone equipped with a RedMX multispectral camera in 2022 with point cloud data collected by a Velodyne lidar. One example of a target occluding the understory was an L-shaped cardboard box, 0.6 meters high and 1 square meter in area. Another example was a horizontally placed blue, sloping, bare target, measuring 0.4 meters × 0.4 meters. The dataset contains 781,832 points, with land cover types including elm, maple, willow, bald cypress, shrubs, grassland, asphalt roads, and concrete roads. The HIT Campus 2023 dataset was generated by fusing 3D reconstructed data from 10-band images captured by a drone equipped with a RedMX multispectral camera in the Harbin Institute of Technology Science Park in 2023 with point cloud data collected by a Reigl-MiniLiDAR. The canopy shading target was five camouflage tents under a dense larch forest. The data contained 1,015,522 points, and the terrain features included larch, dawn redwood, grassland, roads, tents under the forest, and wooden walkways.
[0077] Figure 2The image shows the top view of the HIT Campus 2022 data acquisition area and two target exposure conditions, as well as the environmental map and top view of the target obscured by the vegetation canopy. (a) is a map of the HIT Campus 2022 data acquisition experimental site, (b) is a top view of the cardboard box target, (c) is a top view of the blue target, (d) is an environmental map of the canopy-obscured target, and (e) is a top view of the canopy-obscured target. It can be seen that the obscured target is completely obscured by the canopy in the UAV image. Figure 3 The data acquisition area of HIT Campus 2023, the target deployment, and the top view of the target in the image data are shown. (a) is the data acquisition experimental site of HIT Campus 2023, (b) is the environment around the target with canopy occlusion, and (c) is the top view of the target with canopy occlusion. It can be seen that the target is largely occluded by the canopy, and its spatial form is missing.
[0078] During the detection process according to the implementation method, the adaptive spherical shell model, such as Figure 4 As shown, R inner and R outer The spherical shell structure separates the detection point p. i Local background point S i .
[0079] Figure 5 The results of detection using two sets of data according to the implementation method are shown. (a), (b), and (c) correspond to the actual target distribution map, target ground truth map, and detection result map of the HITCampus 2022 data; (d), (e), and (f) correspond to the actual target distribution map, target ground truth map, and detection result map of the HITCampus 2023 data. Brighter points in the graphs represent points with greater spatial-spectral differences from the background, indicating a higher probability of a target. To properly display occluded points, vegetation canopy was removed from the results to show the response values of occluded targets. The experimental results demonstrate that the method of this invention can detect occluded targets with high accuracy even without prior target knowledge. Specific Implementation Method Two:
[0081] This embodiment is a non-prior detection device for vegetation canopy occlusion targets based on UAV multispectral point clouds. The device includes a processor and a memory. It should be understood that this includes any device described in this invention that includes a processor and a memory. The device may also include other units and modules that perform display, interaction, processing, control, and other functions through signals or instructions.
[0082] The memory stores at least one instruction, which is loaded and executed by the processor to implement the aforementioned non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds.
[0083] It should be understood that the instructions include computer program products, software, or computerized methods corresponding to any method described in this invention; the instructions can be used to program computer systems or other electronic devices. Computer storage media may include readable media on which instructions are stored, and may include, but are not limited to, magnetic storage media, optical storage media; magneto-optical storage media include read-only memory (ROM), random access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), and flash memory layers, or other types of media suitable for storing electronic instructions.
[0084] Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application, and can also be used with corresponding devices. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0088] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0089] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds, characterized in that, include: Step 1: Cluster the non-ground points in the UAV multispectral point cloud and count the set of radii formed by the circumcircle radii of each cluster of non-ground points. ; Step 2, Confirm , , Sort the statistical results of c radii from smallest to largest, and in the th... The location value is indexed and rounded up; this is for multispectral point cloud data. any point in , obtain Spatial features of radius and with Spatial characteristics of radius; will be based on Spatial characteristics of radius and Spatial features of radius as points Multiscale spatial features ; obtain Corresponding spectral eigenvectors The spatial-spectral eigenvectors are obtained. ; Get Spatial features of radius and with The process of defining the spatial characteristics of a radius includes: For multispectral point cloud data any point in ,by Spatial features of radius include flatness Surface change rate linearity Verticality ; , , They represent respectively to The eigenvalues of the first three directional vectors obtained by performing principal component analysis on the coordinate vectors of all points within the radius and arranging them from largest to smallest are: For point Normal vector Elevation vector; Obtain in the same way Spatial characteristics of radius; Step 3: Construct a radius-adaptive spherical shell model. The radius-adaptive spherical shell model is a concentric double-sphere model, based on the radius of the inner sphere. Constructing an adaptive outer sphere radius The spherical shell model, obtaining the set of local background points. ; Step 4, based on Local background point set Determination of spatial-spectral eigenvectors of corresponding points Local background point set The spatial-spectral characteristic matrix is Seeking to make and Weights that take the minimum value , recorded as ; Calculate multispectral point clouds any point in the middle error , Values smaller than the threshold are background pixels, while values larger than the threshold are target pixels.
2. The non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 1, characterized in that, Clustering of non-ground points in UAV multispectral point clouds, and statistically analyzing the set of radii formed by the circumcircles of each cluster of non-ground points. The process includes: For any UAV-borne multispectral point cloud data Assuming that multispectral point cloud data contains First, use cloth filtering to remove ground points, and denote the non-ground points as... Using density clustering algorithm to Perform clustering, calculate the radius of the circumsphere of each cluster, and obtain... Statistical results of the radius of each cluster .
3. The non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 1, characterized in that, Take 75%.
4. The non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 1, characterized in that, point Multiscale spatial features .
5. The non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 1, characterized in that, The Corresponding spectral eigenvectors , Recording in spectral point clouds Spectral characteristics of each band.
6. A non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to any one of claims 1 to 5, characterized in that, Adaptive outer sphere radius satisfy , The number of points on the inside sphere. is the dimension of the spatial-spectral eigenvector.
7. The non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 6, characterized in that, Seeking and Weights that take the minimum value The process is as follows: Determine the objective function ; For Lagrange multipliers, It is the identity matrix; retain 95% of the characteristics, namely ,in yes The eigenvalues of the covariance matrix are then obtained. .
8. A non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds according to claim 7, characterized in that, Multispectral point clouds any point in the middle error .
9. A non-prior detection device for vegetation canopy occlusion targets based on UAV multispectral point clouds, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the non-prior detection method for vegetation canopy occlusion targets based on UAV multispectral point clouds as described in any one of claims 1 to 8.