Non-prior detection method and device for vegetation canopy occlusion target based on unmanned aerial vehicle multi-spectral point cloud

Abstract

The invention discloses a non-prior detection method and device for a vegetation canopy shielding target based on unmanned aerial vehicle multi-spectral point cloud, and belongs to the technical field of three-dimensional target detection in remote sensing. In order to solve the problem that a target detection method based on multi-spectral point cloud data is difficult to detect due to lack of target samples caused by vegetation canopy shielding, the method comprises the following steps: firstly, clustering non-ground points of spectral point clouds, and counting the radius of a circumcircle of non-ground point clustering; extracting multi-scale spatial features of the multispectral point clouds according to a clustering result, and constructing spatial-spectral features of the multispectral point clouds; and then constructing a radius-adaptive spherical shell model, obtaining a local background point set in a spherical shell corresponding to any point, further performing joint expression on space-spectral features of pi, and calculating residual errors between all expression results and original features, thereby realizing detection of a shielded target.

Description

Technical Field

[0001] The present invention belongs to the technical field of stereo target detection in remote sensing, and relates to a method and device for detecting targets blocked by vegetation canopies. Background Art

[0002] In recent years, multispectral point cloud data collected by drones has become a hot topic in agriculture and forestry due to its high-precision spatial and spectral information, and has also begun to be used in target detection. However, in target detection applications, the reduced probability of target exposure due to vegetation canopy obstruction and the difficulty in obtaining samples due to the lack of spatial morphology have limited the accuracy of sample-based deep learning methods trained on large numbers of samples.

[0003] Currently, there is limited research on detecting targets obscured by vegetation canopies. The mainstream methods currently employed rely on large sample training runs, training detection models based on the spatial-spectral information of existing samples. This places high demands on both sample quantity and quality. In natural environment drone platform applications, canopy obstruction often leads to detection challenges such as the lack of prior samples for the target and the partial loss of spatial morphology, further complicating detection. Therefore, it is crucial to develop a non-a priori detection method for targets obscured by vegetation canopies using multispectral point cloud data from drones. Summary of the Invention

[0004] The present invention aims to solve the problem that target detection methods based on multispectral point cloud data have difficulty in detecting targets due to lack of target samples due to obstruction by vegetation canopy.

[0005] A non-a priori detection method for vegetation canopy occluded targets based on UAV multispectral point cloud, including:

[0006] Step 1: Cluster the non-ground points in the multispectral point cloud of the UAV, and count the radius set R consisting of the circumscribed circle radius of each cluster of non-ground points;

[0007] Step 2: Determine R1 = min(R). The statistical results of c radii are sorted from small to large, the value at the σ% position is indexed, and rounded up; for any point p in the multispectral point cloud data P i , obtain the spatial features with R1 as radius and the spatial features with R2 as radius; take the spatial features with R1 as radius and the spatial features with R2 as radius as point p i The multi-scale spatial features P spat (p i );

[0008] Get p i The corresponding spectral feature vector P spec (p i), and obtain the spatial-spectral feature vector P i =(P spec (p i ),P spat (p i ));

[0009] Step 3: Construct a radius-adaptive spherical shell model. The radius-adaptive spherical shell model is a concentric double-sphere model. According to the inner sphere radius R inner =R2 constructs the adaptive outer sphere radius R outer The spherical shell model, the local background point set S i ={p j ∈P|R inner <||p j -p i ||2≤R outer (p i )};

[0010] Step 4: Based on p i Local background point set S i The spatial-spectral eigenvector of the corresponding point determines p i Local background point set S i The spatial-spectral feature matrix is Seek to obtain and The weight α that takes the minimum value is recorded as Calculate any point p in the multispectral point cloud P i The error E(P i ), E(P i ) is smaller than the threshold value, it is a background pixel; otherwise, it is a target point.

[0011] Furthermore, the process of clustering the non-ground points in the multispectral point cloud of the UAV and counting the radius set R consisting of the circumscribed circle radius of each cluster of the non-ground points includes:

[0012] For any UAV-mounted multispectral point cloud data P, assuming there are l points in the multispectral point cloud data, first use cloth filtering to remove ground points, and record the non-ground points as P cover ; Using density clustering algorithm to cover Perform clustering and calculate the radius of the sphere outside each cluster to obtain the radius statistics of c clusters R = (r1, r2, ..., r c ).

[0013] Furthermore, σ% is set to 75%.

[0014] Furthermore, the process of obtaining the spatial features with a radius of R1 and the spatial features with a radius of R2 includes:

[0015] For any point p in the multispectral point cloud data P i , the spatial characteristics with R1 as radius include flatness Surface change rate Linearity Verticality λ1, λ2, and λ3 represent the eigenvalues of the first three direction vectors obtained by principal component analysis of the coordinate vectors of all points within the radius of R1, arranged from large to small. n is the eigenvalue of point p. i Normal vector, k is the elevation vector;

[0016] The same method is used to obtain the spatial features with R2 as the radius.

[0017] Further, click p i Multi-scale spatial features

[0018] Furthermore, the p i The corresponding spectral feature vector P spec (p i )=(band1,...,band k ), band1, band2, ..., band k Record the spectral features of k bands in the spectral point cloud.

[0019] Furthermore, the adaptive outer sphere radius R outer satisfy is the number of inner sphere points, and m is the dimension of the spatial-spectral feature vector.

[0020] Further, seek to make and The process of taking the minimum weight α is as follows:

[0021] Determine the objective function β is the Lagrange multiplier, I is the identity matrix; keep 95% of the features, i.e. where λ i yes The eigenvalues of the covariance matrix of

[0022] Furthermore, any point p in the multispectral point cloud P i Error

[0023] A non-a priori detection device for vegetation canopy obstruction targets based on drone multispectral point clouds, the device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the non-a priori detection method for vegetation canopy obstruction targets based on drone multispectral point clouds.

[0024] The present invention proposes a non-a priori stereo target detection method for UAV-mounted multispectral point clouds based on an adaptive spherical shell model. This method does not require target sample training and utilizes multi-scale spatial-spectral features to construct an adaptive spherical shell model to obtain local background points to express the detected points. It can robustly detect targets obscured by canopy, and can effectively solve the problem of difficult detection of targets obscured by canopy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the implementation process of the non-a priori detection method for vegetation canopy occluded targets.

[0026] Figure 2 The data collection area for HIT Campus 2022 is shown, along with two top-down views of the target under bare conditions and one under the vegetation canopy.

[0027] Figure 3 This shows the HIT Campus 2023 data collection area, target layout, and a bird's-eye view of the targets in the image data.

[0028] Figure 4 Schematic diagram of the adaptive spherical shell model.

[0029] Figure 5 The detection results of the two sets of data are shown in Figure 2. DETAILED DESCRIPTION

[0030] To address the problems in the background technology, the present invention proposes a target detection algorithm based on no prior knowledge. This algorithm can detect multispectral point cloud targets without prior knowledge, without the need for sample training. Furthermore, this method can achieve relatively accurate detection of both understory targets partially obscured by vegetation canopies and exposed targets, demonstrating strong robustness. This will be described in detail below in conjunction with specific implementations.

[0031] Specific implementation method 1: Combination Figure 1 To explain this embodiment,

[0032] This embodiment is a non-a priori detection method for vegetation canopy obstruction targets based on multispectral point clouds from drones, including:

[0033] Step 1: Cluster the non-ground points in the multispectral point cloud of the UAV and calculate the circumscribed circle radius R of the non-ground point clusters:

[0034] For any UAV-mounted multispectral point cloud data P, assuming there are l points in the multispectral point cloud data, first use cloth simulation filtering (CSF) to remove the ground points, and record the non-ground points as P cover ;

[0035] Using the density-based spatial clustering of applications with noise (DBSCAN) algorithm to cluster P cover Perform clustering and calculate the radius of the sphere outside each cluster to obtain the radius statistics of c clusters R = (r1, r2, ..., r c ), where r1, r2, ..., r c All are radius values.

[0036] Step 2: Extract the multi-scale spatial features of the multispectral point cloud based on the clustering results and construct the spatial-spectral feature matrix of the multispectral point cloud:

[0037] S201, perform multi-scale spatial feature extraction:

[0038] In the process of spatial feature extraction, in order to ensure the robustness of the algorithm, we use two spheres of different scales R1 and R2 to extract the spatial features of the multispectral point cloud data P. In order to ensure that the spatial features of the smallest cluster can be correctly extracted and not be submerged in the features of the surrounding points, the minimum scale radius is defined as R1 = min(R). At the same time, based on the empirical value obtained by verification of multiple data sets, when When , the target and background can be better separated. Sort the c radius statistics results from small to large, index the value at the 75% position, and round it up; here It is actually the value at the 75% position of R.

[0039] Therefore, the above two scales are used to extract the spatial features of the multispectral point cloud data P carried by the UAV. Among them, the spatial features that are more sensitive to the occluded targets include flatness (P), surface variation (SV), linearity (L), and verticality (V). For each test point p i The spatial feature expression obtained with radius R1 is as follows:

[0040]

[0041] Among them, λ1, λ2, and λ3 respectively represent the eigenvalues of the first three direction vectors obtained by principal component analysis of the coordinate vectors of all points within the radius of R1, arranged from large to small, λ1(R1)=max{λ|Cv=λv}, λ2(R1)=max{λ|Cv=λv,v⊥v1}, λ3(R1)=max{λ|Cv=λv,v⊥v1,v⊥v2}, where C is the covariance matrix of the coordinates of all points within the radius of R1, v is the eigenvector, v1,v2,...,v i-1 is the 1st to i-1th eigenvectors that have been obtained, and n is the point p i Normal vector, k is the elevation vector.

[0042] For multispectral point cloud data P, its multi-scale spatial feature extraction results can be recorded as:

[0043]

[0044] S202, obtaining the spatial-spectral feature vector P of the multispectral point cloud i :

[0045] For any point p in the multispectral point cloud data P i , the original data is:

[0046] p i =(x i ,y i ,z i ,band1,band2,...,band k )

[0047] Among them, x i ,y i ,z i For p i The spatial coordinates of band1, band2, ..., band k Record the spectral features of k bands in the spectral point cloud;

[0048] The spectral feature vector is denoted as P spec (p i )=(band1,...,band k ), the spatial-spectral feature vector of each point can be recorded as:

[0049] P i =(P spec (p i ),P spat (p i ))

[0050] Step 3: Construct a radius-adaptive spherical shell model and obtain the value of any point p. iCorresponding to the local background point set S in the spherical shell i :

[0051] The radius adaptive spherical shell model is an optimization of the concentric double sphere model. The inner sphere radius R inner and the outer sphere radius R outer , using the spherical shell area between the inner sphere and the outer sphere, the multispectral point cloud is adaptively segmented locally. For any point p i , it is necessary to obtain its local background point set S i , avoid the local background point set S i It solves the problem of reduced accuracy caused by target contamination and improves the robustness of the algorithm.

[0052] At the same time, in order to ensure that the local background point set S i Does not pollute the spatial features P of the detected points spat , we directly define R inner =R2, according to the inner sphere radius R inner , construct the adaptive outer sphere radius R outer Spherical shell model, R outer The setting needs to meet the following restrictions:

[0053] In order to avoid the local background point set S i The covariance matrix M of spatial-spectral features i The problem of pseudo-inverse caused by insufficient rank requires M i Full rank, then the dimension of the spatial-spectral feature vector m and the number of local background points n = N shell (p i ) needs to satisfy n>m;

[0054] Assuming that the point cloud is evenly distributed, the number of points is proportional to the volume, N shell (p i ) and inside ball points Should meet the following requirements:

[0055]

[0056] Arranged:

[0057]

[0058] Then R outer The following conditions are met:

[0059]

[0060] Arranged:

[0061]

[0062] The local background point set S is obtained by the spherical shell model with adaptive radiusi can be expressed as:

[0063] S i ={p j ∈P|R inner <||p j -p i ||2≤R outer (p i )}

[0064] Step 4: Use S i P i The spatial-spectral features are jointly expressed, and the residuals of all expression results and the original features are calculated as the detection results of the occluded target:

[0065] For any point p i , whose spatial-spectral eigenvector is P i =(P spec (p i ),P spat (p i )), based on p i Local background point set S i The spatial-spectral eigenvector of the corresponding point determines p i Local background point set S i The spatial-spectral feature matrix is Based on the assumption that the spatial-spectral characteristics of the target are different from those of the background, it can be inferred that p i When P is the background point, i The spatial-spectral feature matrix of the surrounding background points can be Linear expression, and when p i When the target point is blocked, it cannot be detected because its spatial-spectral characteristics are different from those of the background. Correctly express it and build a spatial-spectral joint expression model of the local background based on it.

[0066] n is S i The number of points in S, m is i The number of feature dimensions in the equation is used to find the weight α so that and All are minimized, so the objective function is:

[0067]

[0068] So it can be simplified to:

[0069]

[0070] Among them, β is the Lagrange multiplier, I is the unit matrix, which satisfies the minimum of the above formula, and is set to zero. At the same time, in order to further reduce the pollution of the target to the background, we retain 95% of the features, i.e. where λ i yes The eigenvalues of the covariance matrix of , so the estimated value of α is derived It can be expressed as follows:

[0071]

[0072] Calculate any point p in the multispectral point cloud P i The error E(P i ) as the detection result, and the linear expression result in the point cloud P is evaluated by calculating the l2 norm and P i The difference E(p i ), can be expressed as:

[0073]

[0074] The occluded target detection is essentially a two-classification problem. It is compared with the preset threshold τ. If the residual E(P i ) is less than the threshold value, it means that the spatial-spectral characteristics of the point are similar to the background and it is a background pixel; otherwise, it is a target point.

[0075] Example:

[0076] The experimental data used multispectral point cloud data from forests of varying densities at the Harbin Institute of Technology Science Park. The HIT Campus 2022 dataset was generated by fusing 3D reconstructions generated from 10-band imagery captured by a RedMX multispectral camera mounted on a drone in 2022 with point cloud data collected by a Velodyne LiDAR. The understory obstruction target consisted of an L-shaped cardboard box, 0.6 meters high and 1 square meter in area. Another obstruction target consisted of a horizontally placed blue sloped bare object, 0.4 meters by 0.4 meters in size. The dataset contains 781,832 points, representing features such as elm, maple, willow, bald cypress, shrubs, grassland, asphalt roads, and concrete roads. The HIT Campus 2023 dataset was generated by fusing 3D reconstructions generated from 10-band imagery captured by a RedMX multispectral camera mounted on a drone in 2023 at the Harbin Institute of Technology Science Park with point cloud data collected by a Reigl-MiniLiDAR. The canopy occlusion targets are five camouflaged tents under a dense larch forest. The data contains 1,015,522 points, and the ground features include larch, metasequoia, grassland, road, tents under the forest, and wooden sidewalks.

[0077] Figure 2The figure shows the HIT Campus 2022 data collection area, as well as top views of two target exposure conditions and environmental maps and top views of targets obscured by vegetation canopies. (a) is the area map of the HIT Campus 2022 data collection experimental site, (b) is a top view of the cardboard target, (c) is a top view of the blue target, (d) is the environmental map of the target obscured by the canopy, and (e) is a top view of the target obscured by the canopy. It can be seen that the obscured target is completely obscured by the canopy in the drone image. Figure 3 The HIT Campus 2023 data collection area, target layout, and a bird's-eye view of the target in the image data are shown. (a) shows the HIT Campus 2023 data collection experimental site, (b) shows the surrounding environment of the target with canopy occlusion, and (c) is a bird's-eye view of the target with canopy occlusion. It can be seen that the target is largely obscured by the canopy, and its spatial form is lost.

[0078] In the process of testing according to the embodiment, the adaptive spherical shell model is as follows Figure 4 As shown, R inner and R outer The spherical shell structure is separated to detect point p i Local background point S i .

[0079] Figure 5 The test results of two sets of data detected according to the implementation method are shown, where (a), (b), and (c) correspond to the actual target distribution map, target truth map, and test result map of the HITCampus 2022 data; (d), (e), and (f) correspond to the actual target distribution map, target truth map, and test result map of the HITCampus 2023 data. The brighter point in the figure represents that the greater the spatial-spectral difference between the point and the background, the greater the possibility of the target. In order to reasonably display the obscured point, the vegetation canopy is removed from the result map to display the response value of the obscured target. It can be seen from the experimental results that the method of the present invention can realize the detection of obscured targets without target prior conditions and obtain higher-precision detection results. Specific implementation method 2:

[0081] This embodiment is a non-a priori detection device for vegetation canopy obstruction targets based on a multispectral point cloud of a drone. The device includes a processor and a memory. It should be understood that the device includes any device including a processor and a memory described in the present invention. The device may also include other units and modules that perform display, interaction, processing, control, and other functions through signals or instructions.

[0082] At least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the non-a priori detection method for vegetation canopy-occluded targets based on drone multispectral point cloud.

[0083] It should be understood that the instructions include computer program products, software, or computerized methods corresponding to any method described in the present invention; the instructions can be used to program a computer system or other electronic device. Computer storage media may include readable media on which instructions are stored, and may include but are not limited to magnetic storage media, optical storage media; magneto-optical storage media include read-only memory ROM, random access memory RAM, erasable programmable memory (e.g., EPROM and EEPROM) and flash memory layers, or other types of media suitable for storing electronic instructions.

[0084] Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application may be implemented in various computer languages, for example, the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, systems, and computer program products according to the embodiments of the present application, and can also be used for corresponding devices. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0088] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0089] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A non-a priori detection method for vegetation canopy occlusion targets based on UAV multispectral point cloud, characterized by: include: Step 1: Cluster the non-ground points in the multispectral point cloud of the UAV, and count the radius set R consisting of the circumscribed circle radius of each cluster of non-ground points; Step 2: Determine R1 = min(R). The statistical results of c radii are sorted from small to large, the value at the σ% position is indexed, and rounded up; for any point p in the multispectral point cloud data P i , obtain the spatial features with R1 as radius and the spatial features with R2 as radius; take the spatial features with R1 as radius and the spatial features with R2 as radius as point p i The multi-scale spatial features P spat (p i ); Get p i The corresponding spectral feature vector P spec (p i ), and obtain the spatial-spectral feature vector P i =(P spec (p i ),P spat (p i )); Step 3: Construct a radius-adaptive spherical shell model. The radius-adaptive spherical shell model is a concentric double-sphere model. According to the inner sphere radius R inner =R2 constructs the adaptive outer sphere radius R outer The spherical shell model, the local background point set S i ={p j ∈P|R inner <||p j -p i ||2≤R outer (p i )}; Step 4: Based on p i Local background point set S i The spatial-spectral eigenvector of the corresponding point determines p i Local background point set S i The spatial-spectral feature matrix is Seek to obtain and The weight α that takes the minimum value is recorded as Calculate any point p in the multispectral point cloud P i The error E(P i ), E(P i ) is smaller than the threshold value, it is a background pixel; otherwise, it is a target point.

2. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 1 is characterized in that: The process of clustering non-ground points in the multispectral point cloud of UAVs and counting the radius set R consisting of the circumscribed circle radius of each cluster of non-ground points includes: For any UAV-mounted multispectral point cloud data P, assuming there are l points in the multispectral point cloud data, first use cloth filtering to remove ground points, and record the non-ground points as P cover ; Using density clustering algorithm to cover Perform clustering and calculate the radius of the sphere outside each cluster to obtain the radius statistics of c clusters R = (r1, r2, ..., r c ).

3. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 1 is characterized in that: σ% is taken as 75%.

4. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 1, characterized in that: The process of obtaining the spatial features with a radius of R1 and the spatial features with a radius of R2 includes: For any point p in the multispectral point cloud data P i , the spatial characteristics with R1 as radius include flatness Surface change rate Linearity Verticality λ1, λ2, and λ3 represent the eigenvalues of the first three direction vectors obtained by principal component analysis of the coordinate vectors of all points within the radius of R1, arranged from large to small. n is the eigenvalue of point p. i Normal vector, k is the elevation vector; The same method is used to obtain the spatial features with R2 as the radius.

5. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 4 is characterized in that: Click p i Multi-scale spatial features 6. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 1, characterized in that: The p i The corresponding spectral feature vector P spec (p i )=(band1,...,band k ), band1, band2, ..., band k Record the spectral features of k bands in the spectral point cloud.

7. The non-a priori detection method for vegetation canopy obstructed targets based on multispectral point cloud of unmanned aerial vehicle according to any one of claims 1 to 6, characterized in that: Adaptive outer sphere radius R outer satisfy is the number of inner sphere points, and m is the dimension of the spatial-spectral feature vector.

8. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 7, characterized in that: Seek to make and The process of taking the minimum weight α is as follows: Determine the objective function β is the Lagrange multiplier, I is the identity matrix; keep 95% of the features, i.e. where λ i yes The eigenvalues of the covariance matrix of 9. The non-a priori detection method for vegetation canopy obstruction targets based on multispectral point cloud of UAV according to claim 8, characterized in that: Any point p in the multispectral point cloud P i Error 10. A non-a priori detection device for vegetation canopy obstruction targets based on multispectral point cloud of drones, characterized by: The device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the non-a priori detection method for vegetation canopy-obstructed targets based on drone multispectral point cloud as described in any one of claims 1 to 9.

Images