A multispectral point cloud shadow-in-target detection method based on ambient light correction

Abstract

This invention discloses a target detection method in multispectral point cloud shadows based on ambient light correction, belonging to the field of shadow target detection in remote sensing. To address the problem of difficulty in detecting targets in multispectral point clouds due to shadows, this invention first determines the set P of shadow points in the multispectral point cloud. shadow Calculate the sky scene factor and ambient light factor for each point in the multispectral point cloud; combine the sky scene factor, ambient light factor, and the set of shadow points P shadow The reflectance information of the shadow points is recovered to obtain the reflectance-corrected multispectral point cloud P. c Then, the reflectance-corrected multispectral point cloud P is analyzed using the spatial spectral information of the neighborhood background points. c Joint expression is performed, and the residuals between all expression results and the original features are calculated to determine the target detection results under shadow.

Description

Technical Field

[0001] This invention belongs to the field of shadow target detection in remote sensing, and specifically relates to a method for detecting targets in shadows of multispectral point clouds carried by UAVs. Background Technology

[0002] In recent years, multispectral point cloud data collected by UAVs has gradually become a hot application in agriculture and forestry due to its high-precision spatial information and rich spectral information, and has begun to be applied in the field of target detection. However, due to the objective existence of shadow environments, the spectral information of ground objects in shadows degrades, resulting in the phenomenon of "different spectra for the same object," which reduces the difference between the target and the background and lowers the target signal-to-noise ratio, making the target difficult to detect. To solve the above problems, this invention proposes a target detection method in shadows of multispectral point clouds based on ambient light correction. This method does not require sample training and avoids the input of solar elevation angle and azimuth angle. Based on shadow point fitting optimization and the solution of ambient light and sky light, it achieves prior detection of shadow targets in multispectral point clouds, with high accuracy and strong robustness.

[0003] Current research on target detection in shadowed areas of multispectral point clouds is limited. Most existing methods rely on training with large numbers of samples, training detection models using the spatial-spectral information of existing samples. This places high demands on both the quantity and quality of the samples. However, in practical UAV platform applications, canopy occlusion and shadowed areas from artificial features limit the number of training samples and the prior knowledge of the target, leading to degradation and scarcity of spectral information, thus increasing the difficulty of detection. Secondly, most methods focus on optical imagery, using DSM images generated from LiDAR point clouds to detect shadowed areas and remove shadows for subsequent detection and recognition. However, these methods are based on spatially degraded two-dimensional images and do not fully consider the three-dimensional ambient light factors in shadowed areas, resulting in errors. Furthermore, due to the unstructured nature of point clouds, these methods cannot be directly applied to target detection in shadowed areas of multispectral point clouds. Therefore, it is essential to research a target detection method based on UAV-borne multispectral point cloud shadows, given the current conditions. Summary of the Invention

[0004] This invention aims to solve the problem of difficulty in detecting multispectral point cloud targets due to shadows in target detection.

[0005] A target detection method in multispectral point cloud shadows based on ambient light correction includes:

[0006] Determine the set P of shadow points in a multispectral point cloud shadow ; Calculate the sky scene factor F1 and ambient light factor F2 for each point in the multispectral point cloud: For any point p in the multispectral point cloud iWhen the direction (θ,ρ) is not obscured by other points, the visibility function V i (θ, ρ) takes a value of 1 otherwise, where θ is the zenith angle, the angle between the direction of the ray and the vertical direction, and ρ is the angle between the ray and the horizontal direction of true north; and is limited to point p j The ordinate z j Greater than point p i The ordinate z i At time, point p j As point p i The occlusion point is then used to obtain the sky angle Ω. sky Corresponding celestial factors When the point in direction (θ,ρ) The function IsSunlight(θ,ρ) evaluates to 1 if it is positive and 0 otherwise; based on p i The solid angle Ω occupied by neighboring features under sunlight env,sunlight =2π-Ω sky -Ω env,shadow For p i The ambient light is defined accordingly to obtain the ambient light factor. Where Ω env,shadow These are the shadow points of surrounding ground features;

[0007] Combining the skylight factor F1, the ambient light factor F2, and the set of shadow points P shadow The reflectance information of the shadow points is recovered to obtain the reflectance correction result P. c This leads to the corrected multispectral point cloud; for any point p i Acquiring spatial-spectral features P i p is obtained in the same way i Spatial-spectral information of neighboring points, and then the corresponding matrix is ​​obtained. Seeking to make and The weight γ that takes the minimum value is denoted as This leads to the linear expression results. Calculate point p i corresponding With P i The residuals E(P) between i When E(P) i When the value is greater than the residual threshold, it is determined as the target point.

[0008] Furthermore, the set P of shadow points in the multispectral point cloud was determined. shadow The process includes:

[0009] For multispectral point cloud data P, preliminary shadow points P are determined based on the Linear Shadow Ratio Index (LSRI). shadow_spec Suppose that the multispectral point cloud data has l points, denoted as {p}.i |i=1,2,3,…,l}, based on the solar altitude angle α and solar azimuth angle β corresponding to the data acquisition time and geographic coordinates, determine the unit vector corresponding to the direction of sunlight illuminating the ground. The solar altitude and azimuth were fitted using ray tracing, and the point cloud was then mapped according to the direction of illumination. Projecting, for each point with radius r, if in any point p in the direction i The visible area is smaller than πr 2 We assume that point P is in shadow, and thus obtain a set P of shadow points with different solar altitude angles and azimuth angles. solar ;

[0010] According to P shadow_spec and P solar The maximum value of the Intersection over Union (IoU) of the mid-shaded points yields the optimal solutions for the solar altitude angle α and solar azimuth angle β, and the projection result P under this condition. shadow_solar Based on P shadow_solar and P shadow_spec Determine the final set of shadow points P in the multispectral point cloud. shadow .

[0011] Furthermore, the initial shaded point P is determined based on the Linear Shading Ratio Index (LSRI). shadow_spec The process includes:

[0012] Based on the green band reflectance ρ in multispectral point clouds G and near-infrared band reflectivity ρ NIR , through (ρ G -ρ NIR ) / (ρ G +ρ NIR Calculate the linear shading ratio index (LSRI). When the LSRI is less than a threshold, it is identified as a shading point, and the corresponding preliminary shading point P is determined. shadow_spec If the value is not specified, it is determined as a light point, and the corresponding preliminary shadow point P is determined. shadow_spec value.

[0013] Furthermore, the unit vector corresponding to the direction of sunlight illuminating the ground.

[0014] Furthermore, based on P shadow_solar and P shadow_spec Determine the final set of shadow points P in the multispectral point cloud. shadow During the process, P is determined according to the following method. shadow The value is 0 when representing a shadow point and 1 when representing a light point;

[0015] When P shadow_spec Results and Pshadow_solar When the results are consistent, P shadow =P shadow_solar ;

[0016] When P shadow_spec The result is a shaded point and P shadow_solar When the result is the illuminated point, P shadow =P shadow_solar ;

[0017] When P shadow_spec The result is the illumination point and P. shadow_solar When the result is a shaded point, determine it as follows:

[0018] If this point is P shadow_spec If the result is surrounded by shaded points, then that point is corrected to be a shaded point, P. shadow =0; if the point is not controlled by P shadow_spec If the result is surrounded by shadowed points, then that point is determined to be a lit point, P. shadow =1.

[0019] Furthermore, by combining the skylight factor F1, the ambient light factor F2, and the set of shadow points P... shadow The reflectance information of the shadow points is recovered to obtain the reflectance correction result P. c The process includes:

[0020] Based on the ambient light-to-radiative transfer model, the total energy E received by the UAV's multispectral sensor is represented by the sky-view factor F1 and the ambient light factor F2. Then, the ground reflectivity ρ is obtained based on E. t (λ);

[0021] Ground reflectance ρ t (λ) is represented in the shaded area as:

[0022]

[0023] The reflectance of ground features in the sunlit area is expressed as:

[0024]

[0025] Among them, E s (λ) represents solar irradiance; σ represents the solar zenith angle; τ d (λ) represents the downlink energy decay rate, τ u (λ) represents the attenuation rate from the target to the sensor; ρ t (λ) represents the reflectance of ground features; E d1 (λ) is the energy of the descending light from the sky; E adj (λ) represents the energy reflected by neighboring ground features; λ is the wavelength.

[0026] Based on the ground reflectance correction result ρt (λ) yields the reflectance correction result P at the midpoint of the UAV-borne multispectral point cloud. c =(x,y,z,ρ) t (λ)), where (x,y,z) are the coordinates of the point.

[0027] Furthermore, the total energy received by the UAV's multispectral sensor is as follows:

[0028] E(λ)=k×E s (λ)×cosσ×τ d (λ)×ρ t (λ)×τ u (λ) / π

[0029] +F1×E d1 (λ)×ρ t (λ)×τ u (λ) / π

[0030] +F2×E adj (λ)×ρ b (λ)×τ u (λ) / π

[0031] +E u (λ)

[0032] Where k is a parameter representing whether direct solar energy can be received, when p i ∈P shadow If k=0, then k=1.

[0033] Furthermore, for any point p i Acquiring spatial-spectral features P i The process includes:

[0034] For any point p i Based on the set of points within its radius R2, determine p i Spatial features P spat (p i For any point p i Based on the multispectral point cloud P after reflectance correction c The spectral feature vector P is obtained by analyzing the spectral characteristics at different wavelengths. spec (p i ), and thus obtain point p i Spatial-spectral eigenvector P i =(P spec (p i ),P spat (p i )).

[0035] Furthermore, the p i Spatial features Pspat (p i )as follows:

[0036]

[0037] Where D represents p i Let C be the density of points within the sphere with radius R2, and let δ1, δ2, and δ3 represent the spatial coordinate vectors of all points within the sphere. The eigenvalues ​​of the first three principal component vectors are δ1(R2) = max{δ|Cv = δv}, δ2(R2) = max{δ|Cv = δv, v⊥v1}, and δ3(R2) = max{δ|Cv = δv, v⊥v1, v⊥v2}, where C is the covariance matrix of the coordinates of all points within radius R2, and v is the eigenvector, v1, v2, ..., v i-1 These are the eigenvectors from the 1st to the (i-1th)th obtained so far.

[0038] Furthermore, seeking to make and The process of obtaining the minimum value of weight γ includes:

[0039] For point p i Using the points in the concentric double-sphere model represented by radii R1 and R2 as the local background point set, and based on the objective function... Obtain the estimated value of γ Let I be the Lagrange multiplier and I be the identity matrix.

[0040] The present invention has the following advantages:

[0041] This invention can automatically detect shadow points in multispectral point clouds without input and improve shadow point retrieval accuracy by combining ray tracing methods. Secondly, it restores shadow information from the multispectral point cloud based on ambient light information. Finally, it performs prior-free target detection on the restored multispectral point cloud based on a spherical shell model, achieving target detection in shadows. Furthermore, this invention does not rely on sample training for multispectral point cloud shadow target detection and requires no prior target information, making it more universal and applicable to a wider range of needs compared to existing technologies. To verify the performance of the proposed algorithm, experiments were conducted using multispectral point cloud data from the New District Central Park in Jiangbei District of a certain city. The experimental results verified the accuracy and robustness of the UAV-based multispectral point cloud vegetation canopy shadow target detection method based on an adaptive spherical shell model. Attached Figure Description

[0042] Figure 1 This is a schematic diagram illustrating the implementation process of the present invention.

[0043] Figure 2 A schematic diagram of a spherical shell model selected for local background points.

[0044] Figure 3 This is a schematic diagram showing the acquisition range, target type, and multispectral point cloud data.

[0045] Figure 4 The results are compared with those of conventional band shadow detection algorithms, the optimized solution of shadow points in this invention, and the true values ​​of shadow points.

[0046] Figure 5 This is a comparison chart showing the true value of the target distribution, the detection results of the comparison method, and the detection results of the method of this invention. Detailed Implementation

[0047] This invention proposes a target detection method in multispectral point cloud shadows based on ambient light correction. It does not require target sample training, but adaptively optimizes shadow points based on band information, solves for ambient light factors and sky light factors, recovers the reflectivity of shadow points, and combines an adaptive spherical shell model to extract and utilize local spatial spectral features to detect targets in shadows, thus solving the problem of target detection in multispectral point cloud shadows.

[0048] Specific implementation method one: Combining Figure 1 This implementation method is described below.

[0049] This embodiment is a target detection method in multispectral point cloud shadows based on ambient light correction, including the following steps:

[0050] Step 1: Invert the solar elevation angle α and azimuth angle β for the set of shadow points P in the multispectral point cloud. shadow Optimization solution:

[0051] For any UAV-borne multispectral point cloud data P, shadow points can be solved using spectral features. However, the low reflectivity of dark objects leads to errors, thus requiring optimization. Based on the green band reflectivity ρ in the multispectral point cloud... G and near-infrared band reflectivity ρ NIR Through the formula (ρ) G -ρ NIR ) / (ρ G +ρ NIR Calculate the Linear Shadow Ratio Index (LSRI), set a threshold T, and define a point as a shadow point when LSRI < T. The initial shadow point P corresponding to this point is then determined. shadow_spec The value is 0; LSRI≥T is defined as the illuminated point, corresponding to P shadow_spec The value is 1, based on which the preliminary shadow point P in the multispectral point cloud is obtained. shadow_spec For subsequent optimization.

[0052] Assume that the multispectral point cloud data has l points, denoted as {p}. iGiven |i=1,2,3,…,l}, based on the solar altitude angle α and solar azimuth angle β corresponding to the data acquisition time and geographic coordinates, the direction of sunlight illuminating the ground can be represented by a unit vector as follows: The solar altitude and azimuth were fitted using ray tracing, and the point cloud was then mapped according to the direction of illumination. Perform projection, defining the radius of each point as r. If in any point p in the direction i The visible area is smaller than πr 2 If a point is in shadow, then it is defined as being in shadow, and thus a set P of shadow points with different solar altitude angles and azimuth angles is obtained. solar .

[0053] According to P shadow_spec and P solar The maximum value of the Intersection over Union (IoU) of the mid-shaded points yields the optimal solutions for the solar altitude angle α and solar azimuth angle β, and the projection result P under this condition. shadow_solar .

[0054] P is defined according to the following criteria. shadow_solar Optimize and solve to obtain P shadow Its shadow point is 0, and its light point is 1.

[0055] When P shadow_spec Results and P shadow_solar When the results are consistent, P shadow =P shadow_solar ;

[0056] When P shadow_spec The result is a shaded point and P shadow_solar When the result is a lit point, the ground feature is likely a low-reflectivity feature, not a shadowed point, therefore P shadow =P shadow_solar ;

[0057] When P shadow_spec The result is the illumination point and P. shadow_solar When the result is a shaded point, further judgment is required:

[0058] If this point is P shadow_spec If the result shows a point surrounded by shaded areas, then that point is a highlighted feature within the shade and should be corrected to a shaded point. (P) shadow =0; if the point is not controlled by P shadow_spec If the result is surrounded by shadow points, then that point is defined as a lit point, P. shadow =1.

[0059] Step 2: Calculate the sky scene factor F1 and ambient light factor F2 for each point in the multispectral point cloud:

[0060] Solving for the F1 factor of the celestial phenomenon:

[0061] The celestial factor is used to characterize any point p in a multispectral point cloud. i The amount of radiant energy received from the sky, at any point p. i Sky View Angle Ω sky Defined as the solid angle occupied by the unobstructed sky region visible above this point:

[0062] Introducing the visibility function V i (θ,ρ) is used to determine point p. j Is it point p? i The occlusion point is determined and the sky angle Ω is calculated accordingly. sky Where θ is the zenith angle, the angle between the direction of the ray and the vertical direction, and ρ is the angle between the ray and the horizontal direction of true north; when the direction (θ, ρ) is not blocked by other points, V i =1, otherwise V i =0, and is limited to point p j The ordinate z j Greater than point p i The ordinate z i At that time, i.e., z j >z i , point p j Only then can it be used as point p i The occlusion point can be used to obtain the sky angle Ω. sky The definition is as follows:

[0063]

[0064] The intensity of the received skylight is represented by the skylight factor F1, which is actually the skylight angle Ω. sky The ratio of the solid angle of the hemisphere to the solid angle of the hemisphere yields:

[0065]

[0066] Solution for ambient light factor F2:

[0067] Ambient light factor is used to characterize any point p in a multispectral point cloud. i Considering only the secondary reflection of sunlight, the amount of radiation energy reflected by surrounding objects should be represented by point p. i The energy received from sunlight reflected by surrounding objects can be obtained by integrating the solid angle occupied by the objects under sunlight with the reflectivity of the surrounding objects. Therefore, we use the ambient light factor F2 to characterize this energy.

[0068] Point p i The solid angle occupied by the ambient light above can be expressed as Ω. env =2π-Ω skyDefine a function IsSunlight(θ,ρ) to determine whether a point in the direction (θ,ρ) is illuminated by sunlight. Then IsSunlight(θ,ρ)=1, otherwise p j Let p be a point in the shaded area, and IsSunlight(θ,ρ)=0, where for any point p i Only secondary reflections are considered. Since skylight reflected from ground objects in shadows is a tertiary reflection, it is not included in the ambient light factor. i The solid angle occupied by neighboring features under sunlight is Ω. env,sunlight =2π-Ω sky -Ω env,shadow , where Ω env,shadow This represents the shaded area of ​​the surrounding ground features. Therefore, p i By limiting the ambient light, the ambient light factor F2 can be solved as follows.

[0069]

[0070] Step 3: Construct an ambient lighting model, combining the sky scene factor F1, the ambient light factor F2, and the set of shadow points P. shadow The reflectance information of the shadow points is recovered to obtain the reflectance-corrected multispectral point cloud P. c :

[0071] First, the radiative transfer model is optimized based on ambient light. For any wavelength λ, the total energy E received by the UAV multispectral sensor can be decomposed into the energy E reflected by ground objects in the first reflection. d Skylight E sky Secondary reflections from neighboring ground features E env and the radiation part E u energy:

[0072] E = E d +E sky +E env +E u

[0073] The sensor receives solar energy E directly reflected from ground features. d It can be represented as

[0074] E d (λ)=k×E s (λ)×cosσ×τ d (λ)×ρ t (λ)×τ u (λ) / π

[0075] The shaded area cannot receive direct sunlight. Whether or not direct sunlight can be received is characterized by k. When p i ∈Pshadow When the area is shaded, k = 0; otherwise, it is in the unshaded area, k = 1. E s (λ) represents solar irradiance; σ represents the solar zenith angle, which constrains energy attenuation; τ d (λ) represents the downlink energy decay rate, τ u (λ) represents the attenuation rate from the target to the sensor; ρ t (λ) represents the reflectivity of ground features.

[0076] Skylight E sky Part of the energy is the energy refracted by the atmosphere that reaches the target surface and is reflected back to the sensor, which can be expressed as:

[0077] E sky =F1×E d1 (λ)×ρ t (λ)×τ u (λ) / π

[0078] Among them, E d1 (λ) is the downward energy of skylight, and F1 is the skylight factor.

[0079] The secondary reflection energy received by the sensor from ground objects due to the neighborhood effect can be expressed as:

[0080] E env =F2×E adj (λ)×ρ b (λ)×τ u (λ)×π

[0081] Where E adj (λ) represents the energy reflected by neighboring ground features, and F2 is the ambient light factor, which is obtained by expanding the area within the hemisphere surrounding the target and calculating the solid angle of neighboring ground features under sunlight.

[0082] The total energy received by the sensor can be summarized as follows:

[0083] E(λ)=k×E s (λ)×cosσ×τ d (λ)×ρ t (λ)×τ u (λ) / π

[0084] +F1×E d1 (λ)×ρ t (λ)×τ u (λ) / π

[0085] +F2×E adj (λ)×ρ b (λ)×τ u (λ) / π

[0086] +E u (λ)

[0087] In the formula, τ d (λ), τ u (λ), E u τ can be obtained using readily available MODTRAN models, by inputting information such as the time and geographic coordinates at the time of data collection. d (λ), τ u (λ), E u .

[0088] Then the reflectivity ρ of the ground object t (λ) can be represented in the shaded region as:

[0089]

[0090] Similarly, the reflectance of ground features in sunlight can be expressed as:

[0091]

[0092] Therefore, based on the ground reflectance correction result ρ t (λ), we obtain the reflectance correction result P of the UAV-borne multispectral point cloud P. c , where P c The spatial coordinates and reflectance correction results ρ of each point t (λ) constitutes P c =(x,y,z,ρ) t (λ)).

[0093] Step 4: Construct a background dictionary using the spatial-spectral information of neighboring points, and apply it to the corrected multispectral point cloud P. c Anomaly detection without prior sample knowledge is performed, using the residual E between the dictionary representation and the detected point as the detection result for the target in the shadow:

[0094] (1) Construct a concentric double-sphere model by directly giving two different radii R1 and R2, and obtain any detection point p. i The set of local background points S i Used to build the background dictionary:

[0095] For any point p i Given radii R1 and R2, based on the isolation characteristics of the concentric double-sphere model, isolate p i Given the neighborhood of the sphere's center R1 with radius, obtain the p in the spherical shell. i The set of local background points S i ={p j ∈P|R1(p i )<||p j -pi ||2≤R2(p i )}, used for the center point p of the sphere i Linear expression;

[0096] Figure 2 This demonstrates a spherical shell model for selecting local background points. The spherical shell structure formed by R1 and R2 separates the detected point p. i Local background point set S i .

[0097] (2) Find point p i Spatial features P spat (p i );

[0098] For any point p i Based on the point set within a radius of R2, the spatial features P of the point cloud data are obtained. spat (p i This is used for shadow target detection. For a target, its sky angle information (F1), density entropy, planarity, surface features, linearity, and omnidirectional features exhibit significant differences from the local background. Therefore, by extracting these sensitive features, the target can be separated from the local background. For each test point p... i The spatial features derived using radius R² are represented as follows:

[0099]

[0100]

[0101] Where D represents p i Let C be the density of points within the sphere with radius R2, and let δ1, δ2, and δ3 represent the spatial coordinate vectors of all points within the sphere. The eigenvalues ​​of the first three principal component vectors are δ1(R2) = max{δ|Cv = δv}, δ2(R2) = max{δ|Cv = δv, v⊥v1}, and δ3(R2) = max{δ|Cv = δv, v⊥v1, v⊥v2}, where C is the covariance matrix of the coordinates of all points within radius R2, and v is the eigenvector, v1, v2, ..., v i-1 For the eigenvectors obtained from the 1st to the (i-1th)th eigenvectors, F1(p i ) is the p obtained in step 3 i The corresponding celestial angle size.

[0102] For multispectral point cloud data P, the results of its multi-scale spatial feature extraction can be recorded as follows:

[0103] P spat (p i )=

[0104] (F1(p i DensityEntropy(p i ),Plnarity(p i ),Surface(p i ),Linearity(p i ),Omnidirectional(p i ))

[0105] (3) Multispectral point cloud P c Spatial-spectral eigenvector P i :

[0106] For any point p i Multispectral point cloud data P c The corrected reflectance data recorded in the middle are as follows:

[0107] p i =(x i ,y i ,z i ,b1,b2,...,b k )

[0108] Where, x i ,y i ,z i For point p i Spatial coordinates, b1, b2, ..., b k For the multispectral point cloud P after reflectivity correction c The spectral characteristics at different wavelengths, the spectral eigenvector can be denoted as P spec (p i )=(b1,b2,...,b k Therefore, point p i The spatial-spectral eigenvector can be denoted as:

[0109] P i =(P spec (p i ),P spat (p i ))

[0110] (4) Using the background point set S i Background dictionary, linear expression p i Spatial-spectral characteristics, calculation of p i The local expression results and the l2 norm of the original features are used as the detection results for shadow targets:

[0111] For any point p i Its spatial-spectral eigenvector is P i =(P spec (p i ),P spat (p i Based on p i Local background point set S i The spatial-spectral eigenvectors of the corresponding points in the middle determine p i Local background point set S i The spatial-spectral characteristic matrix is Based on the assumption that the spatial-spectral characteristics of the target differ from those of the background, it can be deduced that when p i When P is the background point i It can be obtained from the spatial-spectral feature matrix of the surrounding background points Linear expression, while when p i When the target point is in shadow, it cannot be detected because its spatial-spectral characteristics differ from the background. To accurately represent the local background, a spatial-spectral joint representation model is constructed.

[0112] n is S i The number of points in S, m is the number of points in S. i Given the number of feature dimensions, calculate the weights γ such that... and All values ​​are minimized, therefore the objective function is:

[0113]

[0114] Therefore, it can be simplified to:

[0115]

[0116] in, Let I be the Lagrange multiplier and I be the identity matrix. If the above equation is minimized, then we set it to zero. Therefore, we can derive an estimate of γ. This can be represented by the following expression:

[0117]

[0118] Based on this, the linear expression results were obtained. Calculate any point p in the multispectral point cloud P i The residual E(P) i In this embodiment, the linear representation result in the point cloud P is evaluated by calculating the l2 norm. and P i Difference E(p) i ), can be expressed as:

[0119]

[0120] Object detection in shadows can be viewed as a binary classification problem. By using a preset threshold τ, the residual result E(P) is classified. i Binary classification is performed. Since targets in shadows cannot be represented by features of the surrounding background, therefore E(P) i When ) < τ, we define the spatial-spectral features of this point as similar to the background and can be expressed by the spatial-spectral features of the background points; E(P i When )≥τ, we define it as the target point.

[0121] Experiments were conducted using the above process. The experimental data were verified using multispectral point cloud data from the central park of the new district in Jiangbei District of a certain city. This data was generated in 2024 by fusing 3D reconstructed data from 10-band images captured by a drone equipped with a Rededge MX multispectral camera with point cloud data collected by a DJI L2 lidar. The occlusion targets under the trees were cubic targets, each 1 meter in length, width, and height. The dataset contained 781,832 points, and the ground cover types included elm trees, maple trees, willow trees, grassland, and dirt roads. Figure 3 The diagram shows the data acquisition range, target type, and multispectral point cloud. (a) is the data acquisition area, (b) is a schematic diagram of the shadowed target, and (c) is a schematic diagram of the multispectral point cloud. It can be seen that there are multiple targets distributed in the shadow.

[0122] Figure 4 The images show a comparison of the results of conventional band-based shadow detection algorithms, the optimized solution results of shadow points in this invention, and the true values ​​of shadow points. (a) shows the shadow point results from band calculations, (b) shows the optimized solution results from this invention, and (c) shows the true values ​​of shadow points. It can be seen that the method of this invention is superior. Figure 5 The diagram shows a comparison of the target distribution true value, the detection results of the comparison method, and the detection results of the method of the present invention, where (a) is the target true value distribution, (b) is the detection results of the comparison method, and (c) is the detection results of the method of the present invention. Figure 5 Brighter points indicate a greater spatial-spectral difference between that point and the background, suggesting a higher probability of it being a target. This demonstrates that the proposed method yields relatively accurate detection results. Experimental results show that the method of this invention can detect shadowed targets with high precision even without prior target knowledge.

[0123] The above examples of this invention are merely illustrative of the computational model and process of this invention, and are not intended to limit the implementation of this invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of this invention are still within the scope of protection of this invention.

Claims

1. A method for target detection in multispectral point cloud shadows based on ambient light correction, characterized in that, include: Determine the set P of shadow points in a multispectral point cloud shadow ; Calculate the sky scene factor F1 and ambient light factor F2 for each point in the multispectral point cloud: For any point p in the multispectral point cloud i When the direction (θ,ρ) is not obscured by other points, the visibility function V i (θ, ρ) takes a value of 1 otherwise, where θ is the zenith angle, the angle between the direction of the ray and the vertical direction, and ρ is the angle between the ray and the horizontal direction of true north; and is limited to point p j The ordinate z j Greater than point p i The ordinate z i At time, point p j As point p i The occlusion point is then used to obtain the sky angle Ω. sky Corresponding celestial factors When the point in direction (θ, ρ) The function IsSunlight(θ,ρ) evaluates to 1 if it is positive and 0 otherwise; based on p i The solid angle Ω occupied by neighboring features under sunlight env,sunlight =2π-Ω sky -Ω env,shadow For p i The ambient light is defined accordingly to obtain the ambient light factor. Where Ω env,shadow The shadow points of surrounding ground features; Combining the skylight factor F1, the ambient light factor F2, and the set of shadow points P shadow The reflectance information of the shadow points is recovered to obtain the reflectance correction result P. c This leads to the corrected multispectral point cloud; for any point p i Acquiring spatial-spectral features P i p is obtained in the same way i Spatial-spectral information of neighboring points, and then the corresponding matrix is ​​obtained. Seeking to make and The weight γ that takes the minimum value is denoted as This leads to the linear expression results. Calculate point p i corresponding With P i The residuals E(P) between i When E(P) i When the value is greater than the residual threshold, it is determined as the target point.

2. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 1, characterized in that, Determine the set P of shadow points in a multispectral point cloud shadow The process includes: For multispectral point cloud data P, preliminary shadow points P are determined based on the Linear Shadow Ratio Index (LSRI). shadow_spec Suppose that the multispectral point cloud data has l points, denoted as {p}. i |i=1,2,3,…,l}, based on the solar altitude angle α and solar azimuth angle β corresponding to the data acquisition time and geographic coordinates, determine the unit vector corresponding to the direction of sunlight illuminating the ground. The solar altitude and azimuth were fitted using ray tracing, and the point cloud was then mapped according to the direction of illumination. Projecting, for each point with radius r, if in any point p in the direction i The visible area is smaller than πr 2 We assume that point P is in shadow, and thus obtain a set P of shadow points with different solar altitude angles and azimuth angles. solar ; According to P shadow_spec and P solar The maximum value of the Intersection over Union (IoU) of the mid-shaded points yields the optimal solutions for the solar altitude angle α and solar azimuth angle β, and the projection result P under this condition. shadow_solar Based on P shadow_solar and P shadow_spec Determine the final set of shadow points P in the multispectral point cloud. shadow .

3. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 2, characterized in that, The initial shadow point P is determined based on the Linear Shadow Ratio Index (LSRI). shadow_spec The process includes: Based on the green light band reflectance ρ in multispectral point clouds G and near-infrared band reflectivity ρ NIR , through (ρ G -ρ NIR ) / (ρ G +ρ NIR Calculate the linear shading ratio index (LSRI). When the LSRI is less than a threshold, it is identified as a shading point, and the corresponding preliminary shading point P is determined. shadow_spec If the value is not specified, it is determined as a lit point, and the corresponding preliminary shadow point P is determined. shadow_spec value.

4. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 2, characterized in that, The unit vector corresponding to the direction of sunlight hitting the ground 5. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 2, characterized in that, Based on P shadow_solar and P shadow_spec Determine the final set of shadow points P in the multispectral point cloud. shadow During the process, P is determined according to the following method. shadow The value is 0 when representing a shadow point and 1 when representing a light point; When P shadow_spec Results and P shadow_solar When the results are consistent, P shadow =P shadow_solar ; When P shadow_spec The result is a shaded point and P shadow_solar When the result is the illuminated point, P shadow =P shadow_solar ; When P shadow_spec The result is the illumination point and P. shadow_solar When the result is a shaded point, determine it as follows: If this point is P shadow_spec If the result is surrounded by shaded points, then that point is corrected to be a shaded point, P. shadow =0; if the point is not controlled by P shadow_spec If the result is surrounded by shadowed points, then that point is determined to be a lit point, P. shadow =1.

6. A target detection method in multispectral point cloud shadows based on ambient light correction according to any one of claims 1 to 5, characterized in that, Combining the skylight factor F1, the ambient light factor F2, and the set of shadow points P shadow The reflectance information of the shadow points is recovered to obtain the reflectance correction result P. c The process includes: Based on the ambient light-to-radiative transfer model, the total energy E received by the UAV's multispectral sensor is represented by the sky-view factor F1 and the ambient light factor F2. Then, the ground reflectivity ρ is obtained based on E. t (λ); Ground reflectance ρ t (λ) is represented in the shaded area as: The reflectance of ground features in the sunlit area is expressed as: Among them, E s (λ) represents solar irradiance; σ represents the solar zenith angle; τ d (λ) represents the downlink energy decay rate, τ u (λ) represents the attenuation rate from the target to the sensor; ρ t (λ) represents the reflectance of ground features; E d1 (λ) is the energy of the descending light from the sky; E adj (λ) represents the energy reflected by neighboring ground features; λ is the wavelength. Based on the ground reflectance correction result ρ t (λ) yields the reflectance correction result P at the midpoint of the UAV-borne multispectral point cloud. c =(x,y,z,ρ) t (λ)), where (x,y,z) are the coordinates of the point.

7. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 6, characterized in that, The total energy received by the UAV's multispectral sensor is as follows: E(λ)=k×E s (λ)×cosσ×τ d (λ)×ρ t (l)×t u (l) / p +F1×E d1 (λ)×ρ t (l)×t u (l) / p +F2×E adj (λ)×ρ b (l)×t u (l) / p +E u (l) Where k is a parameter representing whether direct solar energy can be received, when p i ∈P shadow If k=0, then k=1.

8. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 6, characterized in that, For any point p i Acquiring spatial-spectral features P i The process includes: For any point p i Based on the set of points within its radius R2, determine p i Spatial features P spat (p i For any point p i Based on the multispectral point cloud P after reflectance correction c The spectral feature vector P is obtained by analyzing the spectral characteristics at different wavelengths. spec (p i ), and thus obtain point p i Spatial-spectral eigenvector P i =(P spec (p i ),P spat (p i )).

9. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 8, characterized in that, The p i Spatial features P spat (p i )as follows: Where D represents p i Let C be the density of points within the sphere with radius R2, and let δ1, δ2, and δ3 represent the spatial coordinate vectors of all points within the sphere. The eigenvalues ​​of the first three principal component vectors are δ1(R2) = max{δ|Cv = δv}, δ2(R2) = max{δ|Cv = δv, v⊥v1}, and δ3(R2) = max{δ|Cv = δv, v⊥v1, v⊥v2}, where C is the covariance matrix of the coordinates of all points within radius R2, and v is the eigenvector, v1, v2, ..., v i-1 These are the eigenvectors from the 1st to the (i-1th)th obtained so far.

10. The target detection method in multispectral point cloud shadows based on ambient light correction according to claim 8, characterized in that, Seeking to make and The process of obtaining the minimum value of weight γ includes: For point p i Using the points in the concentric double-sphere model represented by radii R1 and R2 as the local background point set, and based on the objective function... Obtain the estimated value of γ Let I be the Lagrange multiplier and I be the identity matrix.

Images