A method and system for detecting the number of people based on remote image analysis by a drone

Abstract

The application discloses a personnel quantity detection method and system based on unmanned aerial vehicle remote image analysis, relates to the field of unmanned aerial vehicle remote image processing technology, and solves the problems of high false positive and serious repeated counting of traditional visual detection under complex terrain, constructs a personnel quantity detection verification system of imaging geometry, terrain topology and observation statistics, and improves counting accuracy.

Description

Technical Field

[0001] This application relates to the field of UAV image processing technology, and in particular to a method and system for detecting the number of people based on remote image analysis from a UAV. Background Technology

[0002] Using drones for personnel detection and counting has significant application value in scenarios such as security for large-scale events, emergency rescue, and crowd control in scenic areas. However, existing technologies have the following main drawbacks:

[0003] Limitations of pure image domain processing: Existing methods (such as target detection based on the YOLO series) process entirely within the two-dimensional image plane, ignoring UAV pose and terrain information; at long altitudes (≥60m), human head pixels are extremely small (about 8-20 pixels), making them easily confused with natural clutter such as rocks and shrub canopies, leading to a sharp increase in false positive rate.

[0004] Superficial application of DEM: A few methods that introduce digital elevation models (DEM) are only used to query slope, which is a shallow application; existing technologies do not use DEM for ray-level forward intersection, nor do they use the statistical characteristics of depth residuals from multiple frames of observation (intersection convergence) as a basis for distinguishing true and false targets.

[0005] Cross-frame duplicate counting: Pixel-domain tracking methods (such as DeepSORT) suffer from systematic drift in pixel coordinates due to drone attitude jitter, making it impossible to distinguish between the same person and different people, resulting in duplicate counting.

[0006] In summary, existing methods for detecting the number of people based on UAV image processing lack a verification system that deeply integrates imaging geometry, terrain structure, and observation statistics, resulting in low reliability of counting results in uneven terrains such as mountains and rocky beaches. Summary of the Invention

[0007] This invention aims to overcome the shortcomings of the prior art by providing a method and system for detecting the number of people based on UAV remote image analysis. The process of counting the number of people is transformed into a process of observable verification of candidate targets based on UAV aerial image analysis. Through the dual physical barriers of ray penetration consistency test and cross-frame intersection convergence criterion, false positives that cannot be identified by pure appearance algorithms are systematically eliminated.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a method for detecting the number of people based on remote image analysis from unmanned aerial vehicles (UAVs) is provided, including: Step S10: Obtain the aerial image of the UAV at time t, the attitude angle of the onboard IMU, the altitude above the ground, and the camera intrinsic parameters. Based on the attitude angle and the altitude above the ground, and combined with the preset digital elevation model, establish a projection mapping from pixel coordinates to ground geographic coordinates, and calculate the position of the camera optical center and the spatial ray corresponding to the pixel. Step S20: Based on the interaction between the spatial ray and the digital elevation model, the local depression angle and dynamic ground sampling distance are calculated pixel by pixel in the aerial image, and the local depression angle and dynamic ground sampling distance are used as the geometric constraint reference for subsequent processing. Step S30: Using the local depression angle and dynamic ground sampling distance, calculate the expected head pixel radius corresponding to different positions in the aerial image. Using the expected head pixel radius as a scale constraint window, perform head candidate detection on the aerial image and output a candidate point set. ; Step S40, for For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. ; Step S50, will Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. ; Step S60, with The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

[0009] Further, in step S10, the step of establishing a projection mapping from aerial image pixels to ground geographic coordinates based on the attitude angle and ground clearance, combined with a preset digital elevation model, includes the following detailed steps: Step S11: Based on the airborne IMU attitude angle, construct a rotation matrix from the coordinate system of the UAV payload camera to the ground geographic coordinate system. Step S12: Simultaneously acquire the latitude, longitude, altitude, and elevation of the digital elevation model at the foot point of the UAV at the current time t, and determine the three-dimensional coordinates of the camera optical center in the ground geographic coordinate system. Step S13: For any image pixel in the aerial image, the pixel coordinates are converted into normalized camera coordinates using the camera intrinsic parameters, and then transformed into the spatial ray direction in the ground geographic coordinate system via the rotation matrix, thereby establishing a projection mapping relationship from aerial image pixels to ground geographic coordinates.

[0010] Furthermore, in step S20, the pixel-by-pixel calculation of the local depression angle and the dynamic ground sampling distance includes the following detailed steps: Step S21, mark the pixels of the aerial image as ,in This represents the horizontal pixel index of the image. The vertical pixel number of the image; query the surface normal vector of the ground point corresponding to that pixel in the digital elevation model; Step S22: Calculate the absolute value of the dot product between the direction of the spatial ray and the surface normal vector, and divide it by the product of the magnitudes of the two vectors to obtain the cosine of the angle between the two vectors; take the inverse cosine of the cosine value to obtain... The local tilt angle corresponding to a pixel; Step S23: Divide the product of the ground clearance and the pixel physical size by the product of the camera's effective focal length and the cosine of the local depression angle. The result is... The dynamic ground sampling distance corresponding to each pixel.

[0011] Furthermore, step S30 includes the following detailed steps: Step S31: Multiply the product of the preset equivalent physical radius of the human head and the effective focal length of the camera by the cosine value of the local depression angle obtained in step S20, and then divide by the height above the ground. The result is the expected pixel radius of the human head. Step S32: Using a convolutional neural network as a detection head, the aerial image is input into the detection head; Step S33: Construct a multi-scale detection anchor box using the expected head pixel radius to restrict the detection head to search for targets only within a scale range that matches the expected radius; Step S34: The detection head outputs all the center points of the detection boxes that meet the confidence threshold as a candidate point set.

[0012] In addition to the RGB three-channel aerial images, the input to the convolutional neural network also includes a local ground sampling distance prior map of the same size as a fourth channel, guiding the convolutional neural network to adaptively adjust its receptive field in different image regions; the training methods for this network include: A training sample set is constructed, with each training sample labeled with a ground truth bounding box containing a head and its corresponding geographic coordinates. During training, the digital elevation model is queried based on the geographic coordinates of the training samples to generate a corresponding local ground sampling distance prior map. The RGB aerial image is then compared with the local ground sampling distance prior map. Figure 1 The convolutional neural network is trained with input, and a combination of focus loss function and smoothing L1 loss is used as the optimization objective to enhance the sensitivity to detection of remote targets.

[0013] Furthermore, step S40 includes the following detailed steps: Step S41: Along the spatial ray originating from the camera's optical center, perform a step search in a direction away from the camera with a preset step size. In each step, query the terrain elevation of the digital elevation model under the current projected coordinates. Step S42: When the current elevation value of the space ray is less than or equal to the terrain elevation, it is determined that the first ground impact has occurred, and the depth at this time is recorded as the depth of the first visible surface intersection point. Step S43: Query the terrain slope at the first visible surface intersection point, and combine it with the dynamic ground sampling distance output in step S20 to perform a physical scale matching test on the candidate point; only when the ratio of the apparent size of the candidate target corresponding to the candidate point to the expected physical size calculated by the dynamic ground sampling distance falls within the preset range, is it allowed to proceed to step S44. Step S44: Calculate the difference between the candidate point inversion depth that has passed the physical scale matching test and the depth of the first visible surface intersection point, and multiply the difference by the spatial ray modulus of the corresponding pixel. The result is the ray penetration residual. Step S45: When the ray penetration residual is greater than the preset physical tolerance, the candidate point is determined to be a false target and is eliminated. The preset physical tolerance is determined as follows: the grid spacing of the digital elevation model or the elevation tolerance value obtained by calibration is taken as the terrain uncertainty base, the preset safety factor is multiplied by the ratio of the ground height to the effective focal length of the camera, and then multiplied by the terrain uncertainty base, and the result is taken as the physical tolerance.

[0014] Furthermore, step S50 includes the following detailed steps: Step S51: Establish a two-dimensional grid covering the task area, with the grid resolution adaptively adjusted according to the dynamic ground sampling distance; set the effective points... Each candidate point is located to a unique grid cell using its geographic coordinates; Step S52: Calculate the inversion depth of all candidate points falling into the same grid cell, and calculate the standard deviation of these inversion depths as the depth fluctuation entropy. Step S53: When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The depth uncertainty budget is determined by multiplying a preset coefficient by the ratio of the average height observed for the grid cell to the effective focal length of the camera, and then multiplying by the sub-pixel positioning error. The result is the depth uncertainty budget.

[0015] Further, in step S60, updating the occupancy probability of the two-dimensional Bayesian occupied grid includes: For valid candidate points verified through steps S40 and S50, the log probability of the grid cell to which the point belongs is accumulated, and the occupancy log probability of the grid cell is increased by an increment that is inversely proportional to the magnitude of the ray penetration residual; for grid cells that do not contain valid candidate points, their occupancy log probability is reduced by a preset attenuation coefficient to reflect the fact that no one is observed in the area. The method for determining the number of grid cells whose occupancy probability exceeds a preset threshold includes: converting the logarithmic probability of occupancy for each grid cell into an occupancy probability using a Sigmoid function; iterating through all grid cells, calculating the total number of cells whose occupancy probability is greater than the preset threshold, and using this total number as an estimate of the number of personnel.

[0016] Secondly, a personnel number detection system based on remote image analysis from unmanned aerial vehicles (UAVs) is provided, including: Airborne perception and geometric reference module: used to acquire aerial images of the UAV at time t, airborne IMU attitude angle, altitude above ground and camera intrinsic parameters. Based on the attitude angle and altitude above ground, combined with a preset digital elevation model, a projection mapping from pixel coordinates to ground geographic coordinates is established, and the position of the camera optical center and the spatial ray corresponding to the pixel are calculated. Parameter calculation module: Based on the interaction between spatial rays and digital elevation model, it calculates the local depression angle and dynamic ground sampling distance pixel by pixel in aerial image, and uses the local depression angle and dynamic ground sampling distance as the geometric constraint benchmark for subsequent processing; The head candidate detection module is used to calculate the expected head pixel radius corresponding to different positions in the aerial image by calling the local tilt angle and dynamic ground sampling distance. Using the expected head pixel radius as a scale constraint window, it performs head candidate detection on the aerial image and outputs a set of candidate points. ; X-ray penetration testing module: used for... For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. ; Intersection convergence determination module: used to determine the intersection convergence determination module. Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. ; Personnel Counting Decision Module: Used for... The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

[0017] The present invention discloses the following technical effects: This invention provides a method and system for detecting the number of people based on remote image analysis from unmanned aerial vehicles (UAVs). This overcomes the problem in existing technologies that rely solely on appearance images (such as texture and color) to distinguish people from rocks or bushes, leading to the failure of identifying small targets at long distances. This invention introduces a ray penetration consistency check (step S40), breaking the limitations of relying solely on vision for personnel number detection. By calculating the residual between the inversion depth of the detection candidate point and the initial impact of the spatial ray, it physically determines whether the target is located on the real ground surface visible to the camera. This mechanism can eliminate a large number of false targets that resemble people (such as rocks on the back of a ridge or artifacts suspended in the canopy), which is impossible with traditional image processing algorithms.

[0018] To address the pixel drift and duplicate counting issues caused by drone attitude jitter, this invention no longer relies on unstable pixel domain tracking. Instead, it constructs geographic grid cells and statistically analyzes the depth fluctuation entropy of the same geographic cell across multiple frames. Utilizing the physical principle that depth fluctuations of real people standing on the ground are constrained by terrain (convergence), while the depth fluctuations of false targets (swaying leaves, texture noise) diverge, this invention effectively removes geometrically unstable targets. Simultaneously, the depth uncertainty budget adaptively adjusts with flight altitude, solving the problem that a single threshold cannot simultaneously accommodate both long-range (>60m) and short-range (<30m) scenarios.

[0019] Existing counting techniques often produce black-box outputs, failing to explain why a person was identified. This invention assigns geometric reliability indicators to each person included in the statistics, including ray penetration residuals, depth fluctuation entropy, and terrain slope. This transforms the counting results from isolated numbers into traceable and verifiable physical evidence. In serious scenarios such as security and search and rescue, this geometric interpretability significantly enhances the system's practical value. Attached Figure Description

[0020] Figure 1This is a flowchart illustrating a method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) according to an embodiment of this application.

[0021] Figure 2 This is a schematic diagram of a personnel number detection system based on remote image analysis from an unmanned aerial vehicle (UAV) according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Example 1: This application provides a method for detecting the number of people based on remote image analysis from a drone, applying it to a typical mountainous, uneven terrain scenario, such as... Figure 1 As shown, the method includes: Step S10: Obtain the aerial image of the UAV at time t, the attitude angle of the onboard IMU, the altitude above the ground, and the camera intrinsic parameters. Based on the attitude angle and the altitude above the ground, and combined with the preset digital elevation model, establish a projection mapping from pixel coordinates to ground geographic coordinates, and calculate the position of the camera optical center and the spatial ray corresponding to the pixel.

[0024] In this embodiment, this step aims to establish a three-dimensional mapping relationship between the pixels of the UAV aerial image and the real world. The specific implementation process is as follows: Acquire aerial images taken by the drone at time t. The resolution is 1920×1080 (pixels); acquire the attitude angles output by the airborne IMU, including the roll angle. Pitch angle and yaw angle The ground altitude of the UAV at time t is obtained by combining radar or barometer with DEM footpoint elevation data. =80m, and load the preset task area digital elevation model (LiDAR DEM), grid spacing =1m.

[0025] Establish a camera coordinate system with the optical center of the camera on the UAV as the origin, and construct a rotation matrix from the camera coordinate system to the ground geographic coordinate system using attitude angles. :

[0026] in, and The respective roll angles of the drone The cosine and sine values, and These are the cosine and sine values ​​of the drone's pitch angle, respectively. and These are the cosine and sine values ​​of the yaw angle of the UAV, respectively.

[0027] By combining the UAV's GNSS coordinates with the elevation of the DEM at the foot point, the three-dimensional coordinates of the camera's optical center in the ground geographic coordinate system are determined. , The camera optical center at time t includes: The latitude and longitude obtained by UAV GNSS are directly converted into plane rectangular coordinates in the ground geographic coordinate system through UTM projection transformation. and In determining the vertical coordinates At that time, because GNSS directly measures the ellipsoidal height (WGS84), there is a deviation from the actual ground elevation, and the error is larger in complex terrain; therefore, the elevation of the DEM at the UAV's foothold (the ground point directly below) was queried. ,Will Ground clearance measured in real time by drone Add them together to obtain the precise altitude of the camera's optical center. This operation uses the DEM to anchor the camera optical center height to the real terrain surface, eliminating the systematic error of GNSS elevation. In addition, considering that the GNSS antenna and the camera lens are not at the same point, the installation offset of the two is compensated by the attitude angle of the airborne IMU, and finally the accurate three-dimensional coordinates of the camera optical center are output.

[0028] Aerial images taken by drones any pixel in Calculate its normalized camera coordinates:

[0029] in, This represents the two-dimensional pixel coordinates of each pixel in an aerial image. It is the pixel number in the horizontal direction. It is the pixel number in the vertical direction; For homogeneous pixel coordinates, Indicates transpose, adding a third dimension This is to facilitate matrix operations; The camera intrinsic parameter matrix describes the internal geometric properties of the camera, and is in the form of:

[0030] in, and These represent the focal lengths (in pixels, obtained from camera calibration) in the x and y directions, respectively. and The pixel coordinates representing the principal point (the image center of the aerial image) are given in this embodiment. =960, =540 (pixels); This represents the inverse of the camera intrinsic matrix, used to reverse-map pixel coordinates to the camera coordinate system. Normalized camera coordinates represent points on a plane at a unit distance in front of the camera, eliminating the influence of camera intrinsic parameters and reflecting only the orientation of aerial image pixels from the camera's perspective.

[0031] The normalized camera coordinates are transformed to a ground geographic coordinate system using a rotation matrix, thereby establishing a projection mapping relationship from aerial image pixels to ground geographic coordinates, and further obtaining the pixel points. Corresponding spatial ray direction vector : Finally, the parametric equations for space rays were established. :

[0032] in, The depth parameter is along the ray direction; this spatial ray is used in subsequent steps to intersect with the DEM.

[0033] Step S20: Based on the interaction between the spatial ray and the digital elevation model, the local depression angle and dynamic ground sampling distance are calculated pixel by pixel in the aerial image, and the local depression angle and dynamic ground sampling distance are used as the geometric constraint reference for subsequent processing.

[0034] In this embodiment, this step obtains the local terrain attributes of the ground point corresponding to the pixel through deterministic geometric calculation (rather than heuristic search), thereby calculating dynamic physical parameters. The specific implementation process is as follows: For pixels that need parameterization (At least covering the candidate region for subsequent head detection, either through sparse sampling or performed on each detection candidate), utilizing its spatial rays. Perform an intersection calculation with the DEM to determine the geographic coordinates of the ground point corresponding to the pixel. and the elevation at that point ,in These are eastward-facing plane coordinates, in meters (m). These are north-facing plane coordinates, in meters (m). The intersection calculation employs the Amanatides-Woo ray traversal algorithm, starting from the camera optical center and proceeding along... Direction, based on DEM grid spacing (1m) is the step size, and the system sequentially enters adjacent DEM grid cells; within each grid cell, the elevations of the four corners of the cell are calculated by bilinear interpolation, and the terrain plane within the cell is fitted. ,express The elevation of the terrain surface at a point; when the space ray is at depth altitude at satisfy: At that time, it was determined that the space ray struck the Earth's surface. and These represent the space rays at depths of [depth]. The east and north coordinates of the location.

[0035] For pixels already determined by the intersection of spatial rays and DEM Corresponding visible surface points Take four adjacent grid points on the DEM. Calculate the elevation difference:

[0036]

[0037] The normal vector is obtained. After unitization, it becomes Based on this, the number of pixels in the aerial image is calculated. Corresponding local depression angle :

[0038] in, It is an inverse cosine function; further calculation of dynamic ground sampling distance. :

[0039] in, This refers to the real-time altitude of the drone. For pixel physical size, This is the effective distance of the camera.

[0040] This step uses the DEM only to locate the regular grid cell to which it belongs based on the plane coordinates of the intersection point, and read (or obtain by bilinear interpolation) the elevation values ​​of the four corners of the cell to construct the normal vector; the whole process is deterministic arithmetic and table indexing operations, and does not involve image domain search windows or external visible field search tools.

[0041] Step S30: Using the local depression angle and dynamic ground sampling distance, calculate the expected head pixel radius corresponding to different positions in the aerial image. Using the expected head pixel radius as a scale constraint window, perform head candidate detection on the aerial image and output a candidate point set. .

[0042] In this embodiment, the local depression angle calculated in step S20 is called. With dynamic ground sampling distance The expected head pixel radius at different locations in an aerial image is calculated using the following formula. :

[0043] in, Represents the pixels of an aerial image The expected head pixel radius corresponding to the location. The equivalent physical radius of the human head is used, and a geometrically permissible scale range is determined based on this expected human head pixel radius. As a scale-constrained window, it performs head candidate detection, among which As a tolerance value, this embodiment sets it to 0.15; the head candidate detection adopts the detection head of the pre-trained target detection model, which is a lightweight convolutional neural network with a multi-scale feature pyramid and a fixed anchor frame size set.

[0044] This convolutional neural network uses aerial images acquired in real time. As input features, the RGB color channels of the aerial image constitute a three-channel tensor for the input features, and additionally, pixel-wise... Organize into a sheet with A prior image of the same size is concatenated as the fourth channel into the input feature tensor, allowing the network to learn during forward propagation to only consider small responses in areas with large GSD and large responses in areas with small GSD. The detection head is restricted to performing object detection on the feature layers corresponding to the scale constraint window; any candidate boxes whose size is outside this range are considered invalid and discarded. During object detection, center points greater than or equal to the confidence threshold are used as candidate points, their center pixels are extracted, and the final output is a set of candidate points. Record each candidate point The center pixel coordinates, detection confidence output by the detection head, dynamic ground sampling distance, and local depression angle.

[0045] Step S40, for For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. .

[0046] In this embodiment, for candidate points Retrieve the spatial ray equation established for this pixel in step S10:

[0047] in, Indicate candidate points The center pixel coordinates; candidate points are obtained by inversion through the projection mapping relationship established in step S10. Inversion depth That is, the distance from the bottom point of the detection frame to the optical center of the camera along the ray.

[0048] In step S20, during the intersection calculation based on the ray-regular grid traversal algorithm, when the geometric determination condition is met for the first time: At that time, the depth parameter at this point is recorded as the depth of the first visible surface intersection. The corresponding visible surface intersection is Further calculations were performed to determine the spatial deviation between the inversion depth and the initial impact depth. : Multiplying the spatial deviation by the spatial ray modulus of the corresponding pixel yields the ray penetration residual; due to It has been standardized, so Candidate points The residual of X-ray penetration.

[0049] Calculate the physical tolerance threshold based on the inherent accuracy and imaging geometry of the DEM. :

[0050] in, For safety factor; if ,determination False targets (e.g., the detection box falls on rocks on the back of a ridge, or floats above a tree canopy and is geometrically invisible) are discarded; if Determine if a point is geometrically valid; collect all candidate points that pass the geometric verification to form the valid point set after geometric verification. This point set has eliminated most terrain artifacts that resemble human figures.

[0051] Step S50, will Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. .

[0052] In this embodiment, this step verifies the set of valid geometric verification points output in step S40. Perform multi-frame statistical convergence testing to remove position jitter or artifact flicker, and output the final valid point set. The specific implementation process is as follows: A fixed two-dimensional geographic grid is established using the UAV's mission area reference origin, and a depth observation queue is maintained for each grid cell. (Empty queue); will Mapped to a 2D geographic grid and written to a depth observation queue, for Each valid point in (Including inversion depth and) Geographic coordinates Calculate the ID of the grid cell, and assign the corresponding ID to each valid point. Press in The grid cell with the current ID is marked as being hit in the current frame; for each grid cell hit in the current frame Calculated when its queue length is greater than or equal to 2 frames:

[0053] in, Indicates the mean depth. That is, a grid cell The corresponding depth fluctuation entropy, when actual people are present on the surface, is constrained by the terrain in terms of depth. The depth is relatively small; however, if artifacts and canopy jitter are present, the depth will fluctuate significantly. It gets bigger.

[0054] Calculate the depth uncertainty budget using the following formula. : ,in, The statistical coefficients are those used for calibration. For subpixel positioning error; when If the current grid cell is marked as unstable, all valid points (or only valid points from this observation) within that grid cell are considered false targets and removed. Otherwise, the grid cell is considered a stable observation, and its valid points are retained. Finally, all retained points are collected, and the final set of valid points after double verification is output. .

[0055] Step S60, with The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

[0056] In this embodiment, Each valid point geographic coordinates By rounding down and mapping to the same geographic grid cell as in step S50, for each valid point... The log-probability of the grid cell is accumulated, increasing its occupancy log-probability by an increment inversely proportional to the magnitude of the ray penetration residual. For grid cells that do not contain valid candidate points, their occupancy log-probability is reduced by a preset attenuation coefficient to reflect the fact that no one is observed in that area. The log-probability is then converted to occupancy probability using a sigmoid function. ,statistics Not less than the preset threshold The number of units is used as an estimate of the number of personnel. .

[0057] This step uses the final set of valid points output in step S50. Using this as input, a Bayesian occupation update is performed on a unified geographic grid, ultimately transforming the personnel count into a statistical problem of how many grid cells are stably occupied.

[0058] Example 2: To fully verify the technical effects of the present invention, this example provides the results of field tests conducted in three typical non-flat terrain scenarios: mountains, hills, and urban edges, including: The drone platform used in this embodiment is the DJI Matrice 300 RTK, and the drone's onboard camera is a Zenmuse H20 with an equivalent focal length of [missing information]. =35mm, pixel physical size =0.002mm; preset parameters include the equivalent physical radius of the human head. =0.15m, safety factor =1.5, statistical coefficient =2.0, subpixel positioning error =0.8.

[0059] Set relative flight altitude =80m, flight speed 5m / s, forward overlap 80%, lateral overlap 60%; a digital elevation model (DEM) generated using 1m resolution LiDAR point cloud data of the mission area, with an elevation mean square error of ±0.15m. Test scenarios include: Scene A (Mountainous Rocky Area): The elevation difference is about 80m. The ground is covered with rocks that are similar in size to a human body. The vegetation is sparse. 30 people wearing high-precision RTK positioning tags are deployed. Scene B (Gentle Slope with Shrubs): The slope is about 25°, and there are many swaying shrubs on the ground. 25 people are deployed there. Scene C (Ruins on the edge of the city): There are many obstructions and suspended structures, and 20 people are deployed.

[0060] By transmitting the geographic coordinates of personnel in real time using RTK tags and combining this with manual annotation, a tag dataset accurate to 0.1m is constructed for training the object detection model.

[0061] The comparison scheme settings include: B1 is the object detection model YOLOv8-n, which relies solely on image features and has no geometric constraints; B2 is the object detection model YOLOv8 combined with DEM slope filtering to remove detection boxes with a slope greater than 35°; B3 is the object detection model YOLOv8 combined with ray penetration test only to verify the effect of step S4 acting alone.

[0062] The evaluation metrics for the application include mean absolute error (MAE), root mean square error (RMSE), false detection rate (FDR), false negative rate (MR), and single-frame processing latency; the comparison results are shown in Table 1: Table 1. Comparison of overall performance of different technical solutions in three scenarios

[0063] The comparison results show that in scenario A, scheme B1 suffers from a false positive rate (FDR) of 34.6% due to false rock detection. Although scheme B2 introduces slope filtering, it cannot remove boulders on gentle slopes (the slope is legal but the geometric position is incorrect), resulting in an FDR of 18.2%. This invention, through the ray penetration test in step S40, successfully identifies these geometrically illegal targets, suppressing the FDR to below 4% and effectively inhibiting false positives. Although scheme B3 reduces the FDR, the same person is counted repeatedly due to drone attitude jitter, resulting in a high dynamic range error (MAE). This invention, through cross-frame statistical convergence in step S50, binds multiple frames of observations of the same geographic unit, eliminating duplicate counting and reducing MAE by 30% compared to B3, thus improving the accuracy of personnel counting. This invention adds ray intersection and grid statistical operations, increasing the single-frame latency by approximately 20ms compared to scheme B1, but still meeting the real-time processing requirements (>20fps).

[0064] Example 3: The personnel number detection system based on UAV remote image analysis provided in this embodiment of the invention can execute the personnel number detection method based on UAV remote image analysis provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method, such as... Figure 2 As shown, it includes: Airborne perception and geometric reference module: used to acquire aerial images of the UAV at time t, airborne IMU attitude angle, altitude above ground and camera intrinsic parameters. Based on the attitude angle and altitude above ground, combined with a preset digital elevation model, a projection mapping from pixel coordinates to ground geographic coordinates is established, and the position of the camera optical center and the spatial ray corresponding to the pixel are calculated. Parameter calculation module: Based on the interaction between spatial rays and digital elevation model, it calculates the local depression angle and dynamic ground sampling distance pixel by pixel in aerial image, and uses the local depression angle and dynamic ground sampling distance as the geometric constraint benchmark for subsequent processing; The head candidate detection module is used to calculate the expected head pixel radius corresponding to different positions in the aerial image by calling the local tilt angle and dynamic ground sampling distance. Using the expected head pixel radius as a scale constraint window, it performs head candidate detection on the aerial image and outputs a set of candidate points. ; X-ray penetration testing module: used for... For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. ; Intersection convergence determination module: used to determine the intersection convergence determination module. Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. ; Personnel Counting Decision Module: Used for... The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

[0065] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0066] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for detecting the number of people based on remote image analysis from unmanned aerial vehicles (UAVs), characterized in that, The method includes: Step S10: Obtain the aerial image of the UAV at time t, the attitude angle of the onboard IMU, the altitude above the ground, and the camera intrinsic parameters. Based on the attitude angle and the altitude above the ground, and combined with the preset digital elevation model, establish a projection mapping from pixel coordinates to ground geographic coordinates, and calculate the position of the camera optical center and the spatial ray corresponding to the pixel. Step S20: Based on the interaction between the spatial ray and the digital elevation model, the local depression angle and dynamic ground sampling distance are calculated pixel by pixel in the aerial image, and the local depression angle and dynamic ground sampling distance are used as the geometric constraint reference for subsequent processing. Step S30: Using the local depression angle and dynamic ground sampling distance, calculate the expected head pixel radius corresponding to different positions in the aerial image. Using the expected head pixel radius as a scale constraint window, perform head candidate detection on the aerial image and output a candidate point set. ; Step S40, for For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. ; Step S50, will Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. ; Step S60, with The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

2. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, In step S10, the step of establishing a projection mapping from pixel coordinates to ground geographic coordinates based on the attitude angle and ground clearance, combined with a preset digital elevation model, includes the following detailed steps: Step S11: Based on the airborne IMU attitude angle, construct a rotation matrix from the coordinate system of the UAV payload camera to the ground geographic coordinate system. Step S12: Simultaneously acquire the latitude, longitude, altitude, and elevation of the digital elevation model at the foot point of the UAV at the current time t, and determine the three-dimensional coordinates of the camera optical center in the ground geographic coordinate system. Step S13: For any image pixel in the aerial image, the pixel coordinates are converted into normalized camera coordinates using the camera intrinsic parameters, and then transformed into the spatial ray direction in the ground geographic coordinate system via the rotation matrix, thereby establishing a projection mapping relationship from aerial image pixels to ground geographic coordinates.

3. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, In step S20, the pixel-by-pixel calculation of the local depression angle and dynamic ground sampling distance includes the following detailed steps: Step S21, mark the pixels of the aerial image as ,in This represents the horizontal pixel index of the image. The vertical pixel number of the image; query the surface normal vector of the ground point corresponding to that pixel in the digital elevation model; Step S22: Calculate the absolute value of the dot product between the direction of the spatial ray and the surface normal vector, and divide it by the product of the magnitudes of the two vectors to obtain the cosine of the angle between the two vectors; take the inverse cosine of the cosine value to obtain... The local tilt angle corresponding to a pixel; Step S23: Divide the product of the ground clearance and the pixel physical size by the product of the camera's effective focal length and the cosine of the local depression angle. The result is... The dynamic ground sampling distance corresponding to each pixel.

4. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step S30 includes the following detailed steps: Step S31: Multiply the product of the preset equivalent physical radius of the human head and the effective focal length of the camera by the cosine value of the local depression angle obtained in step S20, and then divide by the height above the ground. The result is the expected pixel radius of the human head. Step S32: Using a convolutional neural network as a detection head, the aerial image is input into the detection head; Step S33: Construct a multi-scale detection anchor box using the expected head pixel radius to restrict the detection head to search for targets only within a scale range that matches the expected radius; Step S34: The detection head outputs all the center points of the detection boxes that meet the confidence threshold as a candidate point set.

5. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 4, characterized in that, In addition to the RGB three-channel aerial images, the input of the convolutional neural network also includes a local ground sampling distance prior map of the same size as a fourth channel, which guides the convolutional neural network to adaptively adjust the receptive field in different image regions.

6. A method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 4 or 5, characterized in that, The training methods for the convolutional neural network include: A training sample set is constructed, and each training sample is labeled with a ground truth bounding box of a head and its corresponding geographic coordinates. During the training process, the digital elevation model is queried based on the geographic coordinates of the training sample to generate a corresponding local ground sampling distance prior map. The RGB three-channel aerial images and the local ground sampling distance prior map are input together into the convolutional neural network for training, and a combination of focus loss function and smoothing L1 loss is used as the optimization objective to enhance the sensitivity of detection of remote targets.

7. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step S40 includes the following detailed steps: Step S41: Along the spatial ray originating from the camera's optical center, perform a step search in a direction away from the camera with a preset step size. In each step, query the terrain elevation of the digital elevation model under the current projected coordinates. Step S42: When the current elevation value of the space ray is less than or equal to the terrain elevation, it is determined that the first ground impact has occurred, and the depth at this time is recorded as the depth of the first visible surface intersection point. Step S43: Query the terrain slope at the first visible surface intersection point, and combine it with the dynamic ground sampling distance output in step S20 to perform a physical scale matching test on the candidate point; only when the ratio of the apparent size of the candidate target corresponding to the candidate point to the expected physical size calculated by the dynamic ground sampling distance falls within the preset range, is it allowed to proceed to step S44. Step S44: Calculate the difference between the candidate point inversion depth that has passed the physical scale matching test and the depth of the first visible surface intersection point, and multiply the difference by the spatial ray modulus of the corresponding pixel. The result is the ray penetration residual. Step S45: When the ray penetration residual is greater than the preset physical tolerance, the candidate point is determined to be a false target and is eliminated. The preset physical tolerance is determined as follows: the grid spacing of the digital elevation model or the elevation tolerance value obtained by calibration is taken as the terrain uncertainty base, the preset safety factor is multiplied by the ratio of the ground height to the effective focal length of the camera, and then multiplied by the terrain uncertainty base, and the result is taken as the physical tolerance.

8. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step S50 includes the following detailed steps: Step S51: Establish a two-dimensional grid covering the task area, with the grid resolution adaptively adjusted according to the dynamic ground sampling distance; set the effective points... Each candidate point is located to a unique grid cell using its geographic coordinates; Step S52: Calculate the inversion depth of all candidate points falling into the same grid cell, and calculate the standard deviation of these inversion depths as the depth fluctuation entropy. Step S53: When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The depth uncertainty budget is determined by multiplying a preset coefficient by the ratio of the average height observed for the grid cell to the effective focal length of the camera, and then multiplying by the sub-pixel positioning error. The result is the depth uncertainty budget.

9. The method for detecting the number of people based on remote image analysis from an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, In step S60, updating the occupancy probability of the two-dimensional Bayesian occupied grid includes: For valid candidate points verified through steps S40 and S50, the log probability of the grid cell to which the point belongs is accumulated, and the occupancy log probability of the grid cell is increased by an increment that is inversely proportional to the magnitude of the ray penetration residual; for grid cells that do not contain valid candidate points, their occupancy log probability is reduced by a preset attenuation coefficient to reflect the observation fact that no one exists in the area. The method for counting the number of grid cells whose occupancy probability exceeds a preset threshold includes: converting the log-occupancy probability of each grid cell into an occupancy probability using a Sigmoid function; traversing all grid cells, counting the total number of cells whose occupancy probability is greater than the preset threshold, and using this total number as an estimate of the number of people.

10. A personnel quantity detection system based on remote image analysis from unmanned aerial vehicles (UAVs), characterized in that, The system is used to implement the personnel number detection method based on UAV remote image analysis as described in any one of claims 1-9, the system comprising: Airborne perception and geometric reference module: used to acquire aerial images of the UAV at time t, airborne IMU attitude angle, altitude above ground and camera intrinsic parameters. Based on the attitude angle and altitude above ground, combined with a preset digital elevation model, a projection mapping from pixel coordinates to ground geographic coordinates is established, and the position of the camera optical center and the spatial ray corresponding to the pixel are calculated. Parameter calculation module: Based on the interaction between spatial rays and digital elevation model, it calculates the local depression angle and dynamic ground sampling distance pixel by pixel in aerial image, and uses the local depression angle and dynamic ground sampling distance as the geometric constraint benchmark for subsequent processing; The head candidate detection module is used to calculate the expected head pixel radius corresponding to different positions in the aerial image by calling the local tilt angle and dynamic ground sampling distance. Using the expected head pixel radius as a scale constraint window, it performs head candidate detection on the aerial image and outputs a set of candidate points. ; X-ray penetration testing module: used for testing... For candidate points in the model, the first visible surface intersection of the ray and the digital elevation model is obtained along the increasing parameter direction of the spatial ray. The depth of the surface intersection is compared with the inverted depth of the candidate point. When the deviation exceeds a preset physical tolerance, the candidate point is determined to be a false target and is removed. The set of valid points after geometric verification is output. ; Intersection convergence determination module: used to determine the intersection convergence determination module. Mapping to a geographic grid, the depth of multiple frames observed within the same grid cell is statistically analyzed, and the depth fluctuation entropy of that grid cell is calculated. When the depth fluctuation entropy exceeds the depth uncertainty budget determined by the current flight altitude, the grid cell is determined to be an unstable target and is removed. The final valid point set after double verification is output. ; Personnel Counting Decision Module: Used for... The occupancy probability of the updated two-dimensional Bayesian occupied grid is used as input. The number of grid cells whose occupancy probability exceeds a preset existence threshold is counted, and the estimated number of people is output.

Images