Unmanned aerial vehicle-based pile dynamic change monitoring method, system and equipment and medium

Abstract

The invention belongs to the field of unmanned aerial vehicle monitoring, and discloses a pile dynamic change monitoring method, system and device based on an unmanned aerial vehicle and a medium. The monitoring method comprises the following steps: acquiring multi-view images of a preset heap body area by using an unmanned aerial vehicle in different monitoring periods, and generating a dense point cloud model; then extracting point cloud features of a specific monitoring period and a previous period, and realizing consistent alignment of the point cloud models of the two periods under a unified coordinate system; and finally, according to the aligned model, determining an increase and decrease change area of the pile body area, and identifying change information. According to the scheme, the data quality is ensured by dynamically adjusting the aerial photography parameters, the registration precision is improved by adopting deep learning and geometric constraint, the accurate identification and grading of the change area are realized by combining point cloud difference and semantic segmentation, the monitoring capability and response speed of the dynamic change of the heap body are remarkably enhanced, and the real-time performance of the heap body is improved. And reliable support is provided for inventory management and safety early warning.

Description

Technical Field

[0001] This application belongs to the field of unmanned aerial vehicle (UAV) monitoring, and specifically relates to a method, system, equipment and medium for monitoring dynamic changes in a stack based on UAVs. Background Technology

[0002] The illegal dumping of industrial solid waste, urban domestic waste, construction waste, and agricultural waste may cause serious soil pollution through rainwater leaching, dust diffusion, surface erosion, and natural volatilization. In order to strengthen the supervision of these potential pollutants, it is necessary to take timely sampling, analysis, and evidence collection measures. However, traditional manual measurement or fixed equipment collection methods have shortcomings such as long cycle, small coverage, and high labor costs, making it difficult to achieve timely, accurate, and effective inspection and sampling. Summary of the Invention

[0003] To address the aforementioned issues, this application provides a method, system, equipment, and medium for monitoring dynamic changes in soil mounds based on unmanned aerial vehicles (UAVs). This solution utilizes UAVs to inspect newly added or altered soil mounds in mines, farmland, and industrial land. Once an abnormal change in a soil mound is detected, soil samples are taken at the corresponding location and analyzed to determine the presence of pollutants.

[0004] It significantly enhances the monitoring capability and response speed of dynamic changes in the reactor body, providing reliable support for timely and accurate sampling of pollutants.

[0005] To address the aforementioned technical problems, the first aspect of this disclosure proposes a method for monitoring dynamic changes in a stack based on unmanned aerial vehicles (UAVs), the monitoring method comprising: Multi-view images of a preset pile area are acquired by drones during different monitoring periods, and dense point cloud models for each monitoring period are generated based on the multi-view images. Extract point cloud features from the first dense point cloud model of a specific monitoring period and the second dense point cloud model of the previous monitoring period of the specific monitoring period, and align the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features. Based on the aligned first dense point cloud model and second dense point cloud model, the increased or decreased change regions in the preset stack area are determined, and the change information of each increased or decreased change region is identified.

[0006] According to a preferred embodiment of this disclosure, the acquisition of multi-view images of a preset pile area via a drone at different monitoring cycles includes: Based on the terrain boundary of the preset pile area, a drone shooting trajectory is set, and the drone is controlled to capture real-time images according to the drone shooting trajectory; the drone shooting trajectory includes: flight path range, altitude level, and shooting angle; When the drone is taking pictures, the drone's shooting trajectory is adjusted in real time based on the clarity of the real-time image and the slope angle of local areas in the real-time image. The multi-view images are generated based on the real-time images captured.

[0007] According to a preferred embodiment of this disclosure, when the drone is taking pictures, adjusting the drone's shooting trajectory in real time based on the sharpness of the real-time image and the slope angle of a local area in the real-time image includes: The system identifies the image clarity of the real-time image and, when the image clarity is insufficient, adjusts the waypoint spacing in the flight path range, the flight altitude in the altitude level, and / or the direction of the shooting angle to control the flight of the UAV. The average slope angle of the local area is calculated based on the real-time image. When the average slope angle is higher than a preset slope threshold, the direction of the waypoint density and / or shooting angle in the flight path range of the local area is adjusted, and the shooting resolution of the UAV in the local area is improved.

[0008] According to a preferred embodiment of this disclosure, the step of generating dense point cloud models for each monitoring period based on the multi-view images includes: Based on the structure-of-motion method, the camera pose and initial point cloud image of each image in the multi-view imagery are extracted; Based on multi-view stereo matching technology, a multi-view cost volume is constructed using the camera pose of each image and the initial point cloud image, and a pixel depth map is calculated. The dense point cloud model is obtained by fusing the pixel depth map and the multi-view cost volume.

[0009] According to a preferred embodiment of this disclosure, the step of extracting point cloud features from a first dense point cloud model for a specific monitoring period and a second dense point cloud model from the previous monitoring period, and aligning the first and second dense point cloud models in a unified coordinate system based on the point cloud features, includes: Extract the local context features of each point in the first dense point cloud model and the second dense point cloud model as the point cloud features; Based on the local context features, a first point is determined in the first dense point cloud model that matches a second point in the second dense point cloud model; the first point and the second point are taken as a matching point pair. Based on a random algorithm, an initial transformation matrix is ​​determined that minimizes the distance between the first point in the first dense point cloud model and the second point in the second dense point cloud model after transformation. Establish the correspondence between specific points in the first dense point cloud model and the second dense point cloud model; The initial transformation matrix is ​​updated iteratively, and the error of a specific point in the correspondence of a specific point is calculated until the error converges or the final transformation matrix is ​​obtained. The first dense point cloud model is transformed into the coordinate system of the second dense point cloud model using the final transformation matrix, thus achieving consistent alignment of the first and second dense point cloud models in a unified coordinate system.

[0010] According to a preferred embodiment of this disclosure, the step of determining the increased or decreased change regions in the preset stack region based on the aligned first dense point cloud model and second dense point cloud model, and identifying the change information of each of the increased or decreased change regions, includes: Perform a difference operation on the aligned first dense point cloud model and the second dense point cloud model to determine the increase or decrease change area in the preset stacked area; The multi-view images and point cloud fragments corresponding to the increased or decreased regions are input into a pre-trained transformation category discrimination model, which outputs the change information of the increased or decreased regions.

[0011] According to a preferred embodiment of this disclosure, the monitoring method further includes: Generate the geometric outline and center position of the region of increase or decrease; The geometric contour, center position, and change information are combined to generate a stack change report.

[0012] To address the aforementioned technical problems, a second aspect of this disclosure proposes a dynamic change monitoring system for a reactor structure based on unmanned aerial vehicles (UAVs), the monitoring system comprising: The point cloud model building module is used to acquire multi-view images of a preset pile area through UAVs in different monitoring cycles, and generate dense point cloud models for each monitoring cycle based on the multi-view images. The point cloud model alignment module is used to extract the point cloud features of the first dense point cloud model in a specific monitoring period and the second dense point cloud model in the previous monitoring period of the specific monitoring period, and to perform consistent alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features. The stack change recognition module is used to determine the increase or decrease change areas in the preset stack area based on the aligned first dense point cloud model and second dense point cloud model, and to identify the change information of each increase or decrease change area.

[0013] To address the aforementioned technical problems, a third aspect of this disclosure provides an electronic device, comprising: Processor; and A memory storing computer-executable instructions, which, when executed, cause the processor to perform the method described in any of the above embodiments.

[0014] To address the aforementioned technical problems, a fourth aspect of this disclosure provides a computer storage medium that stores one or more programs, which, when executed by a processor, implement the method described in any of the above embodiments.

[0015] Compared with existing technologies, this application has the following advantages: It generates a dense point cloud model by acquiring multi-view images of a preset stockpile area using UAVs at different monitoring cycles; subsequently, it extracts point cloud features from a specific monitoring cycle and its previous cycle, achieving consistent alignment of the two point cloud models in a unified coordinate system; finally, it determines the areas of increase or decrease in the stockpile area based on the aligned model and identifies the change information. This solution ensures data quality by dynamically adjusting aerial photography parameters, improves registration accuracy using deep learning and geometric constraints, and achieves accurate identification and classification of changed areas by combining point cloud differencing and semantic segmentation. This significantly enhances the monitoring capability and response speed for dynamic changes in the stockpile, providing reliable support for inventory management and safety early warning.

[0016] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of a method for monitoring dynamic changes in a stack based on an unmanned aerial vehicle (UAV) according to an embodiment of the present disclosure is shown. Figure 2 A second schematic flowchart of a method for monitoring dynamic changes in a stack based on an unmanned aerial vehicle (UAV) according to an embodiment of this disclosure is shown. Figure 3 A schematic flowchart of a method for real-time adjustment of drone shooting trajectory according to an embodiment of the present disclosure is shown; Figure 4 A schematic flowchart of a method for monitoring dynamic changes in a stack based on an unmanned aerial vehicle (UAV) according to an embodiment of this disclosure is shown in part three. Figure 5 A schematic flowchart of a method for monitoring dynamic changes in a stack based on an unmanned aerial vehicle (UAV) according to an embodiment of the present disclosure is shown in part four. Figure 6 A schematic flowchart of a method for monitoring dynamic changes in a stack based on an unmanned aerial vehicle (UAV) according to an embodiment of the present disclosure is shown in Figure 5. Figure 7 A schematic diagram of a UAV-based stack dynamic change monitoring system is shown according to an embodiment of the present disclosure; Figure 8 A schematic diagram of an electronic device structure according to an embodiment of the present disclosure is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The same reference numerals in the accompanying drawings denote the same or similar elements, components, or parts, and therefore repeated descriptions of the same or similar elements, components, or parts may be omitted below. It should also be understood that although qualifiers such as first, second, third, etc., indicating numbers may be used herein to describe various devices, elements, components, or parts, these devices, elements, components, or parts should not be limited by these qualifiers. That is, these qualifiers are only used to distinguish one from another. For example, a first device may also be referred to as a second device, without departing from the essence of the technical solution of this disclosure. Furthermore, the terms "and / or" and "and / or" refer to all combinations including the first or more of the listed items.

[0021] Please see Figure 1 , Figure 1 This is one of the flowcharts of a method for monitoring dynamic changes in a stack based on unmanned aerial vehicles (UAVs) provided in this disclosure, such as... Figure 1 As shown, the monitoring methods include: S11. Acquire multi-view images of the preset stack area using drones during different monitoring periods, and generate dense point cloud models for each monitoring period based on the multi-view images.

[0022] In this embodiment, a drone is used to acquire multi-view images of a preset pile area at different monitoring cycles. Based on these images, a dense point cloud model for each cycle is generated using 3D reconstruction technology. Through the acquisition of high-precision, large-scale 3D point cloud data, dynamic and quantitative monitoring of the pile morphology is achieved, providing a stable data foundation for subsequent change detection.

[0023] In this embodiment, a flight path is set in the material stacking area and a drone is used to perform high-overlap, multi-angle image acquisition. At the same time, the flight parameters are dynamically adjusted in combination with the terrain complexity and image quality to ensure the coverage and clarity of the subsequent modeling data.

[0024] In this embodiment, based on the acquired multi-view images, an image matching and reconstruction algorithm is used to generate a dense point cloud model, which is then subjected to noise removal and basic calibration to provide a spatial entity basis for dynamic analysis.

[0025] S12. Extract the point cloud features of the first dense point cloud model and the second dense point cloud model of the previous monitoring period for a specific monitoring period, and align the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features.

[0026] In this embodiment, by extracting the dense point cloud model features of adjacent monitoring periods (a specific period and its previous period), a method combining deep learning and geometric constraints is used to achieve high-precision registration of the point clouds of the two periods in a unified coordinate system. This solves the mismatch problem that is prone to occur when the morphology of the stack changes in traditional registration methods, ensures the consistency and alignment of time series data, and provides a reliable spatial reference for subsequent accurate detection of changes in stack volume.

[0027] In this embodiment, the 3D models of the current time and the previous time are registered with high precision. A combination of deep learning features and geometric constraints is introduced to achieve consistent alignment of the two data in a unified coordinate system.

[0028] In this embodiment, the specific monitoring period can be the current monitoring period, thereby determining the recent changes in the stack body based on the dense point cloud model of the current time point and the previous time point.

[0029] S13. Based on the aligned first dense point cloud model and second dense point cloud model, determine the increase or decrease change areas in the preset stack area, and identify the change information of each increase or decrease change area.

[0030] In this embodiment, based on the registered dual-temporal dense point cloud model, the geometric changes on the surface of the stack are calculated using a point cloud difference algorithm. Combined with a deep learning semantic segmentation model, the type (addition / reduction) and physical attributes of the changed areas are identified. Finally, the results are output in a graded manner according to the volume change magnitude, realizing a complete analysis process from geometric difference detection to semantic classification. This can accurately locate the dynamic change areas of the stack and quantify the degree of change, providing high-precision decision support for inventory management and safety early warning.

[0031] In this embodiment, differential calculation is performed on the registered point cloud model to identify the regions of increase or decrease in the stack size. Semantic segmentation technology is used to determine the category of change, and the change is graded according to the magnitude of volume change.

[0032] In this embodiment, the geometric outline and center position of the increased or decreased area can also be generated; the geometric outline, center position, and change information are combined to generate a stack change report. The identified changed areas and volume information are formatted and output to generate comparison images, change boundaries, and statistical charts, etc., to assist in subsequent stack yard management and safety analysis.

[0033] Specifically, the above-mentioned change detection results are further organized into intuitive reports and early warning information to support inventory management and safety monitoring in the mining area. After each monitoring cycle, the system automatically generates a dynamic change report for the stockpile, including the following: 1. Volume changes and levels in each stockpile area. Lists the volume increase or decrease for each stockpile and indicates its change level. 2. Inventory statistics update. Provides the change in the current total inventory of the stockpile based on the detection results, and provides a comparison of increases and decreases with the previous period. 3. Change visualization diagrams. Provides a comparison diagram or change heat map of the stockpile's three-dimensional model before and after the change, intuitively indicating the location and extent of material accumulation or damage. 4. Safety early warning information. Issues safety early warnings for significant changes or abnormal stockpile patterns to assist managers in taking timely measures. Through these reports, managers can comprehensively grasp the daily material changes in the stockpile, achieve refined inventory management, and receive timely early warnings before potential hazards arise to ensure operational safety.

[0034] In this embodiment, a dense point cloud model is generated by acquiring multi-view images of a preset stockpile area using a drone at different monitoring cycles. Subsequently, point cloud features from a specific monitoring cycle and its previous cycle are extracted to achieve consistent alignment of the two point cloud models in a unified coordinate system. Finally, the increased or decreased areas of the stockpile area are determined based on the aligned model, and the change information is identified. This solution ensures data quality by dynamically adjusting aerial photography parameters, improves registration accuracy using deep learning and geometric constraints, and achieves accurate identification and classification of changed areas by combining point cloud differencing and semantic segmentation. This significantly enhances the monitoring capability and response speed for dynamic changes in the stockpile, providing reliable support for inventory management and safety early warning.

[0035] Please see Figure 2 , Figure 2 This is a schematic diagram of the training method for a pipeline leakage analysis model provided in this disclosure, such as... Figure 2 As shown, the training method includes the following steps: S21. Based on the terrain boundary of the preset pile area, set the drone shooting trajectory, and control the drone to shoot real-time images according to the drone shooting trajectory; the drone shooting trajectory includes: flight path range, altitude level and shooting angle.

[0036] In this embodiment, the drone shooting trajectory is pre-planned according to the terrain boundary of the preset pile area, and the parameters such as the flight path range, altitude level and shooting angle are defined. The drone is controlled according to the trajectory to complete the real-time acquisition of multi-view images. The structured trajectory design ensures the coverage integrity and data consistency of the image acquisition, and provides systematic and standardized input data for subsequent 3D reconstruction.

[0037] In this embodiment, a drone equipped with a high-resolution imaging device is used to conduct daily aerial inspections of the open-pit mine stockpile, acquiring multi-angle image data covering the entire stockpile area. This system innovatively introduces an adaptive aerial photography adjustment mechanism based on edge sharpness and slope complexity during data acquisition. During flight, the system analyzes in real-time the sharpness of the stockpile edges and the complexity of the slope in the imagery, dynamically adjusting the drone's flight altitude, flight path density, and camera angle.

[0038] In this embodiment, the system initially sets the flight path range and altitude levels based on the terrain boundaries of the mining area's stockpile. A multi-altitude, multi-angle cross-flight strategy is adopted, with each flight path area having a preset flight altitude, adjacent waypoint spacing, and forward and lateral overlap rates of no less than 80% and 70%, respectively. The default angle coverage settings are three directions: vertical overhead and 45° left and right tilt.

[0039] S22. When the drone is shooting, adjust the drone's shooting trajectory in real time according to the clarity of the real-time image and the slope angle of the local area in the real-time image.

[0040] In this embodiment, during the drone shooting process, by analyzing the image edge sharpness and local area slope angle in real time, parameters such as flight altitude, flight path density, and shooting angle are dynamically adjusted. This solves the problem that traditional fixed flight paths are difficult to adapt to complex terrain, significantly improves image quality and data coverage density in key areas, and provides a reliable data foundation for subsequent high-precision 3D modeling and change detection.

[0041] In this embodiment, during the drone's mission, the system is equipped with an image feedback module and a real-time terrain analysis engine, which dynamically adjust the current aerial photography parameters based on image clarity and slope complexity.

[0042] S23. Generate multi-view images based on the real-time images captured.

[0043] In this embodiment, real-time images with different perspectives acquired during drone shooting are integrated to construct a multi-view image set covering the target area. Through the fusion of multi-angle images, the three-dimensional spatial information of the target pile is fully preserved, providing a comprehensive and highly redundant data foundation for subsequent dense point cloud reconstruction based on multi-view geometry, effectively improving the accuracy and robustness of three-dimensional modeling.

[0044] In a specific embodiment, such as Figure 3 As shown, the method for adjusting the drone's shooting trajectory in real time includes the following steps: S31. Identify the image clarity of real-time images, and when the image clarity is insufficient, adjust the waypoint spacing in the flight path range, the flight altitude in the altitude level, and / or the direction of the shooting angle to control the drone's flight.

[0045] In this embodiment, by evaluating image clarity in real time, parameters such as waypoint spacing, flight altitude, and shooting angle of the UAV flight path are dynamically adjusted to optimize image acquisition quality. This effectively solves the image blurring problem caused by factors such as lighting, occlusion, or distance, ensuring the integrity and high precision of multi-view image data and providing reliable data support for subsequent 3D reconstruction and change detection.

[0046] Specifically, for the current frame image Perform edge gradient calculation: , in, Let be the gradient magnitude of the i-th pixel. This represents the total number of pixels in the image.

[0047] like , If a gradient threshold is set for the system, the current image is considered blurry, triggering the following strategy: reduce flight altitude: , Set the system altitude adjustment. Reduce waypoint spacing: , Set and adjust the spacing for the system. Increase the angle acquisition perspective, such as expanding from 3 directions to 5 directions. The above strategy is only one embodiment of this solution. The image clarity can be improved by triggering other preset strategies to increase the number of acquired images or by adjusting the angle of the captured images.

[0048] S32. Calculate the average slope angle of the local area based on real-time images, and when the average slope angle is higher than the preset slope threshold, adjust the waypoint density and / or shooting angle direction in the local area, and improve the shooting resolution of the UAV in the local area.

[0049] In this embodiment, by analyzing the average slope angle of a local area in real time, when the slope exceeds a preset threshold, the density of waypoints on the flight path in the area is dynamically increased, the shooting angle direction is optimized, and the resolution of UAV image acquisition is improved. This significantly enhances the image coverage density and detail capture capability for complex terrains (such as steep slopes and gullies), effectively solving the data loss problem of traditional fixed flight paths in undulating terrain, and providing a more reliable data foundation for high-precision 3D modeling and terrain change monitoring.

[0050] Specifically, the system utilizes the UAV's inertial measurement unit (IMU) and terrain data to perform sparse 3D reconstruction or depth estimation on consecutive image frames in front, forming a low-resolution local digital elevation model (DEM) within a small area. The height difference between two adjacent points within the grid is then selected. Horizontal distance Calculate the local average slope angle: , Among them, height difference The horizontal distance is estimated by the sparse point cloud model. It is obtained from image reprojection scale and flight trajectory estimation. IMU data is used to supplement the current flight attitude and heading, and to help estimate the actual tilt trend of the terrain corresponding to the image.

[0051] If the slope , Setting a slope threshold for the system marks areas as highly complex. For these marked areas, the following responses are automatically triggered: increasing waypoint density and reducing waypoint spacing within the area; expanding the aerial viewing angle range and setting multi-angle tilt shooting; and forcibly switching to high-resolution mode. This adaptive aerial photography strategy can significantly improve image quality while ensuring efficiency, providing a high-precision data foundation for subsequent 3D reconstruction and change detection. Similarly, the above strategy is only one embodiment of this solution; image clarity can be improved by triggering other preset strategies to increase the number of acquired images or by adjusting the angle of the captured images.

[0052] Please see Figure 4 , Figure 4 This is the third schematic diagram of a method for monitoring dynamic changes in a stack based on unmanned aerial vehicles (UAVs) provided in this disclosure. As shown in the figure, the monitoring method includes the following steps: S41. Based on the structure-of-motion method, extract the camera pose and initial point cloud image of each image in the multi-view image.

[0053] In this embodiment, Structure from Motion (SfM) is a technique that recovers camera parameters and 3D scene structure by analyzing image sequences. This technique is applicable to situations where the camera is moving in a static scene, and the core process includes feature point extraction, image matching, camera pose estimation, and 3D point cloud reconstruction.

[0054] In this embodiment, a motion-based structure restoration method is adopted. By analyzing the geometric constraints between multi-view images, the camera pose (position and orientation) of each image is automatically extracted and the initial sparse point cloud model is reconstructed. High-precision three-dimensional spatial positioning can be achieved without relying on external ranging equipment, providing accurate camera parameters and an initial structural framework for subsequent dense point cloud reconstruction and dynamic change monitoring.

[0055] Specifically, in the SfM stage, scale-invariant feature extraction (SIFT) is used to extract feature points from all input images, constructing a feature matching map between images. For successfully matched image pairs, the system uses the five-point method to solve for the essential matrix. And recover the relative pose between cameras. That is, the first The spatial pose of the image relative to a reference viewpoint, where The rotation matrix represents the camera orientation. This represents the translation vector, i.e., the camera position. In SfM, the poses of all image views are set relative to an initial image, usually the origin of the world coordinate system. One of the goals of the solution process is to recover these poses. Then, in conjunction with the image intrinsic parameter matrix... Constructing a perspective model Furthermore, bundle adjustment is performed on all views to minimize all 3D points. Projection error: , in, For three-dimensional points In the image The observation location in The projection function of the camera model. This indicates that three-dimensional points Through camera parameters Projected onto image The system obtains the pose solutions of all cameras in the world coordinate system through this global optimization process, and also reconstructs a set of sparse but structurally accurate 3D spatial points.

[0056] S42. Based on multi-view stereo matching technology, a multi-view cost volume is constructed using the camera pose of each image and the initial point cloud image, and a pixel depth map is calculated.

[0057] In this embodiment, Multi-View Stereo (MVS) is a technique for calculating the three-dimensional structure of a scene using multi-view images. Its core objective is to establish pixel-level correspondences to derive depth information.

[0058] In this embodiment, the multi-view cost body is the core data structure in multi-view stereo matching (MVS), used to represent the matching cost of pixels under different viewpoints. Its generation and aggregation directly affect the accuracy of depth estimation. The multi-view cost body is constructed by mapping source image features to a reference viewpoint and based on depth sampling values. Specific steps include: Feature extraction: Extracting multi-scale features (e.g., 1 / 4, 1 / 2, 1 resolution) using a pyramid feature network (FPN). Feature mapping: Projecting source view features to the reference viewpoint according to camera parameters (intrinsic and extrinsic parameters) and sampling depth values ​​to form the multi-view cost body. Cost aggregation: Aggregating the multi-view cost body through variance operations or the Transformer attention mechanism to reduce the impact of noise.

[0059] In this embodiment, based on the multi-view stereo matching (MVS) technique, the camera pose and initial sparse point cloud obtained by SfM are used to construct a multi-view cost volume and generate a dense depth map through pixel-level depth calculation. Through multi-view geometric constraints and pixel-level optimization, the initial sparse point cloud is expanded into a high-resolution dense point cloud, which significantly improves the detail representation and spatial accuracy of the 3D model.

[0060] In this embodiment, upon entering the MVS stage, the system takes the camera pose and preliminary point cloud output by SfM as input, constructs a multi-view cost volume using a pixel-consistency-based matching strategy, and estimates the pixel-level depth map through Patch Match. The PatchMatch algorithm is a highly efficient image registration algorithm based on a random search and propagation mechanism, mainly applied in stereo matching, 3D reconstruction, and image inpainting. Its innovation lies in rapidly propagating high-quality matching results through spatial continuity, significantly reducing computational complexity.

[0061] S43. The pixel depth map and the multi-view cost volume are fused to obtain a dense point cloud model.

[0062] In this embodiment, by fusing the pixel depth map generated by multi-view stereo matching with the spatial constraint information of the cost volume from multiple perspectives, a high-precision dense point cloud model is generated using a depth estimation optimization strategy. By utilizing the basic geometric information provided by the depth map and the global consistency optimization of the cost volume, the local ambiguity problem of single-view depth estimation is effectively solved, and the integrity and measurement accuracy of dense point clouds in complex terrain and edge areas are significantly improved.

[0063] In this embodiment, the depth estimation results are fused between views to obtain the final dense point cloud model: .

[0064] During the reconstruction process, a photometric consistency term and an edge preservation term are introduced to jointly construct the cost function, thereby improving the accuracy of depth calculation and the stability of edge regions. The final point cloud model can achieve a density of several thousand points per square meter, providing spatial resolution to support subsequent point cloud differencing, semantic recognition, and volumetric calculation.

[0065] In this embodiment, necessary data preprocessing can also be performed on the generated point cloud model, including removing noise points and outliers, and performing preliminary coordinate calibration, to improve the accuracy and stability of the model and output point cloud data. The processed 3D model of the current moment will serve as the basis for subsequent change detection, and will be used for comparative analysis together with the baseline model of the previous monitoring period.

[0066] Please see Figure 5 , Figure 5 This is the fourth flowchart of a method for monitoring dynamic changes in a stack based on unmanned aerial vehicles (UAVs) provided in this disclosure. As shown in the figure, the monitoring method includes the following steps: S51. Extract the local context features of each point in the first dense point cloud model and the second dense point cloud model as point cloud features.

[0067] In this embodiment, the local neighborhood analysis method is used to extract the local geometric structure information (such as normal vector, curvature, neighborhood point distribution, etc.) of each point in the first and second dense point cloud models in three-dimensional space as point cloud features. This effectively captures the microscopic geometric details and spatial topological relationships of the point cloud data, providing more discriminative and robust feature descriptions for subsequent point cloud registration, change detection, or classification tasks, and significantly improving the accuracy and reliability of three-dimensional data analysis in complex scenes.

[0068] Specifically, for the current 3D model Compared with the previous cycle model Deep feature extraction is performed. The feature extraction network used is a point cloud encoder based on the PointNet++ architecture, and its input is point cloud fragments. The output consists of point-level and fragment-level feature representations. The network learns representation functions through the embedding space. Extract the local context features for each point. The features extracted from the point clouds at two time points are denoted as follows: .

[0069] In this embodiment, PointNet++ is an improved version of PointNet. It effectively solves the problem of local feature extraction from point cloud data through a hierarchical feature learning mechanism and has wide applications in fields such as autonomous driving and 3D modeling.

[0070] S52. Based on local context features, determine the first point in the first dense point cloud model that matches the second point in the second dense point cloud model; and use the first point and the second point as a matching point pair.

[0071] In this embodiment, based on local context feature similarity measurement, the corresponding point in the first dense point cloud model that best matches the geometric structure of the target point in the second dense point cloud model is searched. The precise point-to-point relationship across point cloud models is constructed through feature matching algorithm, realizing the establishment of high-precision correspondence between point cloud data collected from different perspectives or at different times. This effectively overcomes the mismatch problem caused by perspective changes, occlusion or noise, and provides a reliable basic correspondence for subsequent point cloud registration, 3D reconstruction or change detection tasks.

[0072] In this embodiment, by matching in the embedding space and The system can obtain a coarse set of time-matched point pairs. , as the initial alignment reference.

[0073] S53. Based on a random algorithm, determine the initial transformation matrix that minimizes the distance between the first point in the first dense point cloud model and the second point in the second dense point cloud model after transformation.

[0074] In this embodiment, a random algorithm (such as RANSAC) is used to randomly sample and generate a candidate set of initial transformation matrices in the solution space. By evaluating the distance error between the points in the first dense point cloud model after transformation and the corresponding points in the second dense point cloud, the optimal initial transformation matrix that minimizes the spacing between matching point pairs is selected. Through a probabilistic search strategy, local optima traps are efficiently avoided, and robust initial transformation estimates are quickly obtained, providing reliable initial values ​​for subsequent fine registration (such as the ICP algorithm), significantly improving the success rate and convergence speed of point cloud registration in complex scenarios.

[0075] In this embodiment, to further eliminate outlier matches and optimize rigid transformation estimation, the system inputs the set of point pairs into the RANSAC iterative framework. Let the transformation matrix be... The goal is to find an optimal set of rotation matrices and translation vectors that makes the set of matching points in the source model... After transformation, it should approximate the corresponding point in the target model as closely as possible. ,Right now: .

[0076] In this embodiment, Random Sample Consensus (RANSAC) is a powerful algorithm used in the fields of computer vision and image processing to estimate model parameters from a set of data containing outliers or noise. The core idea of ​​RANSAC is to iteratively select a sufficient number of data points from the dataset to fit a model, then evaluate which data points are best suited to this model (i.e., inliers), and finally select the model that best suits the dataset by repeating this process multiple times.

[0077] S54. Establish the correspondence between specific points in the first dense point cloud model and the second dense point cloud model.

[0078] In this embodiment, a cross-model point-level correspondence is established between the first and second dense point cloud models through feature matching (such as local geometric descriptors and deep learning features) or geometric constraints (such as spatial distance and normal vector consistency). This provides an accurate spatial correlation basis for point cloud registration, fusion, or change detection, effectively solving the mismatch problem caused by viewpoint differences, occlusion, or dynamic changes, and significantly improving the alignment accuracy of 3D models and the reliability of multi-source data fusion.

[0079] In this embodiment, robust geometric constraints are combined, such as the Iterative Closest Point (ICP) algorithm supplemented by the RANSAC method to remove outlier matches, to accurately calculate the spatial transformation parameters between the models at different time points. That is, the nearest point correspondence between the current point cloud and the target point cloud is established, and the correspondence is fixed.

[0080] S55. Update the initial transformation matrix iteratively, calculate the error of a specific point in the correspondence of a specific point, until the error converges or the final transformation matrix is ​​obtained.

[0081] In this embodiment, an iterative optimization algorithm (such as ICP and its variants) is used to calculate the matching error of specific corresponding points between point clouds based on the current transformation matrix. The transformation parameters are gradually adjusted using error feedback until the error meets the convergence condition or reaches the maximum number of iterations. Finally, a high-precision transformation matrix is ​​output. The robustness of point cloud registration is significantly improved through the closed-loop feedback mechanism, effectively overcoming problems such as initial estimation error and local optimum solutions. Finally, globally optimal geometric transformation parameters are obtained, providing a reliable guarantee for high-precision alignment and fusion of 3D models.

[0082] In this embodiment, the rigid transformation is iteratively updated. The process continuously minimizes the Euclidean error. It terminates when the error converges or the maximum number of iterations is reached, outputting the final alignment matrix T. Ultimately, the model at the current time step... In transformation The coordinates are then mapped back to the coordinate system of the previous time step: .

[0083] S56. Using the final transformation matrix, transform the first dense point cloud model into the coordinate system of the second dense point cloud model, thus achieving consistent alignment of the first and second dense point cloud models in a unified coordinate system.

[0084] In this embodiment, the optimized final transformation matrix is ​​used to perform spatial coordinate transformation on the first dense point cloud model, so that it shares the same coordinate system reference with the second dense point cloud model. This achieves precise geometric alignment between the two in three-dimensional space, completely eliminating coordinate system deviations caused by differences in acquisition perspective, time, or equipment in multi-source point cloud data. This provides a highly consistent spatial reference for subsequent point cloud fusion, 3D reconstruction, change detection, or feature analysis, significantly improving the accuracy and reliability of multimodal 3D data processing.

[0085] In this embodiment, the dual-temporal consistency registration method combines contextual features extracted by deep learning with traditional geometric matching strategies, overcoming the limitations of traditional ICP (Internal Phase Conformance Program) which is susceptible to local changes and difficult to initially align, significantly improving the registration accuracy of models from different time phases. Through this registration strategy combining deep features and geometric constraints, even if the stockpile morphology undergoes local changes, high-precision alignment between the two periods of data can be achieved, ensuring spatial consistency between the time-series data. After registration, the stockpile models at the current and previous times are unified under the same coordinate system, laying the foundation for subsequent differential analysis and change detection.

[0086] Please see Figure 6 , Figure 6 This is the fifth flowchart of a method for monitoring dynamic changes in a stack based on unmanned aerial vehicles (UAVs) provided in this disclosure. As shown in the figure, the monitoring method includes the following steps: S61. Perform differential operation on the aligned first dense point cloud model and the second dense point cloud model to determine the increase or decrease change area in the preset stack area.

[0087] In this embodiment, point-by-point or pixel-level differential operations are performed on the two dense point cloud models under a unified coordinate system. By comparing the differences in spatial location and geometric attributes, the volume increase or decrease areas within the preset pile body area are accurately identified. This breaks through the limitations of traditional two-dimensional monitoring and realizes millimeter-level quantitative analysis of the three-dimensional deformation of the pile body. It effectively captures local volume changes caused by loading and unloading, settlement or collapse, providing a high-precision dynamic change monitoring method for scenarios such as warehouse management and geological disaster early warning.

[0088] In this embodiment, differential operations are performed on the point cloud data at different times to extract newly added or reduced spatial regions in the stockpile. The M3C2 algorithm is used for differential analysis, and the orthogonal distance difference of each reference point is calculated based on the normal vector direction, local scale, and point density. Set a threshold for the significance of change. ,like If the point is significantly changed, it is considered a significant change point and added to the set of candidate change regions C, which means adding or removing change regions.

[0089] S62. Input the multi-view images and point cloud fragments corresponding to the increased or decreased areas into the pre-trained transformation category discrimination model, and output the change information of the increased or decreased areas.

[0090] In this embodiment, local segments corresponding to areas of increase or decrease in multi-source data (multi-view images and point cloud fragments) are input into a pre-trained deep learning model. The model uses joint analysis of geometric and texture features to automatically identify and output the type of change (such as addition, disappearance, displacement, etc.) and semantic information of the area. Through multimodal data fusion and end-to-end intelligent analysis, the automation level and semantic richness of change detection are significantly improved, effectively solving the limitations of traditional methods that rely on manual interpretation or a single data source. This provides high-precision and interpretable change information output for scenarios such as disaster monitoring and urban renewal.

[0091] Specifically, to avoid misidentifying non-material pile changes such as ground shadows, equipment movement, and robotic arm displacement as dynamic changes in the soil pile, the system integrates a semantic segmentation model to classify changes based on candidate regions. This system employs a deep semantic segmentation network with an improved Transformer-UNet hybrid architecture. The model input consists of image slices and point cloud fragments corresponding to the changed regions. During the training phase, supervised learning is performed using a dataset of finely labeled mining scene data. To address the scarcity of samples in small-volume change regions, a weighted cross-entropy term is introduced into the loss function to enhance the network's ability to identify subtle pile changes. Output: Category label for each point: .

[0092] In this embodiment, the system performs volume integration on each semantically filtered region of change. The point cloud has been divided into regular voxels of a certain volume, and all regions of change are classified into slight change, moderate change, and significant change regions based on volume. Each classified region is simultaneously labeled with its geometric outline, center position, and change type, forming a structured output result. In this way, all volume changes in the stockpile within the current monitoring period are identified and classified, providing a quantitative basis for inventory management and a reference indicator for safety early warning.

[0093] Please see Figure 7 , Figure 7 This disclosure provides a dynamic change monitoring system for a stack based on unmanned aerial vehicles (UAVs). The monitoring system includes: a point cloud model construction module 11, a point cloud model alignment module 12, and a stack change identification module 13.

[0094] In this embodiment, the point cloud model construction module 11 is used to acquire multi-view images of a preset pile area through a drone at different monitoring periods, and generate dense point cloud models for each monitoring period based on the multi-view images.

[0095] In this embodiment, the point cloud model alignment module 12 is used to extract the point cloud features of the first dense point cloud model in a specific monitoring period and the second dense point cloud model in the previous monitoring period of the specific monitoring period, and to perform consistent alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features.

[0096] In this embodiment, the stack change recognition module 13 is used to determine the increase or decrease change areas in the preset stack area based on the aligned first dense point cloud model and second dense point cloud model, and to identify the change information of each increase or decrease change area.

[0097] In this embodiment, the point cloud model construction module 11 is specifically used to set the drone shooting trajectory according to the terrain boundary of the preset pile area, and control the drone to shoot real-time images according to the drone shooting trajectory; the drone shooting trajectory includes: flight path range, altitude level and shooting angle; when the drone is shooting, the drone shooting trajectory is adjusted in real time according to the impact clarity of the real-time image and the slope angle of the local area in the real-time image; multi-view images are generated based on the real-time images that have been shot.

[0098] In this embodiment, the point cloud model construction module 11 is specifically used to identify the image clarity of real-time images, and when the image clarity is insufficient, adjust the waypoint spacing in the flight path range, the flight altitude in the altitude level, and / or the direction of the shooting angle to control the flight of the UAV; calculate the average slope angle of the local area based on the real-time images, and when the average slope angle is higher than the preset slope threshold, adjust the waypoint density in the flight path range of the local area and / or the direction of the shooting angle to improve the shooting resolution of the UAV in the local area.

[0099] In this embodiment, the point cloud model construction module 11 is used to extract the camera pose and initial point cloud image of each image in the multi-view imagery based on the structure of motion reconstruction method; construct the multi-view cost volume based on the camera pose and initial point cloud image of each image based on the multi-view stereo matching technology, and calculate the pixel depth map; and fuse the pixel depth map and the multi-view cost volume to obtain a dense point cloud model.

[0100] In this embodiment, the point cloud model alignment module 12 is specifically used to extract the local context features of each point in the first dense point cloud model and the second dense point cloud model as point cloud features; based on the local context features, determine a first point in the first dense point cloud model that matches a second point in the second dense point cloud model; use the first point and the second point as a matching point pair; based on a random algorithm, determine an initial transformation matrix that minimizes the distance between the first point in the first dense point cloud model and the second point in the second dense point cloud model after transformation; establish a correspondence between specific points in the first dense point cloud model and the second dense point cloud model; update and iterate the initial transformation matrix, calculate the error of specific points in the correspondence between specific points until the error converges, and obtain the final transformation matrix; and transform the first dense point cloud model into the coordinate system of the second dense point cloud model through the final transformation matrix, thus completing the consistent alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system.

[0101] In this embodiment, the pile change recognition module 13 is specifically used to perform differential operation on the aligned first dense point cloud model and the second dense point cloud model to determine the increase or decrease change area in the preset pile area; input the multi-view image and point cloud fragment corresponding to the increase or decrease change area into the pre-trained transformation category discrimination model, and output the change information of the increase or decrease change area.

[0102] In this embodiment, the monitoring system further includes: a report generation module, used to generate the geometric outline and center position of the increased or decreased area; and to combine the geometric outline, center position and change information to generate a pile body change report.

[0103] like Figure 8 As shown, this embodiment of the present disclosure provides an electronic device, including a processor 1110, a communication interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other through the communication bus 1140. Memory 1130 is used to store computer programs; When the processor 1110 executes the program stored in the memory 1130, it implements any of the above methods.

[0104] The electronic device provided in this embodiment of the present disclosure includes a processor 1110 that executes a program stored in a memory 1130 to acquire multi-view images of a preset pile area via a drone during different monitoring periods, and generates dense point cloud models for each monitoring period based on the multi-view images; extracts point cloud features from the first dense point cloud model of a specific monitoring period and the second dense point cloud model of the previous monitoring period, and aligns the first and second dense point cloud models in a unified coordinate system based on the point cloud features; and determines the increased or decreased areas in the preset pile area based on the aligned first and second dense point cloud models, and identifies the change information of each increased or decreased area.

[0105] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, and a component bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0106] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0107] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1130 may also be at least one storage device located remotely from the aforementioned processor 1110.

[0108] The processor 1110 mentioned above can be a general-purpose processor 1110, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0109] This disclosure provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors 1110 to implement the methods of any of the above embodiments.

[0110] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this disclosure is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0111] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for monitoring dynamic changes of a stack based on a UAV, characterized in that, The monitoring method comprises: acquiring multi-view images of a preset stockpile area by a UAV in different monitoring periods, and generating dense point cloud models of each monitoring period based on the multi-view images respectively; extracting point cloud features of a first dense point cloud model of a specific monitoring period and a second dense point cloud model of a previous monitoring period of the specific monitoring period, and performing consistency alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features; determining change regions in the preset stockpile area according to the aligned first dense point cloud model and the second dense point cloud model, and identifying change information of each change region.

2. The monitoring method according to claim 1, characterized in that, The method of acquiring multi-view images of a preset stockpile area by a UAV in different monitoring periods comprises: setting a UAV shooting track according to the terrain boundary of the preset stockpile area, and controlling the UAV to shoot real-time images according to the UAV shooting track; the UAV shooting track comprises a flight route range, a height level and a shooting angle; when the UAV is shooting, adjusting the UAV shooting track in real time according to the influence definition of the real-time images and the slope angle of a local region in the real-time images; generating the multi-view images based on the completed real-time images.

3. The monitoring method according to claim 2, characterized in that, When the UAV is shooting, adjusting the UAV shooting track in real time according to the influence definition of the real-time images and the slope angle of a local region in the real-time images, comprising: identifying the image definition of the real-time images, and adjusting the distance between flight points in the flight route range, the flight height in the height level and / or the direction of the shooting angle to control the UAV flight when the image definition is insufficient; calculating the average slope angle of the local region according to the real-time images, and adjusting the flight point density in the flight route range and / or the direction of the shooting angle in the local region and improving the shooting resolution of the UAV in the local region when the average slope angle is higher than a preset slope threshold.

4. The monitoring method of claim 1, wherein, The method of generating dense point cloud models of each monitoring period based on the multi-view images respectively comprises: extracting camera poses and initial point cloud images of each image in the multi-view images based on a motion recovery structure method; constructing a multi-view cost volume through the camera poses and the initial point cloud images of each image based on a multi-view stereo matching technology, and calculating a pixel depth map; fusing the pixel depth map and the multi-view cost volume to obtain the dense point cloud model.

5. The monitoring method of claim 1, wherein, The method of extracting point cloud features of a first dense point cloud model of a specific monitoring period and a second dense point cloud model of a previous monitoring period of the specific monitoring period, and performing consistency alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features, comprises: extracting local context features of each point in the first dense point cloud model and the second dense point cloud model as the point cloud features; determining a first point in the first dense point cloud model that matches a second point in the second dense point cloud model according to the local context features; taking the first point and the second point as a matching point pair; determining, based on a random algorithm, an initial transformation matrix that minimizes the distance between a first point in the first dense point cloud model and a second point in the second dense point cloud model after transformation; establishing a correspondence between specific points in the first dense point cloud model and the second dense point cloud model; updating the initial transformation matrix iteratively, calculating the error of specific points in the correspondence between specific points, until the error converges or a final transformation matrix is obtained; transforming the first dense point cloud model into the coordinate system of the second dense point cloud model through the final transformation matrix, and completing the consistency alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system.

6. The monitoring method of claim 1, wherein, According to the aligned first dense point cloud model and the second dense point cloud model, the change information of each of the increase / decrease change regions is determined, including: performing difference operation on the aligned first dense point cloud model and the second dense point cloud model to determine the increase / decrease change region in the preset stockpile region; inputting the multi-view image and the point cloud segment corresponding to the increase / decrease change region into a pre-trained transformation category discrimination model to output the change information of the increase / decrease change region.

7. The monitoring method according to any one of claims 1 to 6, characterized in that, The monitoring method further includes: generating the geometric contour and the center position of the increase / decrease change region; combining the geometric contour, the center position, and the change information to generate a stockpile change report.

8. An unmanned aerial vehicle (UAV)-based stockpile dynamic change monitoring system, comprising: The monitoring system includes: a point cloud model construction module configured to acquire multi-view images of a preset stockpile region through a UAV at different monitoring periods, and generate dense point cloud models for each monitoring period based on the multi-view images; a point cloud model alignment module configured to extract point cloud features of a first dense point cloud model of a specific monitoring period and a second dense point cloud model of a previous monitoring period of the specific monitoring period, and perform consistency alignment of the first dense point cloud model and the second dense point cloud model in a unified coordinate system based on the point cloud features; a stockpile change identification module configured to determine increase / decrease change regions in the preset stockpile region according to the aligned first dense point cloud model and the second dense point cloud model, and identify change information of each of the increase / decrease change regions.

9. An electronic device, comprising: including: a processor; and a memory storing computer executable instructions that, when executed, cause the processor to perform the method according to any one of claims 1-7.

10. A computer storage medium, characterized in that wherein, the computer storage medium stores one or more programs that, when executed by a processor, implement the method of any one of claims 1-7.

Images