Multi-mode integrated quick acquisition, analysis and calculation method for river sand excavation data

Abstract

The invention discloses a multi-mode integrated quick acquisition and analysis calculation method for river sand excavation data. The method comprises the following steps: a sand excavation range identification module extracts river sand excavation range data from a high-resolution satellite image based on a deep learning algorithm; the data quick acquisition module receives input river sand excavation range data, and realizes quick acquisition of overwater and underwater topographic data of a sand excavation area; the intelligent fusion calculation module receives the input overwater and underwater topographic data, and realizes automatic seamless splicing of the overwater and underwater topographic data based on an intelligent fusion algorithm; and the spatial analysis visualization module receives the input fused overwater and underwater topographic data, quickly constructs a three-dimensional model and a digital elevation model of the river channel, obtains the sandstone variation of the river channel through multi-temporal analysis, and visually presents the analysis result. According to the invention, quick acquisition, analysis and calculation of river sand excavation data can be realized, data acquisition, modeling, analysis and calculation are carried out through a multi-modal integrated mechanism, and the reliability and timeliness of river sand excavation monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the fields of river and lake ecological environment, digital twins, and smart water conservancy, and specifically to a multi-modal integrated river sand mining data rapid acquisition and analysis calculation method. Background Art

[0002] The management of river sand mining is an important part of protecting rivers, lakes and reservoirs, and it is related to flood control safety, water supply safety, navigation safety, ecological safety and the safety of important infrastructure. Strengthening the management of river and lake sand mining, maintaining the health of rivers and lakes, and ensuring the sustainable use of river and lake functions are the inherent requirements for promoting the construction of ecological civilization and promoting the harmonious coexistence of man and nature. As an important water administration work, river sand mining management can achieve sustainable development of the economy and society through the sustainable use of water and sand resources through administration according to law.

[0003] In recent years, various parts of the country have actively explored and implemented information technology in the management of river and lake sand mining, built a three-dimensional and comprehensive perception system, and achieved remarkable results. However, various monitoring methods are relatively independent, the collaborative processing effect is not good, and there is still room for improvement in overall work efficiency.

[0004] The inventors of this application discovered through research during the process of realizing the present invention that the reliability and timeliness of river sand mining monitoring can be improved by integrating multimodal technologies and equipment in river sand mining monitoring work, making good processing connections, reducing manual intervention, and forming an integrated business process closed loop. Summary of the invention

[0005] The purpose of the present invention is to propose a multi-modal integrated river sand mining data rapid acquisition and analysis calculation method, improve the reliability and timeliness of river sand mining monitoring, and form a standard business processing flow.

[0006] The technical solution of the present invention is:

[0007] A multi-modal integrated river sand mining data rapid acquisition and analysis calculation method uses a sand mining range identification module, a data rapid acquisition module, an intelligent fusion calculation module and a spatial analysis visualization module; the method comprises the following steps:

[0008] Step 1: The sand mining range identification module performs image recognition training on high-resolution satellite image data of the operation area based on a deep learning algorithm, extracts river sand mining range data, and inputs the extracted river sand mining range data into the data rapid acquisition module and the intelligent fusion calculation module;

[0009] Step 2, the data rapid acquisition module receives the river channel sand mining range data input by the sand mining range identification module, automatically plans the water and underwater data acquisition route according to the river channel sand mining range data, realizes the rapid acquisition of water and underwater terrain data of the sand mining area through the water and land two-phase multi-module sensor, and inputs the acquired water and underwater terrain data into the intelligent fusion calculation module;

[0010] Step 3, the intelligent fusion calculation module receives the water surface and underwater terrain data input by the data rapid acquisition module, and realizes automatic seamless splicing of the water surface and underwater terrain data through the intelligent fusion algorithm to obtain the fused water surface and underwater terrain data;

[0011] Step 4: The spatial analysis visualization module receives the fused water and underwater terrain data input by the intelligent fusion calculation module to construct a three-dimensional model of the river and a digital elevation model. Through multi-temporal comparison analysis, the change in sand and gravel before and after sand mining is obtained, and the analysis results are rendered and visualized using GIS and virtual reality technology.

[0012] Furthermore, the step 1 includes:

[0013] Step 1.1: The sand mining range identification module obtains high-resolution satellite image data of the operation area and manually marks the sand mining area;

[0014] Step 1.2: Use the UNet algorithm to perform deep learning on the labeled high-resolution satellite image data to extract the river sand mining range and sand mining characteristic range data, and then input the extracted river sand mining range data into the data rapid acquisition module and the intelligent fusion calculation module.

[0015] Furthermore, step 1.1 specifically includes:

[0016] Step 1.1.1, image preprocessing: process the periodic noise of the acquired high-resolution satellite image data of the operating area, and then perform orthorectification to correct the image tilt deviation and the error generated during the projection process; perform image enhancement on the orthorectified high-resolution satellite image data; and finally adjust the color balance of the image;

[0017] Step 1.1.2, feature annotation: collect a sand and gravel feature sample set on the remote sensing image processed in step 1.1.1 by manual annotation.

[0018] Furthermore, step 1.2 specifically includes:

[0019] Step 1.2.1, U-Net architecture design: By comparing high-resolution satellite remote sensing images from different periods, the U-Net network architecture and related training parameters are designed to obtain the U-Net convolutional neural network;

[0020] Step 1.2.2, sample training: select and preprocess the sand and gravel feature sample set obtained in step 1.1.2 to obtain training samples, and input the training samples into the U-Net convolutional neural network designed in step 1.2.1 for training to obtain a trained constant residual type U-Net deep neural network;

[0021] Step 1.2.3, sand mining range extraction: The remote sensing image processed in step 1.1.1 is divided into blocks and input into the constant residual U-Net deep neural network trained in step 1.2.2, and then spliced ​​to obtain the sand and gravel extraction probability map. Finally, the grayscale threshold is set to binarize the image to obtain the river sand mining range data.

[0022] Furthermore, step 2 includes:

[0023] Step 2.1, coordination and integration of multi-module sensors for both land and water phases: Use GPS time and 1PPS to continuously adjust the time of each sensor so that the time of each sensor is always synchronized with the GPS time; determine the deviation between the coordinate system of each sensor and the coordinate system of the carrier, and calibrate through data acquisition software to ensure that the coordinates of each sensor and the coordinate system of the carrier are parallel or coincident;

[0024] Step 2.2, data collection: automatically plan the survey line according to the sand mining range obtained in step 1.2, control each measurement subsystem to carry out independent data collection work for each sensor, and obtain the above-water and underwater terrain data;

[0025] Step 2.3, automatic data preprocessing and rapid transmission: Automatically check, process and filter the surface and underwater terrain data collected in step 2.2, delete unqualified data, retain useful data, and transmit the surface and underwater terrain data collected in real time in step 2.2 to the intelligent fusion computing module through Ethernet, 4G / 5G, long-distance transmission or narrowband Internet of Things communication technology.

[0026] Furthermore, step three includes:

[0027] Step 3.1, data preprocessing: Analyze and process the surface and underwater terrain data transmitted in step 2.3 to generate point cloud data, and perform denoising and filtering on the point cloud data to improve the quality and accuracy of the data;

[0028] Step 3.2, data registration: Based on the point cloud registration algorithm, the point cloud data of different perspectives or at different times processed in step 3.1 are registered to obtain accurate spatial position and posture information, ensuring the consistency of spatiotemporal reference and accuracy of multi-source point cloud data;

[0029] Step 3.3, data fusion: Based on the surface reconstruction algorithm MLS, the multi-source point cloud data registered in step 3.2 are fused and processed, and the discrete point cloud data are converted into a continuous surface representation through least squares fitting, thereby generating a more complete point cloud model;

[0030] Step 3.4, data post-processing: The grid-based hole repair algorithm is used to perform model repair on the point cloud data fused in step 3.3;

[0031] Step 3.5, generation of river channel three-dimensional model and digital elevation model DEM: resample the point cloud data processed in step 3.4 and generate the river channel three-dimensional model and digital elevation model.

[0032] Furthermore, step 3.1 specifically includes:

[0033] Step 3.1.1, analyzing and processing the above-water and underwater terrain data to generate point cloud data;

[0034] Step 3.1.2, analyzing the spatial distance between the point cloud noise points and the target point cloud in the point cloud data, and dividing the point cloud noise points into large-scale outlier noise points and small-scale fluctuation noise points;

[0035] Step 3.1.3, remove large-scale outlier noise:

[0036] For any point p i , the average distance within its k-neighborhood for:

[0037]

[0038] Among them, d ij Indicates p i With neighboring point p j The distance between

[0039] Assuming that the average distance follows a normal distribution, the mean μ and standard deviation σ of the average distance are calculated as follows:

[0040]

[0041] Where N is the number of point clouds;

[0042] The standard range of average distance is calculated based on the mean and standard deviation: (μ-std·σ,μ+std·σ), where std is used to control the size of the average distance threshold. When the average distance of a sampling point is greater than this range, it is considered an outlier and removed.

[0043] Step 3.1.4, smooth small-scale fluctuation noise.

[0044] Furthermore, step 3.2 specifically includes:

[0045] Step 3.2.1, perform rough matching on the point cloud data processed in step 3.1: use the random sampling consistent RANSAC algorithm to perform iterative sampling to obtain the transformation matrix; use the obtained transformation matrix to perform point cloud transformation operations to align two point clouds at different positions;

[0046] Step 3.2.2, perform fine matching on the point cloud data after rough matching in step 3.2.1: use the nearest point iterative ICP algorithm to solve the rotation and translation transformation matrix of the corresponding points, where the corresponding points are the points closest to each other in the two point clouds; use the obtained rotation and translation matrix to perform spatial transformation of the point cloud to obtain the registered multi-source point cloud data.

[0047] Furthermore, step 3.3 specifically includes:

[0048] Step 3.3.1, define the neighborhood: Based on the multi-source point cloud data registered in step 3.2, determine a radius with each point as the center, and select the neighborhood points within the radius;

[0049] Step 3.3.2, weighted fitting: using the least squares method to fit the selected neighborhood points, this is achieved by fitting a local plane or surface;

[0050] Step 3.3.3, calculate the normal: based on the surface fitted in step 3.3.2, for each point P, find its K nearest neighboring points, calculate the covariance matrix between point P and each point in its neighborhood, perform eigenvalue decomposition on the covariance matrix, and obtain its eigenvectors and eigenvalues. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is ​​the normal direction of point P;

[0051] Step 3.3.4, iterative optimization: Through iteration, the fitting parameters are continuously adjusted to optimize the surface model to make it smoother and better fit the original point cloud data.

[0052] Furthermore, step 3.4 specifically includes:

[0053] Step 3.4.1, find the hole boundary of the point cloud model fused in step 3.3, and extract the coordinates of the boundary points;

[0054] Step 3.4.2, fill the holes, perform triangulation analysis on the points adjacent to the hole boundary, or perform surface fitting on the missing parts by fitting the surface of the adjacent points;

[0055] Step 3.4.3, discretize the surface and randomly generate points inside the hole.

[0056] Furthermore, step four includes:

[0057] Step 4.1, spatial analysis: Based on the original terrain data or the previous terrain data and the DEM data generated in step 3.5, the change of sand and gravel before and after sand mining is obtained through multi-temporal comparison analysis;

[0058] Step 4.2, data rendering: Based on the WebGL standard 3D scene rendering technology, with geographic information 3D interactive visual service as the core, the geographic spatial data involved in step 4.1 is rendered in a progressive and refined manner on map tiles;

[0059] Step 4.3, dynamic presentation: Based on the data rendering in step 4.2, the time dimension is introduced to dynamically present the topographic data of the sand mining area at different periods, and to intuitively and vividly present the process of changes in sand mining volume over time.

[0060] The present invention proposes a multimodal integrated method for rapid acquisition and analysis of river sand mining data. The method integrates multimodal monitoring means to form a standard business process, including: sand mining range identification, rapid data acquisition, intelligent fusion calculation and spatial analysis visualization. By integrating multimodal technologies and equipment such as image recognition, unmanned ships and drones into river sand mining management, a digital elevation model of the terrain in the sand mining area is obtained, the terrain difference before and after sand mining and the sediment supply are calculated, the sand mining volume is obtained, and a visualization is performed to form an integrated business process closed loop as a whole, thereby improving the efficiency and accuracy of river sand mining management and solving the problem of quantitative assessment and supervision of over-exploitation of mineable areas during sand mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of a multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of sand mining range identification according to an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of intelligent recognition of the UNet algorithm according to an embodiment of the present invention;

[0064] Figure 4 A schematic diagram of rapid data acquisition according to an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of intelligent fusion calculation according to an embodiment of the present invention;

[0066] Figure 6 A schematic diagram of intelligent fusion computing data preprocessing according to an embodiment of the present invention;

[0067] Figure 7 A schematic diagram of intelligent fusion calculation data registration according to an embodiment of the present invention;

[0068] Figure 8 A schematic diagram of intelligent fusion computing data fusion according to an embodiment of the present invention;

[0069] Fig. 9 This is a schematic diagram of post-processing of intelligent fusion calculation data according to an embodiment of the present invention;

[0070] Fig.10 A schematic diagram of a spatial analysis visualization of an embodiment of the present invention;

[0071] Fig.11 It is a schematic diagram of the overall effect of an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] like Figure 1 The embodiment of the present invention provides a multi-modal integrated river sand mining data rapid acquisition and analysis calculation method, which uses a sand mining range identification module 10, a data rapid acquisition module 20, an intelligent fusion calculation module 30 and a spatial analysis visualization module 40. The method includes the following steps:

[0074] Step 1: The sand mining range identification module 10 performs image recognition training on high-resolution satellite images based on a deep learning algorithm to extract river sand mining range data, and inputs the extracted river sand mining range data into the data rapid acquisition module 20 and the intelligent fusion calculation module 30. Sand mining range identification includes feature labeling and intelligent identification. The processing flow is as follows: Figure 2 , the detailed implementation steps are described as follows:

[0075] Step 1.1: The sand mining range identification module 10 obtains high-resolution satellite image data of the operation area and manually marks the sand mining area. The embodiment of the present invention uses QGIS to mark sand and gravel features, and the detailed implementation steps are described as follows:

[0076] Step 1.1.1, image preprocessing: process the periodic noise of the high-resolution satellite image data of the work area, and then perform orthorectification to correct the image tilt deviation and the error generated in the projection process; perform image enhancement on the high-resolution satellite image data after orthorectification to make the ground feature information more readable and highlight the key targets; finally, adjust the color balance of the image to achieve a better effect in color;

[0077] Step 1.1.2, feature annotation: To ensure the accuracy of the samples, the embodiment of the present invention collects a sandstone feature sample set on the remote sensing image processed in step 1.1.1 by manual annotation.

[0078] Step 1.2: Intelligent identification. The present invention uses the UNet algorithm to perform deep learning on the annotated high-resolution satellite images to extract the sand mining feature range. The processing process is as follows: Figure 3 ,The detailed implementation steps are described as follows.

[0079] Step 1.2.1, U-Net architecture design: By comparing high-resolution satellite remote sensing images from different periods, the U-Net network architecture and related training parameters are designed;

[0080] Step 1.2.2, sample training: select and preprocess the feature samples obtained in step 1.1.2 to obtain training samples, and input the training samples into the U-Net convolutional neural network designed in step 1.2.1 for training to obtain a trained constant residual U-Net deep neural network;

[0081] Step 1.2.3, sand mining range extraction: The remote sensing image processed in step 1.1.1 is divided into blocks and input into the network trained in step 1.2.2, and then spliced ​​to obtain the sand and gravel extraction probability map. Finally, the grayscale threshold is set to binarize the image to obtain the river sand mining range data.

[0082] Step 2: The data rapid acquisition module 20 receives the river channel sand mining range data input by the sand mining range identification module 10, automatically plans the water and underwater data collection route, and realizes the rapid acquisition of the water and underwater terrain data of the sand mining area through the water and land two-phase multi-module sensor, and inputs the acquired water and underwater terrain data into the intelligent fusion calculation module 30. The data rapid acquisition process is as follows: Figure 4 , the detailed implementation steps are described as follows:

[0083] Step 2.1, coordination and integration of multi-modules of land and water phases: To adapt to the river terrain and working conditions in different regions, the measurement accuracy and data integrity are improved by coordinating and integrating multi-module sensors of land and water phases. Specifically, the time of each sensor is continuously adjusted using the GPS time and 1PPS, so that the time of each sensor is always synchronized with the GPS time; the deviation between the coordinate system of each sensor and the carrier coordinate system is determined, and the coordinates of each sensor are parallel or coincident with the carrier coordinate system through calibration by the data acquisition software.

[0084] Step 2.2, data collection: automatically plan the survey line according to the sand mining range obtained in step 1.2, control each measurement subsystem to carry out independent data collection work for each sensor, and obtain the above-water and underwater terrain data.

[0085] Step 2.3, automatic data preprocessing and rapid transmission: Automatically check, process and filter the data collected in step 2.2, delete unqualified data and retain useful data. Rapidly transmit the water and underwater terrain data collected in real time in step 2.2 to the intelligent fusion computing module through Ethernet, 4G / 5G, long-distance transmission or narrowband Internet of Things communication technology.

[0086] Step 3: The intelligent fusion calculation module 30 receives the river channel sand mining range data input by the sand mining range identification module 10 and the water surface and underwater terrain data input by the data rapid acquisition module 20, and realizes the automatic seamless splicing of the water surface and underwater terrain data through the intelligent fusion algorithm. Figure 5 , the detailed implementation steps are described as follows:

[0087] Step 3.1: Data preprocessing. Analyze and process the surface and underwater terrain data transmitted in step 2.3 to generate point cloud data, and perform denoising and filtering on the point cloud data to improve the quality and accuracy of the data. Figure 6 , the detailed implementation steps are described as follows:

[0088] Step 3.1.1, analyzing and processing the above-water and underwater terrain data to generate point cloud data;

[0089] Step 3.1.2, analyze the distance between the point cloud noise points and the target point cloud in space, and divide the point cloud noise into large-scale outlier noise points and small-scale fluctuation noise points;

[0090] Step 3.1.3, remove large-scale outlier noise points. Considering the characteristics of outliers, this example uses a statistical filtering algorithm combined with a geometric filtering algorithm to remove large-scale noise data. The implementation process is as follows:

[0091] For any point p i , the average distance within its k-neighborhood for:

[0092]

[0093] Among them, d ij Indicates p i With neighboring point p j The distance between.

[0094] Assuming that the average distance follows a normal distribution, the mean μ and standard deviation σ of the average distance are calculated as follows:

[0095]

[0096] Where N is the number of point clouds.

[0097] According to the mean and standard deviation, the standard range of average distance can be calculated: (μ-std·σ,μ+std·σ). Among them, std can control the size of the average distance threshold. When the average distance of the sampling point is greater than this range, it is regarded as an outlier and removed.

[0098] Step 3.1.4, smoothing small-scale fluctuation noise;

[0099] Step 3.2, data registration. Based on the point cloud registration algorithm, the point cloud data from different perspectives or at different times processed in step 3.1 are registered to obtain accurate spatial position and posture information, ensuring the consistency of spatiotemporal reference and accuracy of multi-source point cloud data. This example combines the random sampling consensus algorithm and the nearest point iteration algorithm to register the point cloud. The process is as follows: Figure 7 , the detailed implementation steps are described as follows:

[0100] Step 3.2.1, perform rough matching on the point cloud data processed in step 3.1. Use the RANSAC (RANdomSAmple Consensus) algorithm for iterative sampling to obtain a relatively ideal transformation matrix; use the obtained transformation matrix to perform point cloud transformation operations to align two point clouds at different positions as much as possible.

[0101] Step 3.2.2, perform precise matching on the point cloud data after the rough matching in step 3.2.1. Use the ICP (Iterative Closest Point) algorithm to solve the rotation and translation transformation matrix of the corresponding points (the points closest to each other in the two point clouds); use the obtained rotation and translation matrix to perform spatial transformation of the point cloud to obtain the registered multi-source point cloud data.

[0102] Step 3.3, data fusion. Based on the surface reconstruction algorithm MLS (Mobile Least Square), the multi-source point cloud data registered in step 3.2 is fused, and the discrete point cloud data is converted into a continuous surface representation through least squares fitting, thereby generating a more complete point cloud model. The fusion process is as follows Figure 8 , the detailed implementation steps are described as follows:

[0103] Step 3.3.1, define the neighborhood. With each point as the center, determine a radius and select the neighborhood points within the radius. These neighborhood points will be used in the subsequent fitting process.

[0104] Step 3.3.2, weighted fitting. The selected neighborhood points are fitted using the least squares method, usually by fitting a local plane or surface. The purpose of this step is to reduce noise and estimate the normal direction on the surface by local weighted averaging.

[0105] Step 3.3.3, calculate the normal direction. If the normal direction needs to be estimated, it can be calculated simultaneously during the fitting process. This usually involves calculating the normal of the fitted surface at each point, which is very important for subsequent surface analysis and processing. Specifically, based on the surface fitted in step 3.3.2, for each point P, find its K nearest neighbors, calculate the covariance matrix between point P and each point in its neighborhood, perform eigenvalue decomposition on the covariance matrix, and obtain its eigenvectors and eigenvalues. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is ​​the normal direction of point P.

[0106] Step 3.3.4, iterative optimization. Through iteration, the fitting parameters are continuously adjusted to optimize the surface model to make it smoother and better fit the original point cloud data.

[0107] Step 3.4, data post-processing. The grid-based hole repair algorithm repairs the point cloud data fused in step 3.3 to obtain a more ideal point cloud model. The processing flow is as follows: Fig. 9 , the detailed implementation steps are described as follows:

[0108] Step 3.4.1, find the hole boundary of the point cloud model and extract the coordinates of the boundary points;

[0109] Step 3.4.2, fill the holes, perform triangulation analysis on the points adjacent to the hole boundary, or perform surface fitting on the missing parts by fitting the surface of the adjacent points;

[0110] Step 3.4.3, discretize the surface and randomly generate points inside the hole.

[0111] This example uses multi-harmonic radial basis functions (RBF) to perform mesh surface fitting repair on the point cloud model. The function form is:

[0112]

[0113] Where p is a low-order polynomial; the basic function is a real function on [0,∞), usually unbounded and non-compact, x i is the center point of the RBF.

[0114] Step 3.5, DEM data generation: resample the point cloud data processed in step 3.4 and generate a three-dimensional river model and a digital elevation model DEM.

[0115] Step 4: The spatial analysis visualization module 40 receives the above-water and underwater terrain data input by the intelligent fusion calculation module 30, obtains the change in sand and gravel before and after sand mining through multi-time comparison analysis, and renders and visualizes the analysis results using GIS and virtual reality technology. Fig.10 , the detailed implementation steps are described as follows:

[0116] Step 4.1, spatial analysis: Based on the original terrain data (or the terrain data of the previous period) and the DEM data generated in step 3.5, the change of sand and gravel before and after sand mining is obtained through multi-temporal comparison analysis;

[0117] Step 4.2, data rendering: Based on the WebGL standard 3D scene rendering technology, with geographic information 3D interactive visual service as the core, the geographic spatial data involved in step 4.1 is rendered in a progressive and refined manner on map tiles;

[0118] Step 4.3, dynamic presentation: Based on the data rendering in step 4.2, the time dimension is introduced to dynamically present the topographic data of the sand mining area at different periods, and to intuitively and vividly present the process of changes in sand mining volume over time.

[0119] The present invention integrates multimodal technologies and equipment such as image recognition, unmanned boats and drones into river sand mining management to obtain a digital elevation model of the terrain in the sand mining area, calculate the terrain difference before and after sand mining and the amount of sediment replenishment, obtain the sand mining volume, and present it visually. The implementation effect is as follows: Fig.11 As shown, an integrated business process closed loop is formed as a whole, thereby improving the efficiency and accuracy of river sand mining management and solving the problem of quantitative assessment and supervision of over-exploitation of mineable areas during the sand mining process.

[0120] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-modal integrated river sand mining data rapid acquisition and analysis calculation method, characterized in that: A sand mining range identification module (10), a data rapid acquisition module (20), an intelligent fusion calculation module (30) and a spatial analysis visualization module (40) are used; the method comprises the following steps: Step 1: the sand mining range identification module (10) performs image recognition training on high-resolution satellite image data of the operation area based on a deep learning algorithm, extracts river sand mining range data, and inputs the extracted river sand mining range data into the data rapid acquisition module (20) and the intelligent fusion calculation module (30); Step 2, the data rapid acquisition module (20) receives the river channel sand mining range data input by the sand mining range identification module (10), automatically plans the surface and underwater data acquisition route according to the river channel sand mining range data, realizes the rapid acquisition of surface and underwater terrain data of the sand mining area through the water and land two-phase multi-module sensor, and inputs the acquired surface and underwater terrain data into the intelligent fusion calculation module (30); Step 3, the intelligent fusion calculation module (30) receives the water surface and underwater terrain data input by the data rapid acquisition module (20), and realizes automatic seamless splicing of the water surface and underwater terrain data through an intelligent fusion algorithm to obtain fused water surface and underwater terrain data; Step 4: The spatial analysis visualization module (40) receives the fused water and underwater terrain data input by the intelligent fusion calculation module (30) to construct a three-dimensional model of the river channel and a digital elevation model, obtains the change in sand and gravel before and after sand mining through multi-time comparison analysis, and uses GIS and virtual reality technology to render and visualize the analysis results.

2. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 1 is characterized in that: The step one comprises: Step 1.1: The sand mining range identification module (10) obtains high-resolution satellite image data of the operation area and manually marks the sand mining area; Step 1.2: Use the UNet algorithm to perform deep learning on the annotated high-resolution satellite image data to extract the river channel sand mining range and sand mining feature range data, and then input the extracted river channel sand mining range data into the data rapid acquisition module (20) and the intelligent fusion calculation module (30).

3. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 2 is characterized in that: Step 1.1 specifically includes: Step 1.1.1, image preprocessing: process the periodic noise of the acquired high-resolution satellite image data of the operating area, and then perform orthorectification to correct the image tilt deviation and the error generated during the projection process; perform image enhancement on the orthorectified high-resolution satellite image data; and finally adjust the color balance of the image; Step 1.1.2, feature annotation: collect a sand and gravel feature sample set on the remote sensing image processed in step 1.1.1 by manual annotation.

4. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 3 is characterized in that: Step 1.2 specifically includes: Step 1.2.1, U-Net architecture design: By comparing high-resolution satellite remote sensing images from different periods, the U-Net network architecture and related training parameters are designed to obtain the U-Net convolutional neural network; Step 1.2.2, sample training: select and preprocess the sand and gravel feature sample set obtained in step 1.1.2 to obtain training samples, and input the training samples into the U-Net convolutional neural network designed in step 1.2.1 for training to obtain a trained constant residual type U-Net deep neural network; Step 1.2.3, sand mining range extraction: The remote sensing image processed in step 1.1.1 is divided into blocks and input into the constant residual U-Net deep neural network trained in step 1.2.2, and then spliced ​​to obtain the sand and gravel extraction probability map. Finally, the grayscale threshold is set to binarize the image to obtain the river sand mining range data.

5. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 1 is characterized in that: Step 2 includes: Step 2.1, coordination and integration of multi-module sensors for both land and water phases: Use GPS time and 1PPS to continuously adjust the time of each sensor so that the time of each sensor is always synchronized with the GPS time; determine the deviation between the coordinate system of each sensor and the coordinate system of the carrier, and calibrate through data acquisition software to ensure that the coordinates of each sensor and the coordinate system of the carrier are parallel or coincident; Step 2.2, data collection: automatically plan the survey line according to the sand mining range obtained in step 1.2, control each measurement subsystem to carry out independent data collection work for each sensor, and obtain the above-water and underwater terrain data; Step 2.3, automatic data preprocessing and rapid transmission: automatically check, process and filter the surface and underwater terrain data collected in step 2.2, delete unqualified data, retain useful data, and transmit the surface and underwater terrain data collected in real time in step 2.2 to the intelligent fusion computing module (30) via Ethernet, 4G / 5G, long-distance transmission or narrowband Internet of Things communication technology.

6. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 1 is characterized in that: Step three includes: Step 3.1, data preprocessing: Analyze and process the surface and underwater terrain data transmitted in step 2.3 to generate point cloud data, and perform denoising and filtering on the point cloud data to improve the quality and accuracy of the data; Step 3.2, data registration: Based on the point cloud registration algorithm, the point cloud data of different perspectives or at different times processed in step 3.1 are registered to obtain accurate spatial position and posture information, ensuring the consistency of spatiotemporal reference and accuracy of multi-source point cloud data; Step 3.3, data fusion: Based on the surface reconstruction algorithm MLS, the multi-source point cloud data registered in step 3.2 are fused and processed, and the discrete point cloud data are converted into a continuous surface representation through least squares fitting, thereby generating a more complete point cloud model; Step 3.4, data post-processing: The grid-based hole repair algorithm is used to perform model repair on the point cloud data fused in step 3.3; Step 3.5, generation of river channel three-dimensional model and digital elevation model DEM: resample the point cloud data processed in step 3.4 and generate the river channel three-dimensional model and digital elevation model.

7. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 6 is characterized in that: Step 3.1 specifically includes: Step 3.1.1, analyzing and processing the above-water and underwater terrain data to generate point cloud data; Step 3.1.2, analyzing the spatial distance between the point cloud noise points and the target point cloud in the point cloud data, and dividing the point cloud noise points into large-scale outlier noise points and small-scale fluctuation noise points; Step 3.1.3, remove large-scale outlier noise: For any point pi, the average distance within its k neighborhood for: Among them, d ij Indicates p i With neighboring point p j The distance between Assuming that the average distance follows a normal distribution, the mean μ and standard deviation σ of the average distance are calculated as follows: Where N is the number of point clouds; The standard range of average distance is calculated based on the mean and standard deviation: (μ-std·σ, μ+std·σ), where std is used to control the size of the average distance threshold. When the average distance of a sampling point is greater than this range, it is considered an outlier and removed. Step 3.1.4, smooth small-scale fluctuation noise.

8. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 6 is characterized in that: Step 3.2 specifically includes: Step 3.2.1, perform rough matching on the point cloud data processed in step 3.1: use the random sampling consistent RANSAC algorithm to perform iterative sampling to obtain the transformation matrix; use the obtained transformation matrix to perform point cloud transformation operations to align two point clouds at different positions; Step 3.2.2, perform fine matching on the point cloud data after rough matching in step 3.2.1: use the nearest point iterative ICP algorithm to solve the rotation and translation transformation matrix of the corresponding points, where the corresponding points are the points closest to each other in the two point clouds; use the obtained rotation and translation matrix to perform spatial transformation of the point cloud to obtain the registered multi-source point cloud data.

9. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 6 is characterized in that: Step 3.3 specifically includes: Step 3.3.1, define the neighborhood: Based on the multi-source point cloud data registered in step 3.2, determine a radius with each point as the center, and select the neighborhood points within the radius; Step 3.3.2, weighted fitting: using the least squares method to fit the selected neighborhood points, this is achieved by fitting a local plane or surface; Step 3.3.3, calculate the normal: based on the surface fitted in step 3.3.2, for each point P, find its K nearest neighboring points, calculate the covariance matrix between point P and each point in its neighborhood, perform eigenvalue decomposition on the covariance matrix, and obtain its eigenvectors and eigenvalues. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is ​​the normal direction of point P; Step 3.3.4, iterative optimization: Through iteration, the fitting parameters are continuously adjusted to optimize the surface model to make it smoother and better fit the original point cloud data.

10. The multi-modal integrated river sand mining data rapid acquisition and analysis calculation method according to claim 6 is characterized in that: Step 3.4 specifically includes: Step 3.4.1, find the hole boundary of the point cloud model fused in step 3.3, and extract the coordinates of the boundary points; Step 3.4.2, fill the holes, perform triangulation analysis on the points adjacent to the hole boundary, or perform surface fitting on the missing parts by fitting the surface of the adjacent points; Step 3.4.3, discretize the surface and randomly generate points inside the hole.

Images